Good Science, Bad Science

A great bit of equine-related research has hit my desk lately: A group at Washington State University quantified lowered cortisol levels in kids enrolled in equine-facilitated learning programs. The study, a randomized trial, gives EFL programs a huge leg up in legitimization. It should help stakeholders and insurance companies understand james2what us horse owners already appreciate: being around horses is good for you. Read about it here.

The report is getting some decent traction on the Internet. No more attention, however, than some seriously sketchy science promoted by the International Society of Equitation Science (ISES).

ISES released a study by Cecilie Mejdell and her colleagues at the Norwegian University of Life blanketing-preference-study-2Sciences that described how horses can choose for themselves whether or not to wear a blanket. The team trained 23 horses to read cue cards, essentially “asking” horses if they would like to wear a one.

I asked my vet about the research, she replied:

It sounds like a lot of assumptions were made – horses understanding consequences of their actions, for example.

  • What if the preference was for the symbol rather than the action afterwards?
  • How are they sure the message was communicated uniformly across the horses?
  • What was the state of the horses’ body temperature under the conditions?
  • What was the length/thickness of the hair coat at time of experience?
  • Where did these horses’ come from and what sort of conditions were they exposed to prior to the study?

pyr of evidenceAs a general rule, if a study begs more questions than it answers, something’s wrong.

My vet understood there’s a big difference between a proceedings journal (ISES) and published research (the EFL study). She also knows about the Gold Standard of Scientific Work and the Hierarchy of Evidence. Those principles state that meta analyses and randomized controlled trials are the best, most reliable and most respected kind of work.

The least reliable is expert opinion and case studies. ISES, though well-meaning in its efforts, promotes and disseminates mostly case studies. Read more about ISES.

That’s all well and good if you know how to review scientific studies. But what about the rest of us? Too often on the Internet, gold mixes with fool’s gold. Media outlets skim a study’s surface. If it’s fun and topical, they’ll give it good play. Readers on social media gobble it up without considering sources or fact checking.

The result? Lots of misinformation.

Want some tips on how to navigate through the junk to find the gems? Read tips for how to judge research here.

Read about getting caught reading junk here.

 

 

 

 

Posted in General and tagged .

0 Comments

  1. Several years ago a young relative would trailer to my barn wth her Standardbred for lessons. The horse stood nicely to be brushed but when we came with the saddle she was back forward and sideways as much as the crosstie allowed. One day we were riding western and when we came with the saddle she stood as quietly for that as she did to be brushed. Noticing that we got her English saddle. As we approached with it she was back forward and sideways. We switched her English sadde and had no more trouble with her not standing quietly. Today I ride an NSA. She is quiet and sensible but when I come with my endurance saddle (western style) she is all over the place. When I use my English saddle she stands quietly to be saddled. I feel I need to listen to the body language so I’m no longer using the endurance saddle.

Leave a Reply

Your email address will not be published. Required fields are marked *