Teaching kids to assess goopy health claims

In this blog for our #UnderstandingEvidence series, Matt Oxman talks about the Informed Health Choices project, which helps people sort the wheat from the chaff of evidence about treatment effects, starting with school children.

Page checked 4 April 2023

Evidently Cochrane makes evidence accessible, but so does Gwyneth Paltrow on her blog Goop. The difference is in the quality of evidence. However, if most people cannot tell apart a systematic review from Paltrow’s anecdote about the benefits of “bee venom therapy”, where do you think they will go for their health advice? Cochrane certainly cannot compete in terms of celebrity.

“I was recently given ‘bee venom therapy’ for an old injury and it disappeared.” ~ Gwyneth Paltrow

Before the will to access high-quality evidence must come appreciation of what makes high-quality evidence. Without this understanding, people are prone to be misled by unreliable health claims and make uninformed choices, leading to waste and unnecessary suffering much more serious than bee stings.

This is the problem that my colleagues and I—mostly based in East Africa and Norway, but also the UK and other countries—are addressing with the Informed Health Choices project. We are developing and evaluating learning resources to help people separate the wheat from the chaff of evidence about treatment effects, starting in primary schools.

Matt Oxman/Informed Health Choices project
The teacher, in red, directs his class in Rwanda, where we pilot-tested an early version of the primary school resources.
(Matt Oxman/Informed Health Choices project)

In 2014, we began developing the first resources, targeting Ugandan 10-year-olds and their parents, respectively. For the kids, we have made a combination textbook and comic book, with supplemental materials including an exercise book and a guide for teachers using the materials in their classrooms.

For the parents, we have produced a podcast, including a theme song.

The resources cover concepts such as the difference between personal experiences (anecdotal evidence) and a large fair comparison (randomised trial). They also include the concept of weighing benefits versus harms—for example, living like a celebrity versus being stung by bees.

We targeted children because they have the time and open minds to learn. Adults, on the other hand are, busy. Moreover, they often have to unlearn misconceptions and give up beliefs tied to their identities.

We targeted Uganda not because Ugandans face more unreliable claims than Norwegians or Brits, and are less able to assess them, but because the less you have, the less you can afford to make uninformed health choices. In fact, science teachers at an international school in Norway have already made using our materials a permanent part of their students’ science education.

Matt Oxman/Informed Health Choices project
Children at an international school in Norway are using the final version of The Health Choices Book on tablet computers.
(Matt Oxman/Informed Health Choices project)

Recently, we tested the resources in two randomised trials, in Central Uganda. About 15 000 children participated in the trial of the primary school resources. Soon, we will find out whether the resources showed an effect on participants understanding and abilities to apply the concepts, measured by their answers to multiple-choice, scenario-based questions.

While we wait for the results, we have already started planning improvements to the resources and the development of new ones. Many of my colleagues have one foot in the project and another in Cochrane, and we are looking at ways of connecting the resources directly to Cochrane reviews in the form of interactive Summary of Findings tables. For more information about what we have done and plan to do, visit our website, where you can download the resources for free.

Finally, a warning: when I wrote about the project in a guest blog post for HealthNewsReview.com, I also used a celebrity claim as the main example. That celebrity was subsequently elected President of the most powerful country on Earth. Goop, by the way, means “a stupid person”. Paltrow’s blog appears to have been named without an Intention to be Ironic (ITI) analysis.

Matt Oxman has nothing to disclose.

Featured Image:  Children in Uganda show off the original prototype of what would become The Health Choices Book: Learning to think carefully about treatments.  (Matt Oxman/Informed Health Choices project).

Teaching kids to assess goopy health claims by Matt Oxman

is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International

2 Comments on this post

  1. […] Teaching kids to assess goopy health claims Matt Oxman, Evidently Cochrane […]

  2. An insightful read! Matt Oxman’s work with the Informed Health Choices project underscores the pressing need to educate both young minds and adults about discerning quality evidence in healthcare. It’s both alarming and fascinating to realize that celebrity anecdotes can overshadow robust scientific evidence in public perception. Initiatives like these not only empower individuals to make informed health decisions but also pave the way for a more scientifically literate society. Kudos to the entire team!

    Shafi Ahamed / Reply

Leave a Reply

Your email address will not be published. Required fields are marked *