In this blog for our Understanding Evidence series, Selena Ryan-Vig, Cochrane UK’s Knowledge and Engagement Officer, highlights some important considerations when reading research by looking at Cochrane evidence on cocoa and blood pressure
At first glance, a recently updated Cochrane Review exploring the effect of cocoa on blood pressure (Reid, Fakler & Stocks, 2017) may come as happy reading for chocolate lovers (well, lovers of flavanol-rich chocolate, specifically). The review is also an interesting case study. It provides food for thought about several issues we need to be aware of when we’re making sense of research in general and the types of claims that can be drawn from the evidence available to us. Before we get to that, let’s look at the review.
Interest in exploring the medicinal potential of chocolate comes from the fact that cocoa products contain flavanols. These are plant-based nutrients, thought to lower blood pressure by widening the blood vessels.
This review updated a previous version published in 2012, and included 17 more studies. This gave a total of 35 studies involving 1804 (mainly healthy) participants. Every day for at least two weeks, intervention group participants consumed flavanol-rich chocolate (30 to 1218mg of flavanols in 1.4 to 105 grams of chocolate). Control group participants consumed either low-flavanol-containing chocolate or a flavanol-free product.
There was moderate-quality evidence that, compared with control participants, individuals consuming flavanol-rich chocolate experienced a small, statistically significant reduction in blood pressure (1.8mmHg for both systolic and diastolic blood pressure).
Statistically significant, but is it clinically important?
Before we all dash off to stock up on flavanol-rich chocolate (or indeed any intervention which has been demonstrated to lead to a statistically significant benefit) it’s important to pause and think. The small reduction in blood pressure was statistically significant, but is it clinically important?
Let’s consider the size of the reduction in blood pressure. For context, NHS Choices (2016) states:
- high blood pressure (systolic blood pressure/diastolic blood pressure) is considered to be 140/90 mmHg or higher
- ideal blood pressure is considered to be between 90/60mmHg and 120/80mmHg
So the observed reduction (1.8mmHg) is indeed modest, but the review authors suggest even this may be clinically important. Consuming flavanol-rich chocolate, they propose, might complement other treatment options and could, at a population level, lead to fewer cardiovascular events. At this stage however, while this sounds plausible and promising, it remains speculative. This is largely owing to this next point…
The focus, and short-term nature, of the research leaves us with unanswered questions…
As the review authors allude to, what we’d really like to know is whether flavanol-rich chocolate could have beneficial effects not just on blood pressure but on the long-term consequences of high blood pressure. Could it reduce the incidence of heart attacks or stroke for instance? Unfortunately, we just don’t have the data to say one way or the other.
This is because the included studies: a) are of short duration (most lasted between two and 12 weeks, with only one lasting 18 weeks) and b) did not look at important outcomes such as heart disease or stroke. So we’re left in the dark about long-term effects.
This speaks to an issue affecting research more generally: often studies do not focus on the outcomes we’re really interested in due to reasons of cost and convenience. Events such as heart attacks and strokes are relatively rare. As such, we’d need studies with very large sample sizes, and/or spanning many years, to capture enough of these events to be able to determine whether there is a meaningful difference in the rate of their occurrence among those receiving intervention X vs. intervention Y. So instead research often focuses on ‘surrogate outcomes’, such as blood pressure, as a substitute for these important clinical outcomes or endpoints. While this is not unreasonable, we can’t just assume that beneficial effects on these surrogate outcomes will necessarily translate into real health benefits felt by patients.
Additionally, without long-term trials, we cannot be sure whether there are side effects associated with long-term daily consumption of cocoa products. In the short-term it seems people are happy to take their chocolatey medicine. Side effects, including “dislike of the trial product” and “digestive complaints” were reported by only 1% of the intervention participants. But longer term would individuals get sick of daily chocolate consumption (literally or figuratively)?
When the individual studies paint a varied picture…
Another issue is that the studies were highly statistically heterogeneous. This means that some studies suggested considerable benefits of cocoa on blood pressure, whereas others suggested no benefit.
The authors struggled to account fully for this variation, so much so that they had to rate the overall quality of the evidence as ‘moderate’ rather than ‘high’. Nonetheless, they did identify a number of possible explanations. There were tentative indications that younger individuals (18-49 years) tended to experience greater reductions in blood pressure than older individuals (50-73 years) as did those who had higher blood pressure at baseline.
A couple of other explanations for the heterogeneity are worth highlighting too, as these are issues which crop up time and time again to undermine a variety of healthcare research…
Beware of poorly blinded studies
There was a trend towards participants experiencing greater benefits when they were not “blinded” i.e. they were aware that they were consuming flavanol-rich chocolate or not. Further research is needed to confirm whether expectations (e.g. “dark chocolate is good for me”) can influence blood pressure. Nonetheless, this serves as an important reminder about the need for caution when drawing inferences from poorly blinded studies.
Beware of industry-sponsored research
It’s known that, across a spectrum of health research, industry-funded studies significantly more often favour the sponsors’ products than non-industry-sponsored studies (Lundh et al. 2017). This can lead us to overestimate the benefits, and underestimate the harms of interventions.
Indeed this may be at play here. The review authors explored what would happen when they removed six studies authored by individuals sponsored by industry. Excluding them “revealed a marked difference in results”, making the benefits of flavanol-rich chocolate seem less promising. So much so that the reduction in systolic blood pressure became non-significant. This is an important reminder to remain sceptical of claims made by those with conflicting interests.
So it seems that – as is so often the case with Cochrane evidence – the review’s take-home message is more nuanced than we might have hoped. Important questions remain to be answered, ideally in long-term, well-blinded, non-industry sponsored studies. I wonder whether there are any currently recruiting participants. I suppose I’m willing to tuck into some chocolate in the name of medical research. If you’d like to do the same (well, it might not be chocolate), you can find a list of ongoing trials recruiting participants here: https://www.ukctg.nihr.ac.uk/
Selena Ryan-Vig has nothing to disclose.
References may be found here.