Cochrane UK undertook some qualitative work to explore what the comments posted on Evidently Cochrane blogs might tell us about what readers get from our blogs. In this blog, the Editors of Evidently Cochrane, Sarah Chapman and Selena Ryan-Vig, and qualitative researcher Fran Toye, talk about the methods they used, the insights they gained, and the subsequent changes made to the blogs to make them more useful to readers.
Page updated 2 May 2021 and last checked 3 July 2023
Evidently Cochrane is Cochrane UK’s blog site, where we share blogs which aim to make Cochrane evidence really accessible and encourage discussion about it, to help inform people’s health choices. What do readers get from our blogs and how can we make them more useful? We’ve been mulling this for a while. While we can capture things like the number of page views and time spent on the site, it’s hard to put flesh on the bones of these website statistics. Mostly, we never know what value, if any, someone has derived from the blogs. We don’t know if a blog has informed someone’s health decision, left them with a new insight – perhaps from reading a personal or professional reflection in a blog, or helped them find their way to information that could be useful to them.
Sometimes, we get a glimpse of this in the comments that people write. While many blogs receive no comments, some continue to get many long after publication. (Evidently Cochrane is perhaps unusual for having blogs that are updated, something we do to ensure they reflect the most recent evidence and remain useful to readers).
We are often moved by the personal experiences shared in the comments and also see that many people who comment are looking for further information and guidance. We recognised the potential for the comments to give us some insights into how people experience the blogs and how we might make them more useful. We used qualitative research methods to see what we could learn from the comments.
What we did
We chose the five blogs on Evidently Cochrane which had received the most comments, as of November 2020. These focused on menopause, frozen shoulder, stillbirth and subsequent pregnancy, care after major stroke, and tubal flushing to increase the chance of conceiving. Four of these included the blogger’s relevant personal experience and one included data from in-depth interviews with patients and family members. The blogs were published between 2015 and 2019. We copied all comments posted on those blogs from the date they were first published until 6th November 2020. These comments were the data we used for the analysis.
Ensuring confidentiality
Although the comments are in the public domain, we wanted to be respectful of people’s choice to post comments, which they did without expecting them to be analysed. We sought advice from the University of Oxford ethics committee, and took the following measures to ensure confidentiality:
- we removed identifying information such as geographical location or names
- we rephrased what people had written, whilst taking care to retain original meaning, so that it would not be possible to return to an individual comment from the details within this report
- we uploaded the anonymised data (the rewritten comments) to Nvivo 11, software to facilitate qualitative analysis
We also sought permission from each blogger and invited them to take part in the analysis of the data.
How we analysed the data
There were 463 comments from the five blogs, almost 59,000 words in all. The comments from each blog were analysed separately first of all. Fran ‘coded’ the data – distilling the meaning of each bit of narrative into a few words, then organised these ‘codes’ into themes and overarching categories. Selena and Sarah independently organised the codes and themes and helped to refine them. Fran wrote a sentence encapsulating the meaning of each. All the bloggers were invited to contribute to the analysis at this point and we incorporated their suggestions into the content and wording of the final themes. This process of collaboration contributed to the trustworthiness of our findings.
What we found
Figure 1 shows the codes, themes and categories that we developed to help us understand people’s use of Evidently Cochrane blogs. The themes and categories were derived from comments on all the blogs and show that the people posting comments about different health experiences share a great deal, despite, on the face of it, their very different experiences. The three overarching categories are (1) Negotiating ambiguity and uncertainty; (2) Testament to my suffering; (3) Needing and finding solidarity (figure 1).
The extracts below, which illustrate the themes, are not verbatim quotes from individuals. Rather, they are rewritten narratives that capture the essence of multiple comments. This allows us to protect the anonymity of the people who have posted comments.
Negotiating ambiguity and uncertainty
People in the comments shared ‘the agony and ambiguity of making and living with difficult decisions’.
For example:
Some whose relatives were living with the consequences of a major stroke described the agonising decision to switch to end-of-life care:
“The decision to end treatment was horrible. It was the toughest decision that we had ever made and we live with this decision. It breaks my heart. We keep questioning ourselves. My mind is screaming that I made the choice to let them die. I may question my decision for the rest of my life.”
Some who had experienced stillbirth described the difficult decision of trying for another child:
“I am crippled by fear for another pregnancy. I am haunted by my worries. I know the worst can happen. I am worried about leaving it too long. I feel guilty for wanting another baby. I am not looking for a replacement. The longing to be a mother fills my mind.”
Some also questioned, ‘why did this happen and could something different have been done?’
For example, some of those experiencing stillbirth asked what could possibly have gone wrong:
“It was an ordinary uncomplicated pregnancy, so what went wrong? Why did this happen? I am still trying to process it. I am full of unanswered questions. The doctors just said it can just happen. I might have to accommodate never knowing.”
Similarly, some who had lost relatives following stroke looked retrospectively for answers:
“I have so many questions. I can’t understand what is happening. I would like some advice if anyone has been through this. We still don’t know what happened to them. I am still puzzled. I want to understand if I did the right thing.”
Some people seemed to be ‘trying to decide what now’, seeking and offering experiential knowledge to help negotiate ambiguity.
For example:
Those considering tubal flushing reached out for experiential knowledge in the face of ambiguity:
“Please, I really need help. Is there anything I can do to help me get pregnant? Will tubal flushing work? I have read so many stories with different outcomes. What should I do next? Can I get pregnant? Can I unblock my tubes? What did you do? Did it work for you? I am so confused.”
Similarly, some of those experiencing menopause sought (“has anyone else had this problem?”) and exchanged information and experience (“I have tried this…”):
“I’ve been scouring the internet. Has anyone else had this? Is this common? Is this normal? Is hormone replacement the answer? Can someone recommend pain relief? Has anyone out there got any suggestions? Advice would be appreciated.”
“Herbal remedies. Yoga. Meditation. Exercise. I keep crackers handy for nausea. Avoid oily, sour or spicy food. Drink lots of water. Cut back on coffee. Antidepressants. HRT. Stay positive. Do things you enjoy. Make space for you. Say no. Find the right doctor.”
Testament to my suffering and loss
It appears that people felt compelled to write their story, using the blog as a space where they could safely voice their suffering, and that writing it down gave them some relief. They described ‘heartbreak, despair and desperation in the face of physical and emotional suffering’:
For example, those with frozen shoulder used the blog to describe the physical and mental toll of living with this condition:
“This pain is horrible. My arm is useless. Sudden or unexpected movements are excruciating. I try to keep my arm still. I can’t sleep. Normal things become a monumental task. I can’t get my clothes on, I can’t even flush the loo. I cry every day. It is so emotional. I don’t enjoy anything. It has a physical and mental hold on me. It is the hardest thing that I have had to deal with. It is exhausting. I want to start living again.”
People also described ‘profound and life altering grief’, adding further support for the blog as a space to testify suffering:
For example:
Those experiencing menopause described grieving for the person that they had once been:
“I can’t hold down a job or relationship. My life as I knew it is over. It has destroyed my self-esteem and confidence. I am trapped in an alien body. I don’t recognise myself. It has stolen my future and ruined my life. I have become moody, unhappy and unpleasant. I want to be me again.”
Those who had experienced stillbirth described the theft of motherhood:
“I left with empty arms. A mother should never lose a child. I became a mum that day but had no baby. It is so painful to lose a baby. The best day of my life became my worst. My arms are empty. I have milk and no baby. My breasts are a constant reminder of my loss.”
Needing and finding solidarity
Something that we often saw, in all of the blogs, was a feeling of shared suffering and solidarity. People often expressed that ‘we are in the same boat sharing our suffering’, describing the value of finding community.
For example:
Those considering tubal flushing expressed the idea that others commenting ‘could be writing about me’:
“I am going through this. I feel your pain. We are in the same boat. Are you pregnant yet? I will keep you posted. Any update yet? This sounds like my story. I feel for you.”
Similarly, some of those experiencing menopause described relief from knowing that others are in the same boat (“it is therapy to feel that I am not alone”):
“I could have written those identical words. It is like therapy to hear your words. You are describing me. I don’t feel alone reading this. I want to connect with others who are going through this. I thought it was all in my head. I felt I was on an island on my own before. I feel normal now. Thanks for being brave and sharing.”
Occasionally, people mentioned ‘feeling like an outsider: stigmatised, isolated and alone’.
For example:
Some of those experiencing menopause described the experience as ‘living alone on an island’. They did not talk about their experience for fear of judgement and the blog provided a safe space to share and to feel connected:
“I pull away from other people. I am alone on an island. No one to turn to. Menopause is silent. We don’t talk about it because we don’t want to be criticised. People think you are crazy. It is hard to explain to someone who has not experienced this. It is horrible to feel alone.”
What changes has this led to?
As a result of what we’ve learned, we’ve made some changes, hoping to make the blogs more useful. We have revised our guide to blogging for Evidently Cochrane, asking bloggers to signpost readers to sources of information and support, and to make suggestions that could help readers think about their treatment options and discuss them with their healthcare providers.
We have also reflected on our policy for moderating online comments. We remain committed to protecting bloggers and readers from offensive comments. At the same time, this work has shown the value of allowing readers to respond freely to each others’ shared experiences. At times, this may mean that readers will recommend treatments that are not supported by evidence. We have now revised the disclaimer in the blogs to clarify that we do not fact check comments or endorse treatments mentioned by readers.
Declaration of interest: Fran Toye has nothing to disclose. Sarah Chapman and Selena Ryan-Vig work for Cochrane UK and edit the Evidently Cochrane blogs. Sarah Chapman also authored two of the blogs included within the analysis (frozen shoulder and tubal flushing).
With thanks to June Girvin, Susannah Hopkins Leisher, Aleena Wojcieszek, Scott Murray and Marilyn Kendall.
References (pdf) and read the full report of the study (pdf)
