An incomplete and misleading reading of Archie Cochrane

Archie Cochrane

Archie Cochrane

In this guest post, Iain Chalmers comments on a recent publication in which the authors claim that Cochrane’s 1972 book ‘Effectiveness and Efficiency’ was the inspiration for both the Cochrane Collaboration and the Evidence-Based Medicine movement, and that neither movement has paid sufficient attention to Cochrane’s reference to the importance of ‘care’ in health care.

The journal Medical Humanities has recently published an article entitled ‘Who cares? The lost legacy of Archie Cochrane’. It is co-authored by Askheim, Sandset and Engebretsen (2016), of the Institute of Health and Society and the Faculty of Humanities, University of Oslo, Norway. The authors believe they have contributed to “a more thorough reading” of Archie Cochrane’s book Effectiveness and Efficiency than has been done previously. In his book Cochrane had emphasized the need to assess ‘care’ as well as ‘cure’ when evaluating health services. As they believe that Cochrane’s book led to the Cochrane Collaboration and the Evidence-Based Medicine (EBM) movement, Askheim and colleagues take the Collaboration and EBM to task for paying insufficient attention to the need to assess ‘care’.

they appear to have taken a single document from the early 1970s and read it with 2016 eyes, ignoring the intervening four decades

Cochrane’s book was published nearly 45 years ago (Cochrane 1972). The analysis attempted by Askheim and colleagues is confused because they have not taken any account of Cochrane’s writings, including his autobiography (Cochrane 1989), between 1972 and his death in 1988. The authors might also have referred to material in the collection of essays published in 1997 to mark the 25th anniversary of the publication of Effectiveness and Efficiency (Maynard and Chalmers 1997). In brief, they appear to have taken a single document from the early 1970s and read it with 2016 eyes, ignoring the intervening four decades.

effective-care-iain-chalmersHad Askheim and colleagues consulted other sources published after 1972 they would have realised that, although the Cochrane Collaboration and Evidence-Based Medicine (EBM) have chronologically parallel histories, their origins and histories are distinct, and neither flowed directly from Effectiveness and Efficiency. The history of the Cochrane Collaboration did not start with Cochrane’s 1972 book, but rather with a very specific criticism and challenge issued by Cochrane in 1979, namely: “It is surely a great criticism of our profession that we have not organised a critical summary, by speciality and subspeciality, adapted periodically, of all randomised controlled trials” (Cochrane 1979).

Cochrane’s challenge was taken up initially by people involved in care during pregnancy and childbirth, which Cochrane had dubbed the least scientifically-based speciality within medicine. The year before he died he expressed his approval of this work (Cochrane 1987). In the Foreword published in the report of the project he wrote: “The systematic review of the randomized trials of obstetric practice that is presented in this book is a new achievement. It represents a real milestone in the history of randomized trials and in the evaluation of care, and I hope it will be widely copied by other medical specialties.” It was this ‘pilot experience’ that led to the creation of the first Cochrane Centre in 1992 (Chalmers et al. 1992), and to the Cochrane Collaboration the following year (Chalmers 1993).

“It is surely a great criticism of our profession that we have not organised a critical summary, by speciality and subspeciality, adapted periodically, of all randomised controlled trials” (Cochrane 1979)

Instead of drawing on these published sources to find out how Cochrane’s thinking evolved between 1972 and 1988, and to understand how, specifically, the Cochrane Collaboration responded to this, Askheim and colleagues unaccountably chose to take the writings of a literary theorist as the guide for their analysis. As a consequence, their reading of Cochrane has been less thorough than they had imagined, and their analysis inevitably muddled.

Just as Effectiveness and Efficiency was not the inspiration of the Cochrane Collaboration, so also was it not the inspiration for the ‘Evidence-Based Medicine’ movement. The EBM movement was inspired and led in the early 1990s by the writings and influence of David Sackett, Brian Haynes, Peter Tugwell, Gordon Guyatt and others at McMaster University. As someone who left clinical care in 1973 I have never felt qualified to write or speak about EBM, so I will leave it to those who have been integral to that history to comment on other misrepresentations in the article by Askheim and colleagues. The authors could have avoided some of their misunderstandings by interviewing some of the several people in Oslo who have contributed to one or both of these histories.

Askheim and colleagues claim that “care, which is harder to capture statistically, using purely quantitative techniques of epidemiology, is placed on the margins of what medicine is.” If they are seriously concerned about this, I urge them to look more carefully at efforts made both within the Cochrane Collaboration and in EBM to evaluate and improve care, using both qualitative and quantitative methods; and then to offer practical advice and illustrations of how these efforts might be improved.

Acknowledgements

I am grateful to Andy Oxman and Mike Clarke for helpful comments on an earlier draft.

Iain Chalmers has nothing to disclose.

References can be found here.

Related Post


Iain Chalmers

About Iain Chalmers

view all posts

Iain Chalmers, Coordinator, James Lind Initiative Iain Chalmers was founding director of the National Perinatal Epidemiology Unit (www.npeu.ox.ac.uk) between 1978 and 1992, and founding director of the UK Cochrane Centre (http://uk.cochrane.org/) between 1992 and 2002 (www.cochrane.nhs.uk). Since 2003, he has coordinated the James Lind Initiative. He edits The James Lind Library (www.jameslindlibrary.org) and Testing Treatments interactive English (www.testingtreatments.org), and he is a member of the Informed Health Choices team (www.informedhealthchoices.org). He contributed to the 2014 Lancet series on reducing waste and adding value in biomedical research, and he continues to lobby to reduce research waste (www.rewardalliance.net).

6 Comments on this post

  1. Reply to Iain Chalmers
    We would first like to thank Iain Chalmers for taking the time to comment on our article ‘Who cares? The lost legacy of Archie Cochrane’ (1). However, we believe that his reading of our paper is misunderstood and misleading, to the extent that we find it difficult to recognize our argument in his commentary.

    First of all, we do not claim that Cochrane’s book Effectiveness and efficiency (1972) ‘led to the Cochrane Collaboration’, or that Evidence-based medicine ‘flowed directly from Effectiveness and efficiency’. What we do claim, is that Effectiveness and Efficiency was an important step and inspiration in the creation of both EBM and the Cochrane Collaboration. This should be an uncontroversial assertion, based on several sources, including Chalmers and colleagues own 1992 article ‘Getting to grips with Archie Cochrane’s agenda’. (2) One does not have to read more than the first paragraph of Chalmer’s 1992-article to find the sentence: “Twenty years ago, in Effectiveness and Efficiency, Archie Cochrane emphasised the importance of randomised controlled trials in guiding decisions about health care.” This sentence also illustrates another claim we make in our paper, namely that Effectivity and effectiveness is often read as a book advocating the use of RCTs in health care. We show that this reading of Cochrane’s book is widespread, but also highly selective and tendentious.

    Secondly, as we state early in our article, our aim is to perform a close reading of Effectiveness and efficiency. We are not trying to ‘to find out how Cochrane’s thinking evolved between 1972 and 1988’ or how ‘the Cochrane Collaboration responded to this’. We know the sources, but that’s a different story, it’s not our story. It is difficult to answer criticism where the essence of the critique is that we should have written a totally different paper.

    Thirdly, we are not insulted by being accused of using a ‘literary theorist’ as a guide to our close reading of a text. Just as we would not react if someone dared using a theoretical physicist to explain the laws of gravity. Within the field of medical humanities the practice of close reading or of textual analysis, is a common method, just like RCTs is a common method when biomedical researchers try to figure out whether a new drug works or not.

    Our intention with the article was:
    1. to point out that Archie Cochrane is more than the Cochrane Collaboration and praise of RCTs. And that the most central aspects of Effectiveness and efficiency are usually ignored.
    2. to throw light on a potential and unintended consequence of this ignorance, namely the marginalization of the care dimension of medicine, in the work of those who claim to follow in Cochrane’s footsteps.
    3. to ask the question of whether the Cochrane Collaboration and proponents of EBM might have something to learn from this lost legacy of Archie Cochrane.

    These issues inspire several interesting debates, which Chalmers acknowledges at the end of his comment, where he urges us to “look more carefully at efforts made both within the Cochrane Collaboration and in EBM to evaluate and improve care, using both qualitative and quantitative methods; and then to offer practical advice and illustrations of how these efforts might be improved.” We would be happy to discuss these issues further, with the aim of contributing to the improvement of medical care in the broadest sense of the term. But we believe that critical perspectives are crucial to this debate, and that we need contributions from different people from different fields, and with different backgrounds.

    Clemet Askheim, Tony Sandset and Eivind Engebretsen

    References
    1 Askheim C, Sandset T and Engebretsen E. Who cares? The lost legacy of Archie Cochrane. BMJ 2016. (http://mh.bmj.com/content/early/2016/10/06/medhum-2016-011037.full)

    2 Chalmers I, Dickersin K and Chalmers TC. Getting to grips with Archie Cochrane’s agenda. BMJ 1992;305:786-7.
    (http://ac.els-cdn.com/S0140673695916105/1-s2.0-S0140673695916105-main.pdf?_tid=0195d9f8-99e7-11e6-b6dd-00000aacb361&acdnat=1477312992_4f273bcc3a131c515adbe85d72bd0d69)

    Clemet Askheim / Reply
  2. We would first like to thank Iain Chalmers for taking the time to comment on our article ‘Who cares? The lost legacy of Archie Cochrane’ (1). However, we believe that his reading of our paper is misunderstood and misleading, to the extent that we find it difficult to recognize our argument in his commentary.

    First of all, we do not claim that Cochrane’s book Effectiveness and efficiency (1972) ‘led to the Cochrane Collaboration’, or that Evidence-based medicine ‘flowed directly from Effectiveness and efficiency’. What we do claim, is that Effectiveness and Efficiency was an important step and inspiration in the creation of both EBM and the Cochrane Collaboration. This should be an uncontroversial assertion, based on several sources, including Chalmers and colleagues own 1992 article ‘Getting to grips with Archie Cochrane’s agenda’. (2) One does not have to read more than the first paragraph of Chalmer’s 1992-article to find the sentence: “Twenty years ago, in Effectiveness and Efficiency, Archie Cochrane emphasised the importance of randomised controlled trials in guiding decisions about health care.” This sentence also illustrates another claim we make in our paper, namely that Effectivity and effectiveness is often read as a book advocating the use of RCTs in health care. We show that this reading of Cochrane’s book is widespread, but also highly selective and tendentious.

    Secondly, as we state early in our article, our aim is to perform a close reading of Effectiveness and efficiency. We are not trying to ‘to find out how Cochrane’s thinking evolved between 1972 and 1988’ or how ‘the Cochrane Collaboration responded to this’. We know the sources, but that’s a different story, it’s not our story. It is difficult to answer criticism where the essence of the critique is that we should have written a totally different paper.

    Thirdly, we are not insulted by being accused of using a ‘literary theorist’ as a guide to our close reading of a text. Just as we would not react if someone dared using a theoretical physicist to explain the laws of gravity. Within the field of medical humanities the practice of close reading or of textual analysis, is a common method, just like RCTs is a common method when biomedical researchers try to figure out whether a new drug works or not.

    Our intention with the article was:
    1. to point out that Archie Cochrane is more than the Cochrane Collaboration and praise of RCTs. And that the most central aspects of Effectiveness and efficiency are usually ignored.
    2. to throw light on a potential and unintended consequence of this ignorance, namely the marginalization of the care dimension of medicine, in the work of those who claim to follow in Cochrane’s footsteps.
    3. to ask the question of whether the Cochrane Collaboration and proponents of EBM might have something to learn from this lost legacy of Archie Cochrane.

    These issues inspire several interesting debates, which Chalmers acknowledges at the end of his comment, where he urges us to “look more carefully at efforts made both within the Cochrane Collaboration and in EBM to evaluate and improve care, using both qualitative and quantitative methods; and then to offer practical advice and illustrations of how these efforts might be improved.” We would be happy to discuss these issues further, with the aim of contributing to the improvement of medical care in the broadest sense of the term. But we believe that critical perspectives are crucial to this debate, and that we need contributions from different people from different fields, and with different backgrounds.

    Clemet Askheim, Tony Sandset and Eivind Engebretsen

    References
    1 Askheim C, Sandset T and Engebretsen E. Who cares? The lost legacy of Archie Cochrane. BMJ 2016. (http://mh.bmj.com/content/early/2016/10/06/medhum-2016-011037.full)

    2 Chalmers I, Dickersin K and Chalmers TC. Getting to grips with Archie Cochrane’s agenda. BMJ 1992;305:786-7.
    (http://ac.els-cdn.com/S0140673695916105/1-s2.0-S0140673695916105-main.pdf?_tid=0195d9f8-99e7-11e6-b6dd-00000aacb361&acdnat=1477312992_4f273bcc3a131c515adbe85d72bd0d69)

    The link to the article by Chalmers and colleagues that we refer to in our reply, is wrong. The correct link is: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1883470/
    We are sorry about the mistake.

    Eivind Engebretsen / Reply
    • I am pleased that Askheim et al. have recognised that they did not provide a link to the 1992 BMJ article setting out a draft agenda (Chalmers et al. 1992) for dealing with the challenge that Cochrane had issued in 1979 (Cochrane 1979). Readers need to use this link to understand what this discussion is about: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1883470/.

      As readers will see, the 1992 paper opens with this quote from Cochrane’s 1979 paper:

      “It is surely a great criticism of our profession that we have not organised a critical summary, by speciality and subspecialty, adapted periodically, of all randomised controlled trials” (Cochrane 1979.

      Askheim et al. clearly need reminding that it was this specific challenge by Cochrane, not his 1972 book Effectiveness and Efficiency with its several challenges, which effectively provided the mission statement for the Cochrane Collaboration. Askheim et al. may wish that Cochrane’s challenge had not been formulated in these specific terms, but it was. Continuing to ignore this reality will not rescue their misleading analysis.

      I am pleased that Askheim et al. would be happy to discuss the issues I raised in my comments on their article. They state that “critical perspectives are crucial … and that we need contributions from different people from different fields, and with different backgrounds.” I could not agree with them more, but with an important proviso – that discussions should distinguish opinions based on evidence from opinions based on impressions and wishful thinking.

      Even though their article in Medical Humanities and their reply to my criticism ignores Cochrane’s very specific challenge to prepare and maintain ‘critical summaries… of all randomised controlled trials’, their paper alleges that the themes of ‘care’ and ‘equity’ in Cochrane’s Effectiveness and Efficiency had received insufficient attention in the Cochrane Collaboration. They made this allegation without any attempt to provide evidence to substantiate their judgement. Because their paper lacked evidence to support their conclusions, I urged them to “look more carefully at efforts made … within the Cochrane Collaboration … to evaluate and improve care, using both qualitative and quantitative methods; and then to offer practical advice and illustrations of how these efforts might be improved.”

      My commentary on their paper included an image of a 1500-page book about care during pregnancy and childbirth, as well as a reference to documents published at the inception of the Cochrane Collaboration illustrating the importance of care and quoting Cochrane (e.g. Chalmers 1993). A simple search of the Collaboration’s website today using the word ‘care’ retrieved seven Cochrane Review Groups and 125 reviews with titles including the word ‘care’. And the Cochrane Equity Methods Group is the focus of the other element of Cochrane’s 1972 agenda that Askheim et al. allege is being ignored.

      I urge Askheim, Sandset and Engebretsen to come down from their ivory tower and explore how they may be able to influence the Cochrane Collaboration in the ways they believe it needs to be influenced. The deficiencies in their article mean that their intentions in writing it are unlikely to be achieved among people who, although not uninterested in opinions, want them to be supported with evidence when relevant evidence is available.

      References

      Askheim C, Sandset T, Engebretsen E (2016). Who cares? The lost legacy of Archie Cochrane. Medical Humanities. Doi:101136/medhum-2016-011037.

      Chalmers I, Dickersin K, Chalmers TC (1992). Getting to grips with Archie Cochrane’s agenda: a register of all randomized controlled trials. BMJ 305:786-788.

      Chalmers I (1993). The Cochrane Collaboration: preparing, maintaining and disseminating systematic reviews of the effects of health care. In: Warren KS, Mosteller F, eds. Doing more good than harm: the evaluation of health care interventions. Annals of the New York Academy of Sciences 703:156-163.

      Cochrane AL (1972). Effectiveness and efficiency. Random reflections of health services. London: Nuffield Provincial Hospitals Trust.

      Cochrane AL (1979). 1931-1971: a critical review, with particular reference to the medical profession. In: Teeling-Smith G, ed. Medicines for the Year 2000. London: Office of Health Economics, pp 1-11.

      Iain Chalmers / (in reply to Eivind Engebretsen) Reply
      • Once again, we will like to thank Dr. Chalmers for engaging in this interesting discussion. In the first draft of our paper we had not been clear enough about Cochrane’s different uses of the word ‘care’. Thanks to useful comments by the journal’s anonymous reviewers, we added some clarifying remarks about the two basic meanings of ‘care’ used in Effectivity and effectiveness. Perhaps we were still not clear enough about this distinction. Cochrane distinguishes between care as the complement of cure, or therapy, and care as in ‘health care’ in general. His distinction follows the standard division between curative medicine (therapy) and “the ‘caring’ aspect of the profession”, as Cochrane puts it in his 1978 talk to the “Medicines for the year 2000”-symposium held in London (Cochrane 1979:7). Counting the number of hits the word ‘care’ generates on the Cochrane Collaboration website, is hardly the best method for evaluating whether “board and lodging and tender, loving ,care” (Cochrane 1972:3), is marginalized within the EBM-discourse. But we agree with Chalmers that a closer look at this question would be interesting.

        Our claim about the marginalization of care is first of all, based on the theoretical argument that the method of study comes to define the object of study, and that researchers are only able to see what they are looking for, with the tools they have at hand. If these tools do not capture what Cochrane defines as care, then we believe there is a logical consequence that the care aspect becomes largely invisible. But we ascribe no evil intent either to the part of the researchers, the research administrators, or the developers of the research policies. On the contrary, we believe this marginalization happens despite intentions of the opposite.

        Following the same line of reasoning, one might argue that Chalmers treats our claim about the marginalization of care in the same way as EBM handles care: he simplifies the claim to the extent that it can be refuted with the methods he has at his disposal, in this case a web-search. This is what we, following Bruno Latour (1993), refer to as ‘purification’, and it leads to a reduction of the original issue. Cochrane’s ‘care’ as the complement of cure (therapy), is not the same as health care in general. Similarly can our claim that ‘care’ is marginalized within the EBM-discourse, not be reduced to a claim about the amount of attention given to the word ‘care’ within the Cochrane Collaboration.

        In his first commentary to our paper, Chalmers wants to distinguish the Cochrane Collaboration and himself from the EBM-movement, claiming that these are different entities with different origins and different histories. In our paper, whenever we claim that the care dimension of medicine is being suppressed, overlooked, marginalized or excluded, we refer to EBM and EBM-discourse in general, not the Cochrane Collaboration. This claim is of course, based on a generalization, and we are very much aware that there are exceptions, but we are not convinced that the Cochrane Collaboration is one of these exceptions. But if Chalmers think that the Cochrane Collaboration is not even a part of the EBM-movement, then why does he presuppose that the claim included them in the first place?

        As stated both in our article and in our reply to Chalmers, our intention was to draw attention to Cochrane’s book Effectiveness and efficiency (1972), and what we perceive to be the dominant way of reading and appropriating this book in the EBM-literature. We have not written a paper on the Cochrane Collaboration. When we used the expression ‘Cochrane’s mission statement’, we refer to Cochrane’s mission in Effectiveness and efficiency, not the mission statement of the Cochrane Collaboration.

        When Chalmers wants us to provide evidence for the claims in our paper he touches upon the fundamental question of the nature of evidence. When EBM-proponents talk about evidence, they usually mean clinically relevant results from RCTs and systematic reviews that can be used by practitioners to make better informed decisions and thereby provide more accurate diagnosis, or more effective treatments. This is all well and good, but it does not exhaust the meaning of the term ‘evidence’. In our article and within our academic field, evidence usually consists of textual evidence that can support arguments, generalizations and categorizations. We believe we have provided such evidence in our paper for a more nuanced interpretation of Cochrane’s book than the dominant reading. We also believe we have provided evidence showing that the reductive and selective reading we criticize is in fact dominant within the EBM-movement.

        When we refer to Chalmers and colleagues 1992 BMJ-article, it is partly because it begins, after Chalmers’ favorite Cochrane-quote who serves as the article’s epigraph, with the sentence we quoted in our last reply: “Twenty years ago, in Effectiveness and Efficiency, Archie Cochrane emphasized the importance of randomised controlled trials in guiding decisions in health care.” We claim that this way of reading and summarizing Cochrane’s book is selective. And as evidence for this claim we refer to several places in the book where Cochrane nuances this picture.

        In Chalmers own 1993-article, which we do not refer to in our paper, he begins by stating: “Archie Cochrane is best known for his influential book, Effectiveness and Efficiency: Random Reflections on Health Services,’ published in 1972. The principles he set out in it so clearly were straightforward: he suggested that, because resources would always be limited, they should be used to provide equitably those forms of health care which had been shown in properly designed evaluations to be effective.” This way of characterizing the book is much more in line with our own reading. Chalmers adds the sentence: “In particular, he stressed the importance of using evidence from randomized controlled trials (RCTs) because these were likely to provide much more reliable information than other sources of evidence.” This insistence on the important role of RCTs in Cochrane’s book is more in line with what we believe is the dominant reading. But all in all this description is much more nuanced than the one from 1992.

        In 1997 Chalmers and Alan Maynard edited a collection of articles celebrating the 25th anniversary of Effectiveness and efficiency. In the foreword they write: “The radical critique of health and social services presented in Cochrane’s monograph had a substantial and lasting impact both within and outside the United Kingdom. He highlighted the absence of an adequate knowledge base for much of the care provided, and made a strong case for the evaluation of new and existing forms of care in controlled trials.” (p. xiii) This is not fully in line with our reading of the book, but it is not in absolute contrast to it either. But in Chalmers own chapter in the book (written with David Sackett and Chris Silagy) on the origin and early history of the Cochrane Collaboration, they state: “Cochrane’s simple proposal in Effectiveness and efficiency was that such evidence (‘from systematically assembled, reliable research’) is essential to inform the choices made within the health services by policy makers, practitioners, and patients.” (p. 232) This is again a selective reading of the book, as it includes ‘practitioners and patients’, ignoring Cochrane’s reservation that we quote in our paper: “‘It is in this sense that I use the word ‘effective’ in this book, and I use it in relation to research results, as opposed to the results obtained when a therapy is applied in routine clinical practice in a defined community.” (Cochrane 1972:2).

        An interesting point related to these different kinds of evidence and to differences in academic traditions is the use and non-use of page references in the research literature. In our reading of Cochrane’s book, we use quotations and page references to support our claims, a well-established procedure of argumentative openness, which we believe fits rather well with the general beliefs of Archie Cochrane. And even though we have tried to reach higher ground to be able to present an overview of the reception of Cochrane’s book within the EBM-movement, and to look further than the most obvious reading of the book, we can’t recall ever entering an ivory tower.

        Clemet Askheim, Tony Sandset and Eivind Engebretsen

        References
        Chalmers I, Dickersin K, Chalmers TC (1992). Getting to grips with Archie Cochrane’s agenda: a register of all randomized controlled trials. BMJ 305:786-788.
        Chalmers I (1993). The Cochrane Collaboration: preparing, maintaining and disseminating systematic reviews of the effects of health care. In: Warren KS, Mosteller F, eds. Doing more good than harm: the evaluation of health care interventions. Annals of the New York Academy of Sciences 703:156-163.
        Chalmers I, Sackett D, Silagy C (1997). The Cochrane Collaboration. In: Maynard A, Chalmers I (1997). Non-random reflections on health services research: on the 25th anniversary of Archie Cochrane’s ‘Effectiveness and Efficiency’. London: BMJ Books, pp. 231-249.
        Cochrane AL (1972). Effectiveness and efficiency. Random reflections of health services. London: Nuffield Provincial Hospitals Trust.
        Cochrane AL (1979). 1931-1971: a critical review, with particular reference to the medical profession. In: Teeling-Smith G, ed. Medicines for the Year 2000. London: Office of Health Economics, pp 1-11.
        Maynard A, Chalmers I (1997). Foreword. In: Non-random reflections on health services research: on the 25th anniversary of Archie Cochrane’s ‘Effectiveness and Efficiency’. London: BMJ Books, pp. xiii-xiv.
        Latour, B (1993). We have never been modern. Cambridgde, MA: Harvard University Press.

        Clemet Askheim / (in reply to Iain Chalmers) Reply
  3. An incomplete and misleading reading of Archie Cochrane
    I am pleased that Askheim et al. have considered some of the sources I suggested they should have taken in account their original article. That said, it is sad that they feel no need to draw attention to evidence that challenges their analysis. How reassuring it would have been, for example, if they had referred to a section entitled ‘Good decisions about health care rely on more than good reviews of the results of research’ in an article introducing the Cochrane Collaboration in the year the initiative was inaugurated (Chalmers 1993, p 160-161). That article included Cochrane’s moving account in Effectiveness and Efficiency of the care he had offered to a dying Soviet soldier in a prisoner of war camp.
    Because of their selective reading of the sources, the additional 1300 words Askheim et al. have offered in defence of their article remain muddled. They write that:
    “[I]n our paper, whenever we claim that the care dimension of medicine is being suppressed, overlooked, marginalized or excluded, we refer to EBM and EBM-discourse in general, not the Cochrane Collaboration.”
    This claim is contradicted in their paper, which, as I have pointed out in both my responses to it, confounds the distinct histories and objectives of two separate initiatives. Askheim et al. are not the first to have muddled these up by doing inadequate research. In their original paper, they wrote as follows:
    “(I)n a critical assessment of ‘evidence’ in EBM and the dogma of ‘best available evidence’, Alvan Feinstein and Ralph Horwitz, while acknowledging that EBM started as an anti-authoritarian movement in the spirit of Cochrane, wonder if it might end up as ‘a new form of dogmatic authoritarianism’ coming from ‘Cochranian Oxford’. Symptomatically, in their first reaction to the establishment of the Cochrane Centre, The Lancet wrote an editorial which said: ‘Specialty by specialty, authoritarianism based on mere seniority is giving way to authoritativeness based on randomized trials’. Only three years later, they took a more critical stand: ‘The voice of evidence-based medicine has grown over the past 25 years or so from a subversive whisper to a strident insistence that it is improper to practice medicine of any other kind’.”
    The first of the two Lancet editorials referred specifically to the agenda of the recently established Cochrane Centre: this was to promote the preparation and maintenance of systematic reviews of randomised trials. The second Lancet editorial was a reaction to David Sackett’s promotion of the practice of Evidence-Based Medicine, which was upsetting some senior members of the British medical establishment. Editors at The Lancet came to regret the reactionary stance their editorial had taken.
    Unreferenced assertions by Askheim et al. come across as quite arrogant. One fairly trivial illustration of this is what they refer to as ‘Chalmers’ favourite Cochrane quote’. Had they asked me what my favourite Cochrane quote is, they would have learned that is ‘All effective treatment should be free’. A more substantial example of their willingness to make ex cathedra statements without supporting evidence is their characterisation of what proponents of Evidence-Based Medicine mean by ‘evidence’. Thus:
    “When EBM-proponents talk about evidence, they usually mean clinically relevant results from RCTs and systematic reviews that can be used by practitioners to make better informed decisions and thereby provide more accurate diagnosis, or more effective treatments.”
    This unreferenced assertion strikes me as arrogant because it makes no reference to EBM’s use of evidence from professional experience and about patient preferences. I leave it to practitioners of EBM to respond to this limited conception of what the practice of EBM entails.
    As Askheim et al. state, “researchers are only able to see what they are looking for”; and this is as true of them as it is of others. However, that is no excuse for their failure to take their own advice and use “the tools they have [had] at hand”. They appear reluctant to acknowledge that they could have done a better job had they consulted a wider range of documents more assiduously, spoken to a range of informants, and perhaps used ethnomethodological methods to observe what the practice of EBM entails.
    My main criticism of the article by Askheim et al. is that it fails to use the data collection tools they had at hand and that – as a result – it is an avoidably muddled analysis. What is more worrying than the authors’ initial failure is that they clearly find it difficult to understand why opinion offered without reference to reliable supporting evidence is unacceptable. I conclude that there is little point in further interaction with academics who clearly find it difficult to acknowledge that they might have done a better job had they drawn on the various forms of evidence that were available to them.

    Reference
    Chalmers I. The Cochrane Collaboration: preparing, maintaining and disseminating systematic reviews of the effects of health care. In: Warren KS, Mosteller F, eds. Doing more good than harm: the evaluation of health care interventions. Annals of the New York Academy of Sciences 1993;703:156-163.

    Iain Chalmers / Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

*

UA-49496932-1