Sarah Chapman looks at the evidence from a new Cochrane rapid reviewA rapid review is a simplified systematic review that can be done in a few weeks to produce timely evidence for decision-making. on the accuracy of antibody tests for COVID-19 and at other important considerations about the roll-out of testing. Read more blogs about COVID-19.
Antibody testing for SARS-CoV-2 is in the news with the publication of a letter in the BMJ from a group of senior academics, expressing concerns about its establishment in England and, hard on their heels, a new Cochrane Rapid Review, which brings together the best available evidence from studies up to April 2020 on Antibody tests for identification of current and past infection with SARS-CoV-2.
With any test, it’s important to know how accurate it is and also why it’s being done – what will change as a result of doing it (for example, it might indicate a need for treatmentSomething done with the aim of improving health or relieving suffering. For example, medicines, surgery, psychological and physical therapies, diet and exercise changes.). A programme of testing across a populationThe group of people being studied. Populations may be defined by any characteristics e.g. where they live, age group, certain diseases. will have multiple impacts. The Cochrane ReviewCochrane Reviews are systematic reviews. In systematic reviews we search for and summarize studies that answer a specific research question (e.g. is paracetamol effective and safe for treating back pain?). The studies are identified, assessed, and summarized by using a systematic and predefined approach. They inform recommendations for healthcare and research. addresses the former, while the letter to the BMJ raises wider concerns about the roll-out of antibody testing and its implications.
Antibody tests and COVID-19
When someone has COVID-19, their immune system will respond by producing blood proteins – antibodies – that can attack the virus. Detecting these antibodies in a person’s blood may indicate that they have, or have had, COVID-19; we can’t be certain, as the tests are not 100% accurate. Finding out how accurate they are for diagnosing COVID-19 in people with or without symptoms, and for establishing if someone has had COVID-19, were the objectives of the new Cochrane Review.
Antibody tests are done using samples of blood taken from a vein or from a finger prick. These are different from PCR swab tests for antigens (part of the virus, triggering the immune system to make antibodies), to see if someone has COVID-19 currently. These swab tests aren’t perfect either.
Cochrane evidence on antibody tests for COVID-19
The review authors looked for studies that reported results of any antibody tests in groups of people known to have (or have had) COVID-19 and others known not to have had it. They found 54 relevant studies, reporting test results for nearly 16,000 samples. The studies looked at three types of antibody, IgA, IgG and IgM, alone or in combination.
The main message from the review is that timing makes a big difference to the accuracy of these tests. Professor Jon Deeks, Professor of Biostatistics and head of the Test Evaluation Research Group at the University of Birmingham, puts it simply: “Use them at the wrong time and they don’t work.”
When tests of the IgG and IgM antibodies were done at 1 to 7 days from when symptoms started, they correctly identified only 30% of people who had COVID-19, but this rose to 70% at 8 to 14 days and 90% at 15 to 35 days. There isn’t enough evidence to indicate the accuracy of these tests after 35 days.
The tests wrongly diagnosed COVID-19 in only one to two percent of people who didn’t actually have it.
The review authors show how this could look in two groups of people, based on results from IgG/IgM tests done three weeks after symptoms started:
If 1000 people had antibody tests, and 50 (5%) of them really had COVID‐19 (as might be expected in a national screening survey):
- 58 people would test positive for COVID‐19. Of these, 12 people (21%) would not have COVID‐19 (false positiveIn diagnostic tests, a conclusion that a person does have the disease or condition being tested, when they actually do not. result).
- 942 people would test negative for COVID‐19. Of these, 4 people (0.4%) would actually have COVID‐19 (false negativeIn diagnostic tests, a conclusion that a person does not have the disease or condition being tested, when they actually do. result).
If 1000 healthcare workers (in a high‐riskA way of expressing the chance of an event taking place, expressed as the number of events divided by the total number of observations or people. It can be stated as ‘the chance of falling were one in four’ (1/4 = 25%). This measure is good no matter the incidence of events i.e. common or infrequent. setting) who had had symptoms were tested, and 500 (50%) of them really had COVID‐19:
- 464 people would test positive for COVID‐19. Of these, 7 people (2%) would not have COVID‐19 (false positive result).
- 537 people would test negative for COVID‐19. Of these, 43 (8%) would actually have COVID‐19 (false negative result).
The review authors say the evidence shows that antibody tests may have a role in diagnosing COVID-19 in people who have had COVID-19 symptoms for two weeks or longer but did not have a swab test or tested negative.
Some limitations of the evidence
The studies overall looked at dataData is the information collected through research. from 25 commercial tests and others developed ‘in-house’, representing only a small proportion of the available antibody tests. It was not possible to compare the accuracy of different tests. Most studies were done in China and the people tested had been admitted to hospital and were likely to have severe disease.
Professor Deeks commented:
“While these first COVID-19 antibody tests show potential, particularly when used two or three weeks after the onset of symptoms, the data are nearly all from hospitalized patients, so we don’t really know how accurately they identify COVID-19 in people with mild or no symptoms, or tested more than five weeks after symptoms started.”
There were other concerns too about the quality of the studies and the ways they were reported.
Some other considerations about antibody testing for COVID-19
It’s important to think about reasons for doing antibody testing for COVID-19 and at scale, and about the limitations and resource implications of this strategy. From a public health perspective, it could be useful for tracing patterns of disease spread and for evaluating public health measures such as quarantine. For individuals, one thing we’ve heard about over all these strange weeks is the possibility of ‘immunity passports’, with some governments suggesting that detecting SARS-CoV-2 antibodies could be the basis for individuals gaining permission to work or travel, for example. However, that is dependent on the assumption that recovery from COVID-19 gives protection from being infected again and as yet there is no evidence to support that. As things stand, those with a positive antibody test are to consider themselves as still at risk of infection and behave accordingly.
This is one of the problems highlighted in the BMJ letter, which points out that these test results are of no use to employers in the face of uncertainty about whether these antibodies confer immunity and nor are they likely to change clinical management. It’s hard to see how this is useful to individuals either and indeed it’s possible that people who have tested positive may be treated in ways that disadvantage them compared to those who test negative or whose antibody status is unknown. Other issues include the lack of data showing how well the tests perform in groups at particularly high risk, such as black and minority ethnic groups, and the rapid pace of rolling out tests at the expense of rigorous assessment of those tests.
The BMJ letter authors express concern that this “push to introduce a non-evidence based test for uncertain gains risks inefficient use of scarce resources”. They conclude that the only justification for large-scale SARS-CoV-2 antibody testing is for research purposes.
This Cochrane Review is a ‘living systematic review’, subject to frequent searches for new studies with any suitable for inclusion then being added. The review authors are already in the process of updating the review with new, more globally relevant studies and the updated version may be published later this summer. This may take us a step nearer to understanding how accurate these tests are, but it feels like we’re a long way off having answers to some pressing questions about their role and impact.
Take-home points
Join in the conversation on Twitter with @CochraneUK @SarahChapman30 @Cochrane_IDG or leave a comment on the blog. Please note, we cannot give medical advice and we will not publish comments that link to commercial sites or appear to endorse commercial products.
References and further reading (pdf).
Read more blogs about COVID-19.
This Cochrane Rapid Review is one of several simplified systematic reviewsIn systematic reviews we search for and summarize studies that answer a specific research question (e.g. is paracetamol effective and safe for treating back pain?). The studies are identified, assessed, and summarized by using a systematic and predefined approach. They inform recommendations for healthcare and research. that have been done quickly to produce timely evidence for decision-making during the pandemic. Go to the Cochrane COVID Rapid Reviews website.
Volunteers for a study of the accuracy of antibody home testing kits are being sought as part of the ongoing surveillance by the Department of Health.
If you are interested in knowing more about Diagnostic Test Accuracy (DTA) reviews, this video gives a brief explanation.
Sarah Chapman has nothing to disclose.
Hi. In this article (https://www.cochrane.org/CD013652/INFECTN_what-diagnostic-accuracy-antibody-tests-detection-infection-covid-19-virus), I found this, which doesn’t say anything about data available after day 21 regarding accuracy: “Antibody tests one week after first symptoms only detected 30% of people who had COVID-19. Accuracy increased in week 2 with 70% detected, and was highest in week 3 (more than 90% detected). Little evidence was available after week 3. Tests gave false positive results in 2% of those without COVID-19”.
How would you interpret this data?
The setting is an office occupied with 40-50 people without dividers on their table.
1st week. Wed – Patient HC had a swab test. On fri PCR result came and negative.
Fri – Patient AB had a swab test. On mon PCR result came and negative.
Sat – Patient RP, AD, RT & RB had a swab test. On mon result came and one was positive and that is RB.
RB, HC, AD, RT & RP are all living in a one room.
2nd week. Tue – All employees submit them selves for rapid test. Only 2 came out rapid test reactive. FV & AZ.
That same day. Swab sample was done on those 2 reactive together with 5 close contacts of RB (positive) even they are rapid test non reactive.
The sequence of swabbing that day was
1. FV (rapid test reactive)
2. JA (rapid test non-reactive)
3. AA (rapid test non-reactive)
4. VT (rapid test non-reactive)
5. KB (rapid test non-reactive)
6. AC (rapid test non-reactive)
7. AZ (rapid test reactive)
8. One lady from other office let herself test.
Thu – result came and all 8 of them are positive.
Fri – Office closed and all employees did PCR including JA’s and AA’s wife and AC’s husband.
Sat – The four housemates of RB (1st positive) repeated swab test
Sun – Result came for that fri swab 63/63 are all negative.
Mon – Result came for 4 housemates of RB and all negative.
How would you interpret that the 8 people on that tue swab are all positive? And 63 people are all negative plus the 4 major close contacts of RB are negative as well.