Antibody tests for COVID-19: new evidence on test accuracy and some considerations

Sarah Chapman looks at the evidence from a new Cochrane rapid reviewA rapid review is a simplified systematic review that can be done in a few weeks to produce timely evidence for decision-making. More on the accuracy of antibody tests for COVID-19 and at other important considerations about the roll-out of testing. Read more blogs about COVID-19.

Antibody testing for SARS-CoV-2 is in the news with the publication of a letter in the BMJ from a group of senior academics, expressing concerns about its establishment in England and, hard on their heels, a new Cochrane Rapid Review, which brings together the best available evidence from studies up to April 2020 on Antibody tests for identification of current and past infection with SARS-CoV-2.

With any test, it’s important to know how accurate it is and also why it’s being done – what will change as a result of doing it (for example, it might indicate a need for treatmentSomething done with the aim of improving health or relieving suffering. For example, medicines, surgery, psychological and physical therapies, diet and exercise changes. More). A programme of testing across a populationThe group of people being studied. Populations may be defined by any characteristics e.g. where they live, age group, certain diseases. More will have multiple impacts. The Cochrane ReviewCochrane Reviews are systematic reviews. In systematic reviews we search for and summarize studies that answer a specific research question (e.g. is paracetamol effective and safe for treating back pain?). The studies are identified, assessed, and summarized by using a systematic and predefined approach. They inform recommendations for healthcare and research. More addresses the former, while the letter to the BMJ raises wider concerns about the roll-out of antibody testing and its implications.

Antibody tests and COVID-19

When someone has COVID-19, their immune system will respond by producing blood proteins  – antibodies – that can attack the virus. Detecting these antibodies in a person’s blood may indicate that they have, or have had, COVID-19; we can’t be certain, as the tests are not 100% accurate. Finding out how accurate they are for diagnosing COVID-19 in people with or without symptoms, and for establishing if someone has had COVID-19, were the objectives of the new Cochrane Review.

Antibody tests are done using samples of blood taken from a vein or from a finger prick. These are different from PCR swab tests for antigens (part of the virus, triggering the immune system to make antibodies), to see if someone has COVID-19 currently. These swab tests aren’t perfect either.

Cochrane evidence on antibody tests for COVID-19

The review authors looked for studies that reported results of any antibody tests in groups of people known to have (or have had) COVID-19 and others known not to have had it. They found 54 relevant studies, reporting test results for nearly 16,000 samples. The studies looked at three types of antibody, IgA, IgG and IgM, alone or in combination.

The main message from the review is that timing makes a big difference to the accuracy of these tests. Professor Jon Deeks, Professor of Biostatistics and head of the Test Evaluation Research Group at the University of Birmingham, puts it simply: “Use them at the wrong time and they don’t work.”

When tests of the IgG and IgM antibodies were done at 1 to 7 days from when symptoms started, they correctly identified only 30% of people who had COVID-19, but this rose to 70% at 8 to 14 days and 90% at 15 to 35 days. There isn’t enough evidence to indicate the accuracy of these tests after 35 days.

The tests wrongly diagnosed COVID-19 in only one to two percent of people who didn’t actually have it.

The review authors show how this could look in two groups of people, based on results from IgG/IgM tests done three weeks after symptoms started:

If 1000 people had antibody tests, and 50 (5%) of them really had COVID‐19 (as might be expected in a national screening survey):

• 58 people would test positive for COVID‐19. Of these, 12 people (21%) would not have COVID‐19 (false positiveIn diagnostic tests, a conclusion that a person does have the disease or condition being tested, when they actually do not. More result).
• 942 people would test negative for COVID‐19. Of these, 4 people (0.4%) would actually have COVID‐19 (false negativeIn diagnostic tests, a conclusion that a person does not have the disease or condition being tested, when they actually do. More result).

If 1000 healthcare workers (in a high‐riskA way of expressing the chance of an event taking place, expressed as the number of events divided by the total number of observations or people. It can be stated as ‘the chance of falling were one in four’ (1/4 = 25%). This measure is good no matter the incidence of events i.e. common or infrequent. More setting) who had had symptoms were tested, and 500 (50%) of them really had COVID‐19:

• 464 people would test positive for COVID‐19. Of these, 7 people (2%) would not have COVID‐19 (false positive result).
• 537 people would test negative for COVID‐19. Of these, 43 (8%) would actually have COVID‐19 (false negative result).

The review authors say the evidence shows that antibody tests may have a role in diagnosing COVID-19 in people who have had COVID-19 symptoms for two weeks or longer but did not have a swab test or tested negative.

Some limitations of the evidence

The studies overall looked at dataData is the information collected through research. More from 25 commercial tests and others developed ‘in-house’, representing only a small proportion of the available antibody tests. It was not possible to compare the accuracy of different tests. Most studies were done in China and the people tested had been admitted to hospital and were likely to have severe disease.

Professor Deeks commented:

“While these first COVID-19 antibody tests show potential, particularly when used two or three weeks after the onset of symptoms, the data are nearly all from hospitalized patients, so we don’t really know how accurately they identify COVID-19 in people with mild or no symptoms, or tested more than five weeks after symptoms started.”

There were other concerns too about the quality of the studies and the ways they were reported.

Some other considerations about antibody testing for COVID-19

It’s important to think about reasons for doing antibody testing for COVID-19 and at scale, and about the limitations and resource implications of this strategy. From a public health perspective, it could be useful for tracing patterns of disease spread and for evaluating public health measures such as quarantine. For individuals, one thing we’ve heard about over all these strange weeks is the possibility of ‘immunity passports’, with some governments suggesting that detecting SARS-CoV-2 antibodies could be the basis for individuals gaining permission to work or travel, for example. However, that is dependent on the assumption that recovery from COVID-19 gives protection from being infected again and as yet there is no evidence to support that. As things stand, those with a positive antibody test are to consider themselves as still at risk of infection and behave accordingly.

This is one of the problems highlighted in the BMJ letter, which points out that these test results are of no use to employers in the face of uncertainty about whether these antibodies confer immunity and nor are they likely to change clinical management. It’s hard to see how this is useful to individuals either and indeed it’s possible that people who have tested positive may be treated in ways that disadvantage them compared to those who test negative or whose antibody status is unknown. Other issues include the lack of data showing how well the tests perform in groups at particularly high risk, such as black and minority ethnic groups, and the rapid pace of rolling out tests at the expense of rigorous assessment of those tests.

The BMJ letter authors express concern that this “push to introduce a non-evidence based test for uncertain gains risks inefficient use of scarce resources”. They conclude that the only justification for large-scale SARS-CoV-2 antibody testing is for research purposes.

This Cochrane Review is a ‘living systematic review’, subject to frequent searches for new studies with any suitable for inclusion then being added. The review authors are already in the process of updating the review with new, more globally relevant studies and the updated version may be published later this summer. This may take us a step nearer to understanding how accurate these tests are, but it feels like we’re a long way off having answers to some pressing questions about their role and impact.

Take-home points

Join in the conversation on Twitter with @CochraneUK @SarahChapman30 @Cochrane_IDG or leave a comment on the blog. Please note, we cannot give medical advice and we will not publish comments that link to commercial sites or appear to endorse commercial products.

References and further reading (pdf).

Read more blogs about COVID-19.

This Cochrane Rapid Review is one of several simplified systematic reviewsIn systematic reviews we search for and summarize studies that answer a specific research question (e.g. is paracetamol effective and safe for treating back pain?). The studies are identified, assessed, and summarized by using a systematic and predefined approach. They inform recommendations for healthcare and research. More that have been done quickly to produce timely evidence for decision-making during the pandemic. Go to the Cochrane COVID Rapid Reviews website.

Volunteers for a study of the accuracy of antibody home testing kits are being sought as part of the ongoing surveillance by the Department of Health.

If you are interested in knowing more about Diagnostic Test Accuracy (DTA) reviews, this video gives a brief explanation.

Sarah Chapman has nothing to disclose.