Coronavirus April 2020—Part 7 The Accuracy of Our Antibody Test

UPDATE: Since the original publication of this post, a study was published in the Journal of Clinical Microbiology on the performance characteristics of the Abbott test that show it’s even more sensitive and specific than Quest’s internal testing showed. Quest’s internal study showed Abbott’s test has a sensitivity of 97 percent and a specificity of 98.5 percent. According to the study in the Journal of Clinical Microbiology, when the test was run on day 17 after the onset of symptoms, the sensitivity was 100 percent (95% confidence interval: 95.1-100%, n=125) and the specificity was 99.9 percent (95% confidence interval: not provided, n=1,020, from the serum of patients collected before COVID-19 had reached the U.S.). We’ve recalculated the PPV and NPV and revisited our reasoning below using these updated numbers.

_____

A (slightly) briefer post this time, prompted by some news: ImagineMD now has access to an antibody test for SARS-CoV-2 that has a sensitivity of 97 percent and a specificity of 98.5 percent (see above for more up-to-date information).

Quest Diagnostics, our national lab partner, is utilizing this ELISA test, made by Abbott Labs, in the Chicagoland area. (Importantly, Quest has another antibody test that has a sensitivity of only 80 percent. We don’t know where this test is being used outside of the Chicagoland area.) The sensitivity and specificity have been confirmed directly by a Quest Diagnostics pathologist who told us that the test characteristics were determined internally by Quest using blood from patients who were RT-PCR positive as well as blood from patients from past years (prior to the COVID-19 outbreak and who were therefore obviously negative) as the gold standards. As before, if you want to skip the reasoning behind our recommendations for who should be tested (it’s not everyone), we’ve highlighted our CONCLUSION below.

We can use the real-world test characteristics noted above to calculate the positive predictive values (PPVs) and negative predictive values (NPVs) both for patients with a history of symptoms consistent with COVID-19 and for patients who haven’t had such symptoms—as we demonstrated how to do in the last post in the series—to figure out in what circumstances we can have confidence in the test results. As before, we need to fill in the following table:

COVID-19 infection present COVID-19 infection absent Predictive Values
(+) antibody test True positive rate False positive rate PPV
(-) antibody test False negative rate True negative rate NPV
Test characteristics Sensitivity=100% Specificity=99.9%

 

To remind you how we calculate an estimate for the prevalence of COVID-19 in the U.S., we start with the total number of symptomatic SARS-CoV-2 positives in the U.S., which is currently 844K (out of 4.5M tested). Given the likelihood that the false negative rate of the RT-PCR test may be 26 percent among patients with infectious symptoms, we also need to take the total number of negative tests, 3.7M (= 4.5M – 844K), and multiply by 26 percent to identify the hidden number of symptomatic SARS-CoV-2 infections, which would be another 962K (= 3.7M x 0.26). Add that number (962K) to the number of symptomatic SARS-CoV-2 positives from above (844K) and we get a total of 1.8M total symptomatic COVID-19 patients in the U.S. as of this writing. This would mean the prevalence of SARS-CoV-2 infection in symptomatic patients the U.S. as of this writing is 40% (= 1.8M divided by 4.5M x 100).

So for people who have had infectious symptoms consistent with COVID-19 since January 2020 (current prevalence of 40 percent), the antibody test we have might be expected to yield the following positive and negative predictive values:

COVID-19 infection present (400 people) COVID-19 infection absent (600 people) Predictive Values
(+) antibody test True positive=400 people False positive=0.6 people PPV=99.8%
(-) antibody test False negative=0 people True negative=599.4 people NPV=100%
Test characteristics Sensitivity=100% Specificity=99.9%

 

There’s been some thinking that people may have developed symptoms consistent with COVID-19 as early as January in certain areas of the country like Chicago, Boston, and San Francisco. We weren’t testing for the disease back then, so it’s impossible to know what the prevalence was at that time. But if we assume the prevalence of COVID-19 in patients with consistent symptoms that far back was even as low as 5 percent—a reasonable clinical guess—the antibody test we have available would yield a PPV of 98 percent and an NPV of 100 percent (calculations not shown). So either test result in people with symptoms consistent with COVID-19 as far back as January will be real.

How well will this antibody test perform in patients who’ve not had symptoms consistent with COVID-19? The prevalence of asymptomatic infections is equal to the total number of people without infectious symptoms who nevertheless have COVID-19 divided by the total number of people without infectious symptoms. The total number of people without infectious symptoms who nevertheless have COVID-19, our numerator, we estimated in our previous post as being equal to the number of people with infectious symptoms who have COVID-19, based on the data from the Diamond Princess study, which would currently be 1.8M. The total number of people without infectious symptoms, our denominator, is equal to the population of the U.S. (331M) minus the total number of people with symptomatic COVID-19 infections. As we wrote in the last post, that latter number is at best an estimate as we’ve been under testing the population. In our last post, we assumed the number of symptomatic COVID-19 patients who haven’t been tested to be ten times the number of symptomatic COVID-19 patient who have been tested, which, as of this writing, would be 18M people. The total number of symptomatic COVID-19 patients in the U.S. as of this writing then would be 18M + 1.8M (symptomatic, tested patients) = 19.8M symptomatic COVID-19 patients. The number of total asymptomatic patients in the U.S. not infected with SARS-CoV-2 would then be 331M – 19.8M = 311M. As of this writing, then, in the U.S., the prevalence of asymptomatic COVID-19 infections would be 1.8M divided by 311M x 100, which yields a prevalence of 0.57 percent.

So for people who haven’t had infectious symptoms since January 2020 (prevalence of 0.57 percent), the antibody test we have would be expected to yield the following positive and negative predictive values:

COVID-19 infection present (5.7 people) COVID-19 infection absent (994.3 people) Predictive Values
(+) antibody test True positive=5.7 people False positive=1 person PPV=85%
(-) antibody test False negative=0 people True negative=993.3 people NPV=100%
Test characteristics Sensitivity=100% Specificity=99.9%

 

The current 85 percent PPV is arguably still too low for us to have solid confidence in a positive test result in the asymptomatic population. However, as the prevalence of asymptomatic infection inevitably grows and our estimates of it improve as RT-PCR testing becomes more widespread on asymptomatic people, we will almost certainly find the PPV increasing into the high 90s.

CONCLUSIONS: If you’ve had symptoms consistent with COVID-19 since January 2020, you can have a high degree of confidence in the results of our antibody test. If you’ve not had symptoms consistent with COVID-19 since January 2020, you can have a very high degree of confidence in a negative antibody test, but a positive antibody test is only 85 percent likely to be accurate. Thus, it may make sense for you to be tested if you had symptoms consistent with COVID-19, but it may not make sense to be tested if you’ve not had symptoms consistent with COVID-19. At this time, ImagineMD is still only testing patients who had symptoms consistent with COVID-19. Once the prevalence of asymptomatic patients rises high enough, we’ll begin antibody testing everyone.

It’s crucial to note, however, that we don’t know what being antibody positive means if you’ve had symptoms consistent with COVID-19 other than you definitely had COVID-19. It says nothing about your degree of immunity, your likelihood of catching SARS-CoV-2 again, or how severe your case will be if you do. As we noted in our previous post in the series, children who develop antibodies to seasonal coronaviruses (which are structurally similar to SARS-CoV-2) will sometimes become reinfected only a few months later even after developing antibodies. Even more potentially bad news: There’s some unsettling data from a study on macaques with a SARS infection (caused by SARS-CoV, a coronavirus structurally similar to SARS-CoV-2) showing that the presence of antibodies to the spike protein of SARS-CoV induced by immunization actually increased the immune reaction in the lungs and contributed to the so-called “cytokine storm” that seems to be responsible for severe lung injury and death in SARS (as well as in COVID-19). This seems to be the mechanism by which human patients with SARS die as well (a faster and more robust antibody response is seen in patients who die from SARS compared to those who don’t). Thus, the reason to be tested now is not so that if you’re positive, you can relax your social-distancing behavior. If anything, if you’re positive, you may want to be even more careful. The reason to be tested now is that once you learn your status, as more data becomes available, you can know which category of patient you fit and what that data might mean for you. We recognize that offering antibody testing at this point is controversial. But we also recognize that if you’ve had symptoms suspicious for COVID-19, it’s entirely natural to want to know if you had it, even if such information provides no additional guidance on how you should behave.

If you would like to be tested for antibodies to SARS-CoV-2 at ImagineMD, you must first be an ImagineMD patient. (If you’re not, we’d be happy to have you. You can sign up to become a patient here). You must also have been symptom-free for at least a week. Then you must call your ImagineMD physician to describe the symptoms you had that were consistent with COVID-19. Your ImagineMD physician will then make an appointment for you to have your blood drawn.

For previous posts related to COVID-19, see:

  1. Coronavirus February 2020—Part 1 What We Know So Far
  2. Coronavirus March 2020—Part 2 Measures to Protect Yourself
  3. Supporting Employee Health During the Coronavirus Pandemic
  4. Coronavirus March 2020—Part 3 Symptoms and Risks
  5. Coronavirus March 2020—Part 4 The Truth about Hydroxychloroquine
  6. Coronavirus April 2020—Part 5 The Real Risk of Death
  7. Coronavirus April 2020—Part 6 Evaluating Diagnostic Tests

Sign up to get notified when a new blog post has been published.

Leave a Reply

  • Thank you for doing the research and publishing the results regarding the differences in accuracy between tests from different test manufacturing companies.

    Without your information, doctors and citizens alike could be grouping totals and statistical results from various antibody tests together in one bucket, as if the data was all the same. It is not.

  • If someone tested positive using this test for SARS-Cov-2 antibodies and it has been, say, a month since they were symptomatic, is it safe to assume that they are no longer contagious? In particular, would it be relatively safe for them to visit an aging and lonely relative? I understand it is not 100% safe because of false positives.

    Jeff: Given the scenario you describe, it would be highly likely that the person would no longer be contagious.

  • Dr. Lickerman,

    Can COVID-19 survivors sustain permanent damage to lungs heart or other bodily systems after recovery? Something like a post COVID-19 infection syndrome?

    If so, and if there are advised treatments or lifestyle changes that would be life enhancing for these patients, I wonder if this may be another reason for people who had symptoms but were not tested to take the antibody test now.

    Linda: There is currently no evidence that COVID-19 causes permanent damage. But it’s too early to know. We’ll have to wait and see. That would be quite unusual—but this virus is proving itself to be quite unusual.