Longitudinal assessment of diagnostic test performance over the course of acute SARS-CoV-2 infection ==================================================================================================== * Rebecca L. Smith * Laura L. Gibson * Pamela P. Martinez * Ruian Ke * Agha Mirza * Madison Conte * Nicholas Gallagher * Abigail Conte * Leyi Wang * Rick Fredrickson * Darci C. Edmonson * Melinda E. Baughman * Karen K. Chiu * Hannah Choi * Tor W. Jensen * Kevin R. Scardina * Shannon Bradley * Stacy L. Gloss * Crystal Reinhart * Jagadeesh Yedetore * Alyssa N. Owens * John Broach * Bruce Barton * Peter Lazar * Darcy Henness * Todd Young * Alastair Dunnett * Matthew L. Robinson * Heba H. Mostafa * Andrew Pekosz * Yukari C. Manabe * William J. Heetderks * David D. McManus * Christopher B. Brooke ## SUMMARY **What is already known about this topic?**Diagnostic tests and sample types for SARS-CoV-2 vary in sensitivity across the infection period. **What is added by this report?**We show that both RTqPCR (from nasal swab and saliva) and the Quidel SARS Sofia FIA rapid antigen tests peak in sensitivity during the period in which live virus can be detected in nasal swabs, but that the sensitivity of RTqPCR tests rises more rapidly in the pre-infectious period. We also use empirical data to estimate the sensitivities of RTqPCR and antigen tests as a function of testing frequency. **What are the implications for public health practice?**RTqPCR tests will be more effective than rapid antigen tests at identifying infected individuals prior to or early during the infectious period and thus for minimizing forward transmission (provided results reporting is timely). All modalities, including rapid antigen tests, showed >94% sensitivity to detect infection if used at least twice per week. Regular surveillance/screening using rapid antigen tests 2-3 times per week can be an effective strategy to achieve high sensitivity (>95%) for identifying infected individuals. ## INTRODUCTION Frequent rapid diagnostic testing is critical for restricting community spread of SARS-CoV-2 by allowing the timely identification and isolation of infected individuals to interrupt the chain of transmission. Quantitative reverse transcription polymerase chain reaction (RTqPCR)-based detection of viral RNA within nasal swab or saliva samples represents the gold standard for sensitivity in detecting the presence of SARS-CoV-2, yet supply shortages, cost, and infrastructure limitations have made it difficult to achieve high testing frequency and volume with the rapid reporting of results needed to mitigate transmission effectively. Recently, there has been considerable interest in the potential of rapid antigen tests to expand diagnostic testing capacity due to the ease of use, availability, cost, and rapid time-to-results1. However, data for their use in screening asymptomatic individuals is sparse. Enthusiasm for their widespread deployment has been further tempered by well-publicized examples of false positive results in people with low pre-test probability of infection, and by reports suggesting they lack sensitivity compared with RTqPCR, potentially making them less effective at mitigating community spread2,3. Here, we compare the sensitivities of nasal and saliva RTqPCR tests with the Quidel Sofia SARS Antigen Fluorescent Immunoassay (FIA) over the course of mild or asymptomatic acute SARS-CoV-2 infection through daily sampling of individuals enrolled early during infection. ## METHODS This study was approved by the Western Institutional Review Board, and all participants consented freely. ### Participants All on-campus students and employees of the University of Illinois at Urbana-Champaign are required to submit saliva for RTqPCR testing every 2-4 days as part of the SHIELD campus surveillance testing program. Those testing positive are instructed to isolate, and were eligible to enroll in this study for a period of 24 hours following receipt of their positive test result. Close contacts of individuals who test positive (particularly those co-housed with them) are instructed to quarantine and were eligible to enroll for up to 5 days after their last known exposure to an infected individual. All participants were also required to have received a negative saliva RTqPCR result 7 days prior to enrollment. Individuals were recruited via either a link shared in an automated text message providing isolation information sent within 30 minutes of a positive test result, a call from a study recruiter, or a link shared by an enrolled study participant or included in information provided to all quarantining close contacts. In addition, signs were used at each testing location and a website was available to inform the community about the study. Participants were required to be at least 18 years of age, have a valid university ID, speak English, have internet access, and live within 8 miles of the university campus. After enrollment and consent, participants completed an initial survey to collect information on demographics and health history, including suspected date of SARS-CoV-2 exposure. They were then provided with sample collection supplies. Participants who tested positive prior to enrollment or during quarantine were followed for up to 14 days. Quarantining participants who continued to test negative by saliva RTqPCR were followed for up to 7 days after their last exposure. All participants’ data and survey responses were collected in the Eureka digital study platform. ### Sample collection Each day, participants were remotely observed by study staff collecting: 1. 2 mL of saliva into a 50mL conical tube. 2. 1 nasal swab from a single nostril using a foam-tipped swab that was placed within a dry collection tube. 3. 1 nasal swab from the other nostril using a flocked swab that was subsequently placed in a collection vial containing viral transport media (VTM). The order of nostrils (left vs. right) used for the two different swabs was randomized. For nasal swabs, participants were instructed to insert the soft tip of the swab at least 1 cm into the indicated nostril until they encountered mild resistance, rotate the swab around the nostril 5 times, leaving it in place for 10-15 seconds. After daily sample collection, participants completed a symptom survey. A courier collected all participant samples within 1 hour of collection using a no-contact pickup protocol designed to minimize courier exposure to infected participants. ### Saliva RTqPCR After collection, saliva samples were stored at room temperature and RTqPCR was run within 12 hours of initial collection. The protocol for direct saliva-to-RTqPCR assay used has been detailed previously4. In brief, saliva samples were heated at 95°C for 30 minutes, followed by the addition of 2X TBS at a 1:1 ratio (final concentration 1X TBE) and Tween-20 to a final concentration of 0.5%. Samples were assayed using the Thermo Taqpath COVID-19 assay. ### Quidel assay Foam-tipped nasal swabs were placed in collection tubes and stored at 4°C overnight based on guidance from the manufacturer. The morning after collection, swabs were run through the Sofia SARS antigen FIA on Sofia 2 devices according to the manufacturer’s protocol. ### Nasal swab RTqPCR Collection tubes containing VTM and flocked nasal swabs were stored at −80°C after collection and were subsequently shipped to Johns Hopkins University for RTqPCR and virus culture testing. After thawing, VTM was aliquoted for RTqPCR and infectivity assays. One ml of VTM from the nasal swab was assayed on the Abbott Alinity per manufacturer’s instructions in a College of American Pathologist and CLIA-certified laboratory. ### Nasal virus culture VeroTMPRSS2 cells were grown in complete medium (CM) consisting of DMEM with 10% fetal bovine serum (Gibco), 1 mM glutamine (Invitrogen), 1 mM sodium pyruvate (Invitrogen), 100 U/ml of penicillin (Invitrogen), and 100 μg/ml of streptomycin (Invitrogen)5. Viral infectivity was assessed on VeroTMPRSS2 cells as previously described using infection media (IM; identical to CM except the FBS is reduced to 2.5%)6. When a cytopathic effect was visible in >50% of cells in a given well, the supernatant was harvested. The presence of SARS-CoV-2 was confirmed through RTqPCR as described previously by extracting RNA from the cell culture supernatant using the Qiagen viral RNA isolation kit and performing RTqPCR using the N1 and N2 SARS-CoV-2-specific primers and probes in addition to primers and probes for human RNaseP gene using synthetic RNA target sequences to establish a standard curve7. ### Data Analysis At the time of analysis, nasal samples from 30 participants had been analyzed by virus culture and RTqPCR. Therefore, analyses that consider either nasal RTqPCR or viral culture results were conducted based on a limited participant set. All confidence intervals around sensitivity were calculated using binconf from the Hmisc package in R version 3.6.2. The sensitivity of each of the tests was analyzed in three different ways: First, the ability of each test (antigen, saliva RTqPCR, or nasal RTqPCR) to detect an infected person on a particular day relative to the day of first positive viral culture (“daily sensitivity”) was calculated. Daily sensitivity was not calculated for timepoints with fewer than 5 observed person-days. Second, the ability of each test to detect an infected person according to their viral culture status (“status sensitivity”) was calculated. Viral culture status was defined as “pre-positive” on days prior to the first positive viral culture result, “positive” on days for which viral culture results were positive, and “post-positive” on days with negative viral culture results that occur after the first positive culture result. Status sensitivity was defined as the proportion of person-days with a positive result. Finally, we calculated the ability of repeated testing over a 14-day period to detect an infected person (“protocol sensitivity”) using a value-of-information approach. Seven different testing frequencies were considered: daily, every other day, every third day, and so on, up to weekly sampling. For each individual, the result of testing on a given schedule was calculated for each potential starting date, with test results interpreted in parallel (all tests must be negative to be considered negative). For instance, each person contributed two observations to the “every other day” schedule, one starting on the first day of the study and the other starting on the second day of the study. The proportion of “observations” with a positive result (at least one positive test in the sampling timeframe) was considered to be the sensitivity of that testing protocol (test and frequency combination). All code used in analyses can be found here: [https://github.com/rlsdvm/CovidDetectAnalysis](https://github.com/rlsdvm/CovidDetectAnalysis) ## Results Table 1 shows demographic information for study participants reported here. The majority of participants (21/30, 70%) were non-Hispanic white and the average age was 32.50 (SD 12.29). View this table: [Table 1:](http://medrxiv.org/content/early/2021/03/22/2021.03.19.21253964/T1) Table 1: Demographic information on participants enrolled in the COVID detect study We first estimated the daily sensitivities of nasal and saliva RTqPCR and antigen tests relative to the day of first nasal swab viral culture positivity, which was used as a surrogate marker of infectious virus shedding (**Table 2, Figure 1**). We also used the viral culture data to measure the status sensitivities of each test before, during, and after viral shedding (**Figure 2**). View this table: [Table 2:](http://medrxiv.org/content/early/2021/03/22/2021.03.19.21253964/T2) Table 2: Daily sensitivity of each test platform by day relative to the day of first nasal swab viral culture positivity. ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/22/2021.03.19.21253964/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2021/03/22/2021.03.19.21253964/F1) Figure 1: Daily sensitivity of each test platform by day relative to the day of first positive viral culture result. Shaded areas represent the 95% confidence interval around the observed proportion. ![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/22/2021.03.19.21253964/F2.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2021/03/22/2021.03.19.21253964/F2) Figure 2: Status sensitivity of each test platform relative to viral culture positivity. Bars indicate the 95% confidence interval around the observed proportion. Pre-positive (n=29) refers to samples taken on days before the first viral culture-positive sample collected from each individual. Positive (n=127) refers to samples taken on days for which viral culture results were positive. Post-positive (n=112) refers to samples taken on days with negative viral culture results that occur after the first positive culture result. Prior to the first day of detectable shedding of infectious virus, both RTqPCR tests had higher daily sensitivity (0.706 for both saliva and nasal) than the antigen test (0.412). For all three tests, daily and status sensitivity peaked during days in which infectious virus shedding was detectable, as would be expected. Antigen test daily sensitivity declined precipitously after infectious virus could no longer be detected in nasal swabs, dropping below 0.5 within a week after the onset of culture positivity, while both nasal and saliva RTqPCR platforms only showed minor decreases in sensitivity, remaining at 0.792 and 0.667 after a week, respectively. We next estimated the protocol sensitivities, or how the ability of each of test platform to detect infected individuals was affected by differences in testing frequencies (**Table 3, Figure 3**). Protocol sensitivity was defined at the schedule level, where the numerator is the number of testing schedules resulting in at least one positive test and the denominator is the number of testing schedules examined, where a testing schedule is defined as a set of samples from one participant taken at a given frequency. In **Figure 3**, we calculated the effects of varying testing frequency on sensitivity to detect infected individuals on days where nasal swabs were viral culture positive in the top panel. In the bottom panel of **Figure 3**, we examined sensitivity to detect infected individuals at any stage of infection. View this table: [Table 3:](http://medrxiv.org/content/early/2021/03/22/2021.03.19.21253964/T3) Table 3: Protocol sensitivity of each test platform to detect an infected person during a 14-day testing period, relative to the frequency of testing. “Any time” refers to detection of the individual at any point in the 14-day testing period; “While VC+” refers to detection of the individual before or during the time in which their viral culture was positive. ![Figure 3:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/03/22/2021.03.19.21253964/F3.medium.gif) [Figure 3:](http://medrxiv.org/content/early/2021/03/22/2021.03.19.21253964/F3) Figure 3: Protocol sensitivity of each test platform to detect an infected person **(top)** before or during days where nasal samples were viral culture positive or **(bottom)** at any time, over a 14-day testing period, relative to frequency of testing. Lines indicate 95% confidence interval around the observed proportion. ## Discussion Our data demonstrate that the sensitivities of RTqPCR and antigen tests vary significantly overthe course of SARS-CoV-2 infection. Prior to the presumed infectious period (here defined as the period during which infectious virus could be detected in nasal swab samples), the daily sensitivities of nasal and saliva RTqPCR tests were higher than that of the Quidel Sofia SARS Antigen FIA, suggesting that RTqPCR tests will be more effective at identifying infected individuals before they transmit to others. Both RTqPCR and antigen tests peak in daily and status sensitivities when infectious virus is detectable in nasal swab samples, suggesting that all three modalities can be effective at identifying individuals during the presumed infectious period. After this period, the daily sensitivity of RTqPCR tests decreased gradually, with saliva RTqPCR dropping faster than nasal RTqPCR. These dynamics are consistent with those described previously for RTqPCR8,9. In contrast, the daily sensitivity of the antigen test declined very quickly, suggesting that this test will be less effective at identifying individuals during later stages of infection. This may limit diagnosis and contact-tracing efforts in test-limited environments. Previous studies have suggested that frequent testing would maximize the ability of a given test modality to detect infected individuals10,11. We found that all testing modalities showed almost 95% protocol sensitivity to detect infection if used at least twice per week. When applied weekly, protocol sensitivity remained very high for nasal RTqPCR, declined slightly to 90% for saliva RTqPCR, and dropped to only 76% for the antigen test. When we compared the abilities of different testing frequencies to identify individuals while infectious virus was detectable in nasal samples, we observed a clear reduction in protocol sensitivity for all testing modalities when testing frequencies decreased below daily. The reduction in protocol sensitivity was most pronounced for the antigen test, which dropped to 0.72 with testing every fourth day, however, both RTqPCR tests were only slightly better at 0.74 (saliva) and 0.77 (nasal). Altogether, these data demonstrate the importance of frequent testing regardless of test modality for identifying individuals while they are contagious. This is the first study to compare the longitudinal performance of rapid antigen and RTqPCR tests with infectious virus shedding in a well-defined population early in SARS-CoV-2 infection. We found that all three diagnostic tests demonstrated a high degree of daily sensitivity during the presumed infectious period, but that the RTqPCR tests exhibited superior daily sensitivities prior to this period. Our data suggest that RTqPCR tests can be more effective than antigen tests at mitigating community spread of SARS-CoV-2, but only if the turnaround time for RTqPCR results is short. Finally, these data also quantitatively demonstrate the importance of frequent (at least twice per week) screening to maximize likelihood of detecting infected individuals regardless of testing modality. ## Data Availability All code used in analyses can be found here: https://github.com/rlsdvm/CovidDetectAnalysis [https://github.com/rlsdvm/CovidDetectAnalysis](https://github.com/rlsdvm/CovidDetectAnalysis) ## Acknowledgments This study was funded by the NIH RADx-Tech program under 3U54HL143541-02S2. The views expressed in this manuscript are those of the authors and do not necessarily represent the views of the National Institute of Biomedical Imaging and Bioengineering; the National Heart, Lung, and Blood Institute; the National Institutes of Health, or the U.S. Department of Health and Human Services. Sofia 2 devices and associated supplies were provided to Carle Foundation Hospital by Quidel, however Quidel played no role in the design of the study or the interpretation or presentation of the data. We also thank Shumon Ahmed, Carly Bell, Nate Bouton, Callie Brennen, Justin Brown, Coleco Buie, Emmaline Cler, Gary Cole, Trey Coleman, Lauren Engels, Savannah Feher, Kelsey Fox, Lexi Freeman, Yesenia Gonzalez, Montez Harris, Dan Hiser, Ayeshah Hussain, Daryl Jackson, Michael Jenkins, Kalombo Kalonji, Syntyche Kanku, Steven Krauklis, Mary Krouse, Elmore Leshoure, Joe Lewis, Angel Lopez, Guadalupe Lopez, Emily Luna, Chun Huai Luo, Colby Mackey, Skyler McLain, Yared Berhanu Melesse, Madison O’Donnell, Savanna Pflugmacher, Denver Piatt, Skyler Pierce, Jessica Quicksall, Gina Quitanilla, Ameera Samad, MacKenzie Scroggins, Monique Settles, Macie Sinn, Pete Varney, Evette Vlach, and Raeshun Williams-Chatman for their efforts supporting recruitment, enrollment, logistics, and sample collection. We also thank Jeffrey Olgin, Noah Peyser, and Xochitl Butler for assistance with the Eureka platform, Michelle Lore for assistance with REDcap, and Gillian Snyder for assistance in development of study protocols and logistics. ## Footnotes * fixed a few typos * Received March 19, 2021. * Revision received March 21, 2021. * Accepted March 22, 2021. * © 2021, Posted by Cold Spring Harbor Laboratory The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission. ## References 1. 1.Krüger LJ, Gaeddert M, Köppel L, Brümmer LE, Gottschalk C, Miranda IB, Schnitzler P, Kräusslich HG, Lindner AK, Nikolai O, Mockenhaupt FP, Seybold J, Corman VM, Drosten C, Pollock NR, Cubas-Atienzar AI, Kontogianni K, Collins A, Wright AH, Knorr B, Welker A, de Vos M, Sacks JA, Adams ER, Denkinger CM, for the study team. Evaluation of the accuracy, ease of use and limit of detection of novel, rapid, antigen-detecting point-of-care diagnostics for SARS-CoV-2 [Internet]. Infectious Diseases (except HIV/AIDS); 2020 Oct. Available from: [http://medrxiv.org/lookup/doi/10.1101/2020.10.01.20203836](http://medrxiv.org/lookup/doi/10.1101/2020.10.01.20203836) 2. 2.Jääskeläinen A, Ahava M, Jokela P, Szirovicza L, Pohjala S, Vapalahti O, Lappalainen M, Hepojoki J, Kurkela S. Evaluation of three rapid lateral flow antigen detection tests for the diagnosis of SARS-CoV-2 infection [Internet]. Infectious Diseases (except HIV/AIDS); 2021 Jan. Available from: [http://medrxiv.org/lookup/doi/10.1101/2020.12.30.20249057](http://medrxiv.org/lookup/doi/10.1101/2020.12.30.20249057) 3. 3.Moreno GK, Braun KM, Pray IW, Seagaloff HE, Lim A, Poulsen K, Meiman J, Borchers J, Westergaard RP, Moll MK, Friedrich T, O’Connor DH. SARS-CoV-2 transmission in intercollegiate athletics not fully mitigated with daily antigen testing [Internet]. Infectious Diseases (except HIV/AIDS); 2021 Mar. Available from: [http://medrxiv.org/lookup/doi/10.1101/2021.03.03.21252838](http://medrxiv.org/lookup/doi/10.1101/2021.03.03.21252838) 4. 4.Ranoa DRE, Holland RL, Alnaji FG, Green KJ, Wang L, Brooke CB, Burke MD, Fan TM, Hergenrother PJ. Saliva-Based Molecular Testing for SARS-CoV-2 that Bypasses RNA Extraction [Internet]. Microbiology; 2020 Jun. Available from: [http://biorxiv.org/lookup/doi/10.1101/2020.06.18.159434](http://biorxiv.org/lookup/doi/10.1101/2020.06.18.159434) 5. 5.Matsuyama S, Nao N, Shirato K, Kawase M, Saito S, Takayama I, Nagata N, Sekizuka T, Katoh H, Kato F, Sakata M, Tahara M, Kutsuna S, Ohmagari N, Kuroda M, Suzuki T, Kageyama T, Takeda M. Enhanced isolation of SARS-CoV-2 by TMPRSS2-expressing cells. Proc Natl Acad Sci U S A. 2020 Mar 31;117(13):7001–7003. PMCID: PMC7132130 [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoicG5hcyI7czo1OiJyZXNpZCI7czoxMToiMTE3LzEzLzcwMDEiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wMy8yMi8yMDIxLjAzLjE5LjIxMjUzOTY0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 6. 6.Pekosz A, Parvu V, Li M, Andrews JC, Manabe YC, Kodsi S, Gary DS, Roger-Dalbert C, Leitch J, Cooper CK. Antigen-Based Testing but Not Real-Time Polymerase Chain Reaction Correlates With Severe Acute Respiratory Syndrome Coronavirus 2 Viral Culture. Clin Infect Dis. 2021 Jan 20; PMCID: PMC7929138 7. 7.Waggoner JJ, Stittleburg V, Pond R, Saklawi Y, Sahoo MK, Babiker A, Hussaini L, Kraft CS, Pinsky BA, Anderson EJ, Rouphael N. Triplex Real-Time RT-PCR for Severe Acute Respiratory Syndrome Coronavirus 2. Emerg Infect Dis. 2020 Jul;26(7):1633–1635. 8. 8.Kissler SM, Fauver JR, Mack C, Olesen SW, Tai C, Shiue KY, Kalinich CC, Jednak S, Ott IM, Vogels CBF, Wohlgemuth J, Weisberger J, DiFiori J, Anderson DJ, Mancell J, Ho DD, Grubaugh ND, Grad YH. SARS-CoV-2 viral dynamics in acute infections [Internet]. Epidemiology; 2020 Oct. Available from: [http://medrxiv.org/lookup/doi/10.1101/2020.10.21.20217042](http://medrxiv.org/lookup/doi/10.1101/2020.10.21.20217042) 9. 9.Byrne AW, McEvoy D, Collins AB, Hunt K, Casey M, Barber A, Butler F, Griffin J, Lane EA, McAloon C, O’Brien K, Wall P, Walsh KA, More SJ. Inferred duration of infectious period of SARS-CoV-2: rapid scoping review and analysis of available evidence for asymptomatic and symptomatic COVID-19 cases. BMJ Open. 2020 Aug;10(8):e039856. 10. 10.Paltiel AD, Zheng A, Walensky RP. Assessment of SARS-CoV-2 Screening Strategies to Permit the Safe Reopening of College Campuses in the United States. JAMA Netw Open. 2020 Jul 31;3(7):e2016818. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2020.16818&link_type=DOI) 11. 11.Larremore DB, Wilder B, Lester E, Shehata S, Burke JM, Hay JA, Tambe M, Mina MJ, Parker R. Test sensitivity is secondary to frequency and turnaround time for COVID-19 screening. Sci Adv. 2021 Jan;7(1):eabd5393. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1126/sciadv.abd5393&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33219112&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F03%2F22%2F2021.03.19.21253964.atom)