Background. The specificity and positive predictive value of human immunodeficiency virus (HIV) RNA assays have not been evaluated in the setting of postexposure prophylaxis (PEP).
Methods. Plasma from subjects enrolled in a nonoccupational PEP study was tested with 2 branched-chain DNA (bDNA) assays, 2 polymerase chain reaction (PCR) assays, and a transcription-mediated amplification(TMA) assay. Assay specificity and positive predictive value were determined for subjects who remained negative for HIV antibody for ⩾3 months.
Results. In 329 subjects examined, the lowest specificities (90.1%–93.7%) were seen for bDNA testing performed in real time. The highest specificities were seen with batched bDNA version 3.0 (99.1%), standard PCR (99.4%), ultrasensitive PCR (100%), and TMA (99.6%) testing. Only the 2 assays with the highest specificities had positive predictive values >40%. For the bDNA assays, increasing the cutoff point at which a test is called positive (e.g., from 50 copies/mL to 500 copies/mL for version 3.0) increased both specificity and positive predictive values to 100%.
Conclusions. The positive predictive value of HIV RNA assays in individuals presenting for PEP is unacceptably low for bDNA-based testing and possibly acceptable for PCR- and TMA-based testing. Routine use of HIV RNA assays in such individuals is not recommended.
The effectiveness of postexposure prophylaxis (PEP) to prevent HIV infection after occupational exposure [1] has led to the growing use of PEP in nonoccupational settings [2–6]. Regardless of the setting, identification of preexisting HIV infection prevents unnecessary antiretroviral use and its attendant cost, toxicity, and risk of development of antiretroviral drug resistance. Screening for HIV infection is particularly important after nonoccupational exposure, becauses the prevalence of preexisting infection is likely to be higher than that in occupational settings.
Antibody-based tests detect the majority of HIV-infected persons but are limited by an inability to identify the preseroconversion window period [7, 8]. HIV RNA-based testing can identify persons in this window period and is used to screen donated blood, where it performs with high sensitivity and specificity [9–11]. Because screening in the context of PEP occurs in clinical laboratories, where testing of HIV-infected samples is common, specificity may be compromised by contamination or other laboratory error. Before RNA assays can be recommended for diagnostic use in the setting of PEP, it must be determined whether the potential advantages of their high sensitivity are outweighed by low specificity. Thus, we evaluated the specificity and positive predictive value of 5 HIV RNA assays in individuals presenting for PEP.
Study design. The Committee on Human Research at the University of California, San Francisco, approved the study protocol. Each participant provided written informed consent. Individuals who reported having had, during the preceding 72 h, a potential sexual or injection drug use exposure to HIV were enrolled in a feasibility study between December 1997 and April 1999 [2]; they received 28 days of antiretroviral medications and were observed, for 52 weeks, for HIV seroconversion. Serum was tested for antibodies to HIV, followed by confirmatory testing with either an HIV-1 Western blot or an indirect fluorescent-antibody assay, and plasma was tested for HIV RNA, at presentation and at weeks 4, 26, and 52. All HIV RNA assays were performed according to the manufacturers' instructions [12–17].
Initially, HIV RNA testing was performed on fresh specimens, in real time, by a branched-chain DNA (bDNA) assay, the Quantiplex HIV-1 RNA 2.0 Assay (Bayer), at the Bayer Reference Laboratory; subsequently, from March 1998, the Versant HIV-1 RNA 3.0 Assay (Bayer) was used. The lower limit of detection was 500 copies/mL for the Quantiplex (bDNA 2.0) assay and 50 copies/mL for the Versant (bDNA 3.0) assay. In September 1998, the protocol was modified, and frozen samples were archived and tested, in batches, at the Bayer Reference Laboratory.
A subset of frozen samples was tested, in batches, with standard and ultrasensitive polymerase chain reaction (PCR) assays (Amplicor HIV-1 Monitor Test, version 1.0; Roche), at the UCSF Clinical Microbiology Research Laboratory. The lower limit of quantification was 400 copies/mL for the standard assay and 50 copies/mL for the ultrasensitive assay. These assays can also detect HIV RNA that is below the limit of quantification.
A second subset of frozen samples was tested, in batches, with a transcription-mediated amplification (TMA) assay (Procleix HIV-1 Discriminatory Assay; Gen-Probe), at the Gen- Probe research laboratory. This is a qualitative assay for which the lower limit of detection is 30 copies/mL.
Statistical analysis. The specificity of each HIV RNA assay was determined by limiting the analysis to subjects who were negative for HIV antibody at the time of RNA testing and who continued to be so for ⩾ 3 months after the RNA test. A “best estimate” of specificity was determined, both for each version of each assay and for assay type, for tests performed in real time versus those performed in batches. For subjects who were tested at multiple time points by the same assay, the result of the initial test was used to determine the specificity.
The positive predictive value of a detectable HIV RNA result is the probability that the result is a true-positive result. Positive predictive value is a function of assay specificity and sensitivity and of the true prevalence of the condition [18]. We calculated a positive predictive value (including the upper and lower bounds of the 95% confidence interval [CI]) for the corresponding specificity for each assay. Sensitivity was assumed to be 100%. For prevalence, we assumed a range of values for the percentage of individuals presenting for PEP who were in the preseroconversion window period. A recent study of individuals who received routine HIV testing reported an estimated prevalence of 4.9 such cases per 10,000 [19]. In San Francisco, the estimated incidence of HIV infection in men who have sex with men is ∼2/100 person-years [20]. If we assume that the period of detectable HIV RNA before the development of detectable antibody can be as long as 6 weeks [8], then the equivalent preseroconversion prevalence is 25/10,000. A very high estimate of HIV incidence—8/100 person-years, which might be seen in very-high-risk individuals—is equivalent to a prevalence of 100/10,000. Thus, the positive predictive values were calculated over a prevalence of 1/10,000–100/10,000. We also calculated specificity and positive predictive value by using the following cutoffs to define a positive test result: for bDNA version 2.0, 1,500 and 2,000 copies/mL; for bDNA version 3.0, 200 and 500 copies/mL. Logistic regression, performed by the generalized estimating equations (GEE) technique, was used to evaluate factors associated with false-positive HIV RNA results.
Specificity of HIV RNA testing in plasma, at the time of presentation for PEP and during the 6 months after PEP. In 329 of the 401 subjects examined, negative results on HIVantibody tests were documented at the time of RNA testing and ⩾ 3 months later. After HIV RNA testing, the median number of months during which the results of HIV-antibody tests remained negative was 12.2 (interquartile range [IQR], 11.8– 13.0) for specimens collected at presentation for PEP, 11.2 (IQR, 10.6–12.0) for samples taken at week 4, and 6.2 (IQR, 5.7–6.9) for samples taken at week 26. Of the remaining 72 subjects, 4 had positive results on HIV-antibody tests at baseline and 68 either were lost to follow-up (before the completion of 3 months of observation) or did not undergo HIV RNA testing. There were no documented HIV-antibody seroconversions identified at 6 months of follow-up.
In the 329 subjects available for analysis, the percentage of false-positive results for each RNA assay was 0%–10.3%, and the specificity was 90.1%–100% (table 1). The lowest specificities were seen for bDNA testing performed in real time (90.1% for version 3.0 and 93.7% for version 2.0); the highest specificities were seen for batched bDNA version 3.0 (99.1%), standard PCR (99.4%), ultrasensitive PCR (100%), and TMA testing (99.6%).
Positive predictive value of a detectable HIV RNA test result, across a range of prevalences of persons in the preseroconversion window period (i.e., true HIV antibody-negative HIV RNA-positive).
HIV RNA testing in persons presenting for PEP who were subsequently determined to be negative for HIV antibody for ⩾3 months.
Potential mechanisms of false-positive HIV RNA test results. If some of the detectable HIV RNA results at baseline reflected very early HIV replication resulting from the presenting exposure that subsequently was aborted by PEP, they would be considered true-positive test results, and we would expect that the percentage of tests showing detectable HIVRNA would be greater at baseline than at either week 4 or week 26. However, no tests showed trends in detectable HIV RNA over time (P<.39 for each assay type, by the GEE method; table 1); furthermore, in no subject was detectable HIV RNA shown by more than 1 assay type. The possible presence of biological interfering substances is unlikely, because in no assay did subjects repeatedly show positive results over time, and repeat testing of initially detectable specimens in batch testing by either bDNA version 3.0, PCR, or TMA did not show detectable HIV RNA. However, repeat testing of samples by bDNA versions 2.0 and 3.0 in real time resulted in persistently detectable HIV RNA in 46.2% and 35% of the samples, respectively.
We assessed the potential relationship between false-positive HIV RNA results and factors related to (1) the host (i.e., age, sex, race/ethnicity, and hepatitis C-antibody status), (2) exposure (i.e., type of exposure and time between exposure and the drawing of blood), and (3) exposure source (i.e., the exposed subject's perception as to whether the exposure source was HIV infected) and found no significant associations (data not shown). Only assay type and performance setting (realtime vs. batched testing) were associated with the likelihood that a false-positive HIV RNA test result would be observed. When we used bDNA version 3.0 real-time testing as the reference, bDNA version 3.0 batched testing (odds ratio [OR], 0.16; 95% CI, 0.07-0.36; P<.001), standard PCR batch testing (OR, 0.06; 95% CI, 0.01–0.47; P = .007), ultrasensitive PCR batch testing (OR, undefined; P<.001), and TMA batch testing (OR, 0.04; 95% CI, 0.01–0.16; P<.001) were all less likely to provide false-positive results. Difference in specificity between bDNA version 3.0 performed in real-time testing and performed in batch testing suggests that some error (e.g., sample contamination, mislabeling, or switching of samples) is inherent in real-time testing and is the most likely explanation for the majority of false-positive results.
Positive predictive value of HIV-1 RNA testing. Because there were no subjects with detectable plasma HIV RNA who subsequently seroconverted, the observed prevalence of persons in the preseroconversion window period was zero. Therefore, the observed positive predictive value for all assays and testing conditions could be no greater than zero. However, because the number of subjects was too small for us to estimate the true prevalence of persons in the preseroconversion window period, we estimated the positive predictive value by assuming a prevalence of 25/10,000 [20]. With this assumed prevalence, the positive predictive value of each of the assays and testing conditions (i.e., real-time testing and batch testing) was initially calculated with use of the estimated specificity of each assay and condition (table 2); for example, for the bDNA 3.0 assay performed in real time, the specificity of 90.1% equates to a positive predictive value of only 2%. Even a specificity estimate as high as 99.1%, as is seen for bDNA version 3.0 batch testing, equates to a positive predictive value of only 21%. Only the 2 assays with the highest estimates of specificity—namely, TMA (specificity of 99.6%) and PCR ultrasensitive (specificity of 100%)—had positive predictive values >40%.
We subsequently determined the positive predictive value of the different assays and testing conditions across a range of prevalences, 1/10,000–100/10,000 persons, in the preseroconversion window period (figure 1). As the prevalence (the horizontal axis in figure 1) increases, the positive predictive value also increases; for example, when the prevalence increases to 100/10,000 for the bDNA version 3.0 assay performed in batches, the positive predictive value, when a specificity of 99.1% is assumed, also increases— but only to 53%. For the 2 bDNA assays performed in real time, the positive predictive values are <30%, even when a specificity equal to the upper bound of the 95% CI and a prevalence equal to 100/10,000 are assumed.
For all assays except PCR ultrasensitive (which exhibited 100% specificity), upward adjustment of the cutoff point at which a test is called positive increases both specificity and positive predictive value. For the bDNA version 2.0 assay, a cutoff of 2000 copies/mL results in 100% specificity and positive predictive value, but a cutoff of 1500 copies/mL yields a positive predictive value of only 14%, when a prevalence of 25/10,000 is assumed. For the bDNA version 3.0 assay, conducted in both real time and batches, using a cutoff of 500 copies/mL would yield 100% specificity and a positive predictive value; at a cutoff of 200 copies/mL, the positive predictive value, in both performance settings of the bDNA version 3.0 assay, is still only 29%, when a prevalence of 25/10,000 is assumed.
The inability of HIV-antibody tests to detect infection during the preseroconversion window period has generated interest in the use of HIV RNA-based assays in individuals presenting for nonoccupational PEP. Our study demonstrates, however, that, when the current lower limit of detection is used, the positive predictive value of HIV RNA assays in individuals presenting for PEP is unacceptably low for bDNA-based testing and only possibly acceptable for PCR- and TMA-based testing. Our results are consistent with proficiency-testing data, with studies of individuals being evaluated for primary HIV infection, and with case reports of potentially exposed individuals [14, 19, 21–29]; for example, when the manufacturer's recommened lower limit of detection was used, the specificities of the bDNA, PCR, and TMA tests in a prospective study of 258 subjects with suspected primary HIV infection were 95%, 97%, and 98%, respectively [25]; only in studies in which the lower limit of detection has been increased—for example, to 1800 copies/ mL in a recent study of a PCR assay—have the specificity (99.99%) and corresponding positive predictive value (80%) reached levels acceptable for clinical use [19].
Although, in the present study, PCR- and TMA-based assays were associated with higher specificity than were the bDNA assays, they were performed only in a controlled batch setting; it is not known how these assays would perform in real time in less controlled circumstances, such as in a clinical laboratory. In addition, because of the limited number of samples tested by the PCR and TMA assays, we cannot rule out specificity values as low as 96.9%, the lower limit of the 95% CI. The positive predictive values corresponding to these specificities are <25%, even at the very high prevalence estimate of 100 cases of HIV antibody-negative HIV RNA-positive individuals/10,000.
We do not recommend the routine use of HIV RNA–based assays in persons presenting for PEP, because we think that the potential harm outweighs the potential benefit of very early diagnosis. Receiving an HIV RNA test result that is likely to be a false positive but requires follow-up testing for confirmation is extremely stressful. There is also the risk that PEP might be stopped as a result of a false-positive result and that the person subsequently might develop an infection that could have been prevented. Although the prevalence of preseroconversion HIV infection is higher in nonoccupational settings than in occupational settings, it is still low in absolute numbers— for example, ∼25/10,000 in men who have sex with men in urban areas such as San Francisco [20]. Thus, patients presenting for PEP are unlikely to be in the window period from a previous exposure at baseline; we did not observe any. Despite specificities of up to 99%, the low prevalence of preseroconversion HIV infection leads to an unacceptably low positive predictive value for RNA-based testing. If RNA-based assays are to be used as a diagnostic tool in the setting of nonoccupational PEP, their positive predictive values can be increased by increasing the cutoff for positivity; this will reduce the di agnostic sensitivity, but, outside the blood-donor setting, it is probably a reasonable sacrifice.
We would like to acknowledge Michael Barcellos, Lynette Sawyer and David Chernoff (Bayer); Beverly Dale (Roche); and Cristina Giachetti (Gen- Probe) for laboratory assistance. Stephen May provided assistance in the preparation of the manuscript.
IDSA Members: For your free access to this journal, log in via the IDSA members area.
Open access options for authors visit Oxford Open
This journal enables compliance with the NIH Public Access Policy