Methods

SettingsThis study is part of an ongoing collaborative effort between 3 large managed care organizations in geographically separate US locations. The project pools data from the administrative and clinical databases of these health care organizations to assess over sequential seasons the effectiveness of influenza vaccination among their elderly members

HealthPartners (HP) is a nonprofit health maintenance organization (HMO) with ∼650,000 members in Minnesota and Wisconsin. About 250,000 members receive coverage through a staff model HMO, and the rest through a network HMO model. Delivery systems include traditional HMOs, Medicare and Medicaid plans, and third-party administration of employer-funded benefit plans and dental plans. Oxford Health Plans (OHP) has 1.8 million members in New York, New Jersey, Pennsylvania, and Connecticut. Delivery systems include traditional HMOs, point-of-service plans, third-party administration of employer-funded benefit plans, and Medicare plus choice plans. Kaiser Permanente Northwest (KPNW) is a group model HMO with nearly 420,000 members, located in the Portland-Vancouver (Washington) area and in 2 smaller communities, Salem, Oregon, and Longview-Kelso, Washington

SubjectsPlan members ⩾65 years old as of 1 October for each study year were included in the cohorts for 1996–1997 and 1997–1998 if they were continuously enrolled for the 12 months before 1 October for that study year and through the study year influenza period. Patients who disenrolled during the influenza period were excluded unless the cause was death. Both HP and KPNW included members from their entire catchment areas; OHP included only members who lived in the New York City metropolitan area and adjacent counties. Nursing home patients, who represented only a small fraction at each site, were not included in the final analysis because their influenza immunization status could not be reliably determined

Data collectionBy use of a uniform data dictionary and file structure, each site obtained data from their linked clinical and administrative databases. This was the only source of data from the sites. Data for each site then were transmitted to and pooled at the study coordinating center. Information for all subjects included demographic information, presence of comorbid conditions as listed from inpatient or outpatient encounters during the 12-month baseline period, health care utilization during the baseline period, influenza vaccination status for that season, and outcomes of hospitalization or death during the influenza season. No reliable data on pneumococcal vaccination status were available and therefore were not collected

The following are the comorbid conditions and a summary of the ICD-9-CM codes used (an asterisk indicates that the condition was not retained in the final model): *anemia spleen (280–289 and 759.0), *cirrhosis (571), diabetes and endocrine (250 and 251), heart disease (093, 112.81, 130.3, 391, 393–398, 402, 404, 410–429, 745, 746, 747.1, 747.49, 759.82, 785.2, and 785.3), hematologic cancer (200–208), immunodeficiency and organ transplant (042, 079, 279, V08, and V42), lung disease (011, 460, 462, 465, 466, 480–511, 512.8, 513–517, 518.3, 518.8, 519.9, and 714.81), nonhematologic cancer (140–198 and 199.1), *nutritional deficiencies (254, 255, 259.2, and 260–269), renal disease (274.1, 408, 580–591, 593.71–593.73, and 593.9), dementia, stroke (290–294, 331, 340, 341, 348, and 438), and rheumatologic diseases (446, 710, 714.0–714.4, 714.8, 714.89, and 714.9). Each patient was evaluated for the presence of any of the codes listed and was classified as high risk if any were present in either inpatient or outpatient data files. Outcomes, assessed for the influenza season for each study year, included hospitalizations for pneumonia and influenza (ICD-9-CM codes 480–487) and deaths from all causes

Influenza seasonsThe study outcome periods included both the peak and total influenza seasons defined for each site based on local and national influenza surveillance data [17]. The peak influenza seasons represented periods of continuous influenza activity where ⩾2 influenza isolates were reported each week from the site. The total influenza seasons included all weeks from the first isolate to the last isolate. For both seasons, outcomes were included if they occurred within 2 weeks of the end of that season, to capture outcomes that might represent delayed complications of influenza

In the 1996–1997 season, the dates of peak influenza seasons were the same at all 3 sites: 22 November 1996 through 21 February 1997. The dates of total influenza seasons for the sites differed: HP, 22 November 1996 through 24 May 1997; OHP, 5 October 1996 through 3 May 1997; and KPNW, 22 November 1996 through 22 March 1997 (Centers for Disease Control [CDC], unpublished data)

In the 1997–1998 season, the dates of the peak and total influenza seasons were as follows: HP, 11 January through 21 February and 7 December through 28 March; OHP, 28 December through 14 February and 23 November through 4 April; and KPNW, 11 January through 14 February and 21 December through 7 March (CDC, unpublished data)

Virologic data were collected by local and state health departments and were forwarded to the CDC. In 1996–1997, all the viruses that the CDC characterized from the United States as A/H3N2 or B were the same as the vaccine strains (A/Wuhan 359/59 and B/Beijing/184/93) [17]. In 1997–1998, the circulating viruses were primarily A/H3N2/Sydney-like: the only exceptions were 1 of 17 viruses from Minnesota (A/H3N2/Wuhan/359/59-like, the vaccine strain) and 9 of 80 New York viruses also A/Wuhan-like. The CDC characterized the match on the basis of antigenic similarity on a scale of 0+ to 4+ as 1+ for 1997–1998 and 4+ for 1996–1997 [18]

AnalysisBaseline characteristics of vaccinated and unvaccinated subjects at each site were compared by χ² and Student’s t tests for categorical and continuous variables, respectively. Logistic regression (SPSS 7.5 for Windows; SPSS) was used to compare the study outcomes between vaccinated and unvaccinated subjects while controlling for covariates and potential confounders. For the purposes of the analysis, subjects were grouped into 4 mutually exclusive risk groups: healthy <75 years old, healthy ⩾75 years old, high risk <75 years old, and high risk ⩾75 years old. High risk was defined as having any of the diagnoses listed above during the baseline year. In addition, sex, site, and influenza vaccination status were included in all models. Other variables in the final models included having a history of hospitalization for pneumonia during the baseline year (for hospitalization outcomes) or history of any hospitalization during the baseline year (for death outcomes). The criteria for inclusion of variables in the models were clinical relevance, overall contribution to the fit of the model, simplicity, and consistency of fit for all sites for both study years. The final models were selected by use of stepwise regression procedures. Model fit was assessed by using the Hosmer and Lemeshow goodness-of-fit test. Vaccine effectiveness (shown in tables 1–5) was calculated as 1-adjusted odds ratio. We used the odds ratio as an approximation of relative risk since the outcome events were uncommon

In addition to assessing the risk of outcomes for the pooled data, outcomes also were assessed for each site separately. For these site-specific analyses, the models used were the same; however, the variable for the site was excluded

Results

For the first study year, there were 124,582 eligible members from the 3 plans (HP, 40,889; KPNW, 44,539; and OHP, 39,154). For the second study year, there were 161,608 eligible members (HP, 42,251; KPNW, 44,825; and OHP, 74,532). The large increase in OHP subjects was due to substantial growth in the Medicare managed care market. Table 1 shows the pooled baseline characteristics for subjects by year. The vaccination rates were 57.4% for both years. Vaccinated subjects were more likely to have high-risk conditions and to have had a hospitalization during the baseline period than were unvaccinated subjects. Table 1 also shows the distribution of nursing home patients. When nursing home patients were excluded, 122,974 subjects remained in the study cohort for 1996–1997, and 158,454 subjects remained for 1997–1998. These subjects were analyzed further

In the first study year, there were 495 hospitalizations for pneumonia and influenza, 919 deaths during the peak influenza seasons, and 710 hospitalizations and 1332 deaths during the total influenza seasons. These outcome rates varied by site (table 2). During the second year of the study, there were 502 hospitalizations and 852 deaths during the peak influenza seasons and 930 hospitalizations and 1781 deaths during the total influenza seasons, again with some variation in outcome rates by site (table 3). The risk groups successfully categorized subjects into groups with differing risks for the study outcomes: healthy persons 65–74 years old had the lowest risk, and persons ⩾75 years old with high-risk medical conditions had the highest risk for either hospitalization or death in both study years (table 3)

Estimates of vaccine effectiveness for the individual sites differed, particularly for hospitalization for pneumonia and influenza (table 4), and not all the estimates for vaccine effectiveness within individual sites reached statistical significance, especially for reducing hospitalization for pneumonia and influenza during the peak influenza seasons. Among all sites combined, however, influenza vaccination was associated with significant reductions in hospitalizations for pneumonia and influenza and deaths from all causes during the peak and total outbreak periods during both years (table 5). In 1996–1997, the vaccine prevented 19%–20% of hospitalizations due to pneumonia and influenza and 60%–61% of deaths of all causes. In 1997–1998 (when the match was not as good), the vaccine prevented 18%–24% of hospitalizations due to pneumonia and influenza and 35%–39% of deaths due to all causes

Discussion

This study shows that administrative data from 3 different geographically separate HMOs can successfully be pooled to provide timely ongoing assessments of the effectiveness of influenza vaccination among plan members. For both the 1996–1997 and 1997–1998 influenza seasons, influenza vaccination was associated with substantial benefits, including reductions in hospitalizations and deaths among persons ⩾65 years old. This was true in 1996–1997 when the vaccine was a good match for the circulating virus and in 1997–1998 when the A/Sydney5/97 (H3N2) strain first appeared and was antigenically not well matched to the A (H3N2) vaccine virus. Even when the vaccine/circulating virus match was suboptimal, the influenza vaccine was still associated with a reduction in morbidity and mortality. The effectiveness in preventing hospitalization for pneumonia and influenza did not differ during the 2 years, although the vaccine was less effective in preventing deaths in 1997–1998

Point estimates of vaccine effectiveness for total and peak seasons are similar, and the 95% confidence intervals (CIs) largely overlap. This is most likely due to the imprecision of estimating the influenza season based on relatively few isolates. Thus, the isolates may correlate only moderately well with the influenza activity in the community

Past single-site studies in general showed similar results both for hospitalization for presumed influenza or pneumonia of any cause and for death [7–10]. Our study extends previous observations by demonstrating effectiveness in the same season at 3 different sites. As in the present study, Fedson et al. [11] found a greater impact on death than on hospitalization for pneumonia and influenza [11]

Previous studies of influenza vaccine effectiveness fall into several categories. There have been numerous studies in nursing homes during outbreaks. By use of both clinical and serologic data [11–15], they demonstrated from −7% to 65% effectiveness for the vaccine

Studies in other populations also have highlighted the health benefits associated with influenza vaccination of community-dwelling seniors. A placebo-controlled trial from The Netherlands confirmed that vaccination reduced clinical and serologic influenza illness among the ambulatory elderly by 58% [16]. In observational studies from the United States, Canada, and the United Kingdom, vaccination was associated with lower rates of complications from influenza, such as hospitalization for pneumonia and influenza, and from pneumonia, together with other respiratory conditions [10, 11, 19–22]. Others have shown lower death rates from all causes [9, 11, 23, 24]

A few studies have used large linked databases either in non-US government–managed care systems or, in the United States, in large managed care organizations. These studies have used information from large databases about vaccination status, risk factors for severe disease, and clinical outcomes and have assessed the impact of the influenza vaccine on the rate of morbidity and mortality. In general, morbidity has been measured in terms of hospitalizations for pneumonia. Ahmed et al. [25], who reported on the 1989–1990 season in England, did a matched case-control study of deaths due to influenza and found that influenza vaccination reduced mortality by 41% (95% CI, 13%–60%). Nichol et al. [10] found a 50% reduction in hospitalizations and a 35% reduction in mortality among patients immunized against influenza in a study that spanned 3 seasons

In a study that analyzed the cost benefit of the influenza vaccine, Nichol et al. [9] found an average of $117 in direct medical costs saved for each elderly person vaccinated against influenza. Mullooly et al. [19] and Nichol and Goodman [26] assessed the health and economic benefits of influenza vaccine and found that vaccinations reduced morbidity and mortality, which was associated with direct and total cost savings for high-risk seniors 65–74 years old. Two studies found cost saving for non–high-risk elderly [9, 26]; however, the study by Mullooly et al. [19] did not

Another issue in the study of influenza vaccine effectiveness is the antigenic match between the vaccine and circulating influenza viruses in any year. In most years, the vaccine and circulating viruses are well matched. When the match is suboptimal, lower vaccine effectiveness is expected. The suboptimal match in year 2 of our study may help to explain the lower estimate for vaccine effectiveness against death that we observed

In 1994, Sugaya et al. [27] reported on the 1992–1993 influenza season in Japan when 2 strains circulated—an antigenically drifted A and a well-matched B. They found 68% protection against the drifted A (H3N2) and only 44% protection against the well-matched B strain on the basis of laboratory confirmation of influenza illness. More consistent with the results of this study, Mullooly et al. [19], in a study with several years of data during which the vaccine and circulating virus match varied [19], found that protection against morbidity and mortality varied according to the match. Several smaller studies had similar results (i.e., good protection when no antigenic change occurred and fairly good protection in the face of antigenic drift)

Our study has some limitations. All data are from electronic data sets, and some misclassification is likely of vaccination status, risk status, and other variables. Previous studies of both the HP and KPNW data sets showed a misclassification of immunization status of <3%. Even so, the highest rate of misclassification was probably in vaccination status, where incomplete reporting to the plans of patients receiving influenza vaccination in nontraditional sites likely occurred. This would lead to patients being classified as unvaccinated when in fact they had received an influenza vaccination. In all sites, we believe this misclassification was likely <10% (not shown) and similar to misclassification for other variables. Such misclassification would tend to decrease the estimates of vaccine effectiveness. Thus, we believe this is a conservative estimate of the true effectiveness of the vaccine

Because we could not obtain accurate data on influenza immunization in nursing homes, we excluded these patients from the analysis. Not all important variables were controlled (e.g., smoking status and functional status). We were unable to accurately determine the pneumococcal vaccination status of the subjects from the administrative data sets and excluded this from the model. Influenza vaccination is expected to correlate with pneumococcal vaccination, and some of the effect ascribed to influenza vaccination in this study was likely due to pneumococcal vaccination

The use of large linked databases from multiple managed care organizations is a feasible and useful method for assessing vaccine effectiveness. Health plans with large linked databases can be utilized to obtain data rapidly and efficiently that would otherwise take much longer to gather. Several health plans working together can answer similar questions relating to vaccine effectiveness because they can amass large populations with robust data sets. The data dictionary and methods that were developed for this study, with minor modifications, will be used in the future with these and other health plans to continue to monitor influenza vaccine effectiveness in the elderly and in other age groups

This study provides additional information supporting the importance of ensuring that elderly persons receive an influenza vaccination. Up to 61% of the total mortality in this population during the influenza period was prevented, and nearly one-fourth of all hospitalizations for pneumonia of all causes and presumed influenza was prevented. The results strongly support the concept that health plans should not only cover influenza vaccinations but should actively promote vaccinations among their elderly populations