Abstract
Introduction Immunizing the population is a vital public health priority. This article describes a resident-led continuous quality improvement project to improve the immunization rates of children under 3 years of age at two urban family medicine residency clinics in Salt Lake City, Utah, as well as a break-even cost analysis to the clinics for the intervention.
Methods Immunization records were distributed to provider-medical assistant teamlets daily for each pediatric patient scheduled in clinic to decrease missed opportunities. An outreach intervention by letter, followed by telephone call reminders, was conducted to reach children under 3 years of age who were behind on recommended immunizations for age (total n=457; those behind on immunizations n=101). Immunization rates were monitored at 3 months following start of intervention. A break-even analysis to the clinics for the outreach intervention was performed.
Results Immunizations were improved from a baseline of 75.1% (n=133) and 79.6% (n=223) at the two clinics to 92.1% (n=163) and 89.6% (n=251), respectively, at 3 months following the start of intervention (P<0.01). The average revenue per immunization given was $81.57. The financial break-even point required 36 immunizations to be administered.
Conclusion Significant improvement in the immunization rate of patients under 3 years of age at two family medicine residency training clinics was achieved through decreasing missed opportunities for immunization in clinic, and with outreach through letters and follow-up phone calls. The intervention showed positive revenue to both clinics.
Immunizations are an important public health and primary care priority, leading to billions of dollars in savings to society and over 100 million infectious cases prevented in the United States alone.1,2 Accordingly, Healthy People 2010 and 2020 have made increasing the percentage of those immunized a priority, with a stated national goal of 80% of children aged 19–35 months having received all of their recommended immunizations.3
Two of the main causes of poor immunization rates in a medical practice are missed opportunities by clinicians for immunization, and patients not coming in for appointments.4 Many approaches have been taken to address low immunization rates from these causes. Audit and feedback mechanisms, whereby clinicians are given constant information regarding the immunization rates of their patients, along with electronic medical record (EMR) reminders and templates, clinic-wide standing orders, and vaccine delivery protocols have all been shown to improve immunizations through decreased missed opportunities.5–7 Use of patient registries, automated patient reminders, reminder-recall systems, and home visits have shown to be effective at immunizing those who do not consistently seek medical care.8–10 Telephone calls and home visits have proven to be the most effective but are also the most costly.11,12 Multiple studies have evaluated cost-effective ratios on multicomponent interventions for childhood immunizations,13,14 but none were found that provided a break-even cost analysis for the clinic to implement such quality improvement measures.
Physician leadership in continuous quality improvement (CQI) is fundamental in the success of these interventions,15 underscoring the importance of training resident physicians in CQI principles.16,17 The University of Utah Family Medicine Residency Program has implemented a novel experiential CQI curriculum for residents, which includes each 3rd year resident leading a team of providers and staff from the two residency clinics.18 The curriculum also includes protected time for CQI teams to meet monthly, and didactic training for the 3rd year residents in the CQI FOCUS-PDSA methodology.19 One of the resident-led CQI projects performed during academic year 2011–12 at the Madsen Family Health Center (MHC) showed a significant improvement in the number of children under 2 years of age receiving recommended immunizations through a decrease in missed opportunities by distributing immunization records of all scheduled pediatric patients to provider-medical assistant (MA) teamlets.20 A second residency clinic, the Sugarhouse Family Health Center (SHC), was not included in the initial intervention in 2011–12. Despite this intervention, many patients were still not receiving care at recommended intervals and were thus falling behind on immunizations; these patients would thus not be impacted by the point-of-care intervention to decrease missed opportunities. The rates of recommended immunizations for children under 3 years of age at MHC and SHC in November 2012 were 75.1% (n=133) and 79.6% (n= 223), respectively. This is compared to the national rate of 68.4% and Utah rate of 73% during the same time period.21
The objective of this project was to further improve vaccination rates of children under 3 years of age by expanding the point-of-care intervention to a second clinic, as well as adding an outreach component to impact patients not seeking medical care. A break-even analysis will be performed to evaluate how many patients need to be immunized to provide a financial benefit to the clinic. The goal of 90% of children receiving recommended vaccinations for their age was established by the CQI team. The age range was chosen to coincide with Healthy People 2020 goals.
Methods
Setting
The MHC and SHC are urban University of Utah Family Medicine Residency Program teaching clinics in Salt Lake City, Utah. The two clinics provide over 20,000 patient visits to individuals of all ages and demographics, with over 10% being under 18 years of age. Approximately 3% of the two clinics population is under 3 years of age, with 58% of those under 3 years of age being male.
Pediatric Immunization Continuous Quality Improvement, 2012–13
Similar to other resident CQI projects, a multidisciplinary team was assembled that included the 3rd year resident team leader (author CS), a faculty mentor (author KJ), a second year resident, a licensed practical nurse, and multiple medical assistants. The team included representation from both clinics, as both would be participating in the intervention. During initial team meetings, members identified barriers to achieving full immunization for children under 3 years of age, and brainstormed interventions that could be undertaken by clinic staff to improve immunization rates. Children were considered under-immunized if they had not received all recommended immunizations for their age. The 2011–12 intervention performed at MHC of distributing immunization records for all scheduled pediatric patients to provider-MA teamlets was implemented at the SHC to decrease missed opportunities. The chosen intervention for outreach at both clinics was to mail a letter to parents/guardians of under-immunized children under 3 years of age that provided information on reasons for immunization and encouraged them to make an appointment to obtain missing immunizations for their child. About 6 weeks after mailing outreach letters, MAs made follow-up phone calls to the parents/guardians of children who had not yet received catch-up immunizations in the 4:3:1:3:3:1:3 (4DTaP, 3Polio, 1MMR, 3HIB, 3HepB, 1Var, 3PCV) series.
Baseline rates of clinic children who were not yet 3 years of age, as well as under-immunized children, were identified for both clinics using Utah Statewide Immunization Information System (USIIS) reports. In following the Core Set of Children’s Health Care Quality Measures for Medicaid and CHIP, patients who had not been seen by a medical provider at the clinic within the last year were deemed to no longer be a part of the clinic and were removed from the intervention.22,23 Outreach letters were sent the last week of November 2012. New children who had entered the target age group or the clinic population were similarly identified 1 month later who were not part of the November cohort, and a letter was sent in the second week of January 2013. Follow-up phone calls to those still behind were made 6 weeks after the sending the outreach letters. Repeat phone calls were made until the patient scheduled an appointment or a message was left to call the clinic. We hypothesized that most families would not respond to the letter and would require a phone call before updating the immunizations. Follow up USIIS reports of clinic-wide immunization rates, as well as records of the targeted patients, were generated at 6 weeks and 3 months after mailing of letters to follow up the impact of the interventions. Approval from the University of Utah Internal Review Board (IRB) was obtained for this project.
Education
The resident leader and faculty mentor were present for each of the monthly CQI meetings for the academic year 2012–13. Each team member contributed to the project and assisted with problem-solving throughout its duration. Continuous support was provided by the clinical faculty mentor for the project, as well as the dedicated educational faculty supervisor for the curriculum and didactics.
Resource Assessment
The resources required to complete this project were catalogued, and costs for each resource were documented. Two categories of intervention cost were considered to determine the break-even point: variable and fixed costs. The variable costs consisted of expenses that were intervention specific (alcohol preps, vaccines, syringe, bandages). Fixed intervention costs were the cost of personnel (for planning and management of project, analysis of reports, mailings, and phone calls) and supplies (letterhead, envelopes, postage) required to implement the intervention. The clinics also had non-intervention fixed costs, such as electricity, rent, and phone lines, which were not considered within the analysis, because they were in place before the projects’ inception. Immunizations were identified as resulting from the intervention based on the scheduling of the appointment after the reminder letter or call was received. Cost (variable plus fixed costs) per completed immunization and per immunized child was estimated. Revenue was defined as the reimbursement for vaccines, vaccine administration, and office visits based on Utah Medicaid rates.22 Break-even analysis identified the point at which total cost and total revenue were equal for the outreach portion of the project.
Data Analysis
The individuals immunized before and after the intervention were compared using a test of proportions, with significance level alpha of 0.05. Analysis was conducted in Stata 11.2 (College Station, TX). A break-even analysis was performed to determine the cost of the project and the relationship between the fixed and variable costs versus the revenue received. These values were calculated both per immunization as well as per patient (figure 1).
Results
Immunizations
The average age of the cohort was 15 months, and of the 457 children less than 3 years of age, 265 (58%) were male. Daily immunization reports were given to the providers at both clinics. The baseline immunization proportion for the 4:3:1:3:3:1:3 vaccine series in children less than 3 years of age at MHC (figure 2) and SHC (figure 3) as of August 2012 were 67.9% and 67.4%, respectively. August rates were used due to the CQI curriculum timeline to plan the intervention. During the process, a number of patients were identified as no longer receiving care from these clinics: 9.7% (n=19) patients from MHC and 15.4% (n=51) from SHC. The recalculated baseline rates, after removing these patients, were 75.1% (MHC) and 79.6% (SHC). DTAP was the most common vaccine missing, with 76% of children behind on age-recommended doses. Of the remaining children who still identified as patients of MHC and SHC, 3.9% (n=7) and 2.1% (n=6) received immunization at routine clinic visits, respectively, without experiencing the outreach portion of the intervention. A total of 37 letters were sent out to MHC patients (31 in November of 2012 and 6 in January of 2013), while 51 SHC patients received a letter (42 in November of 2012 and 9 in January of 2013). Of those unimmunized, 41% (n=15) and 37% (n=19) subsequently received their immunization following a letter, and another 36% (n=8) and 9% (n=3) of those remaining ultimately were immunized after telephone communication. Of the 54 individuals called from both clinics, 12 had phone numbers that had been disconnected or were incorrect, while 15 required more than one phone call to establish contact. Data analysis in April 2013 showed overall proportion immunized rising to 92.1% (MHC) and 89.6% (SHC) (P<0.01 for both tests of proportions compared to the recalculated immunization rate at baseline). After the intervention, 43 children were left unvaccinated, with 29 being male and 14 female. Children received all missing immunizations at a single visit.
Break-even Analysis for Outreach Intervention
Total personnel time was 38.5 hours to complete the project. Most of this time came from MA time for intervention (20 hours) and time for team meetings (7.5 hours), with the remainder attributed to the resident leader and faculty mentor leading the intervention and monitoring the progress throughout the project.
Including both variable and fixed costs, the total cost of the project was $9,237.72. Total revenue, including reimbursement for vaccines, vaccine administration, and office visits, was $12,072.14. The average revenue per vaccine administered was $81.57, the average revenue per patient whose immunizations were updated was $268.27. Forty-five patients received 148 vaccines through the project, with each patient receiving an average of 3.29 vaccines.
Variable cost per immunization was $56.31, while variable cost per patient, assuming each patient received 3.29 vaccines, was $185.27, with a fixed cost of $900.65. Thus, 36 immunizations, or 11 patients, were needed to meet the break-even point ($2,911.05) where the costs and revenue were equal. Because 45 patients were updated on their immunizations, the clinics surpassed the break-even point and not only were able to cover the cost of the project but also generate additional revenue.
Discussion
The audit and feedback method of decreasing missed opportunities by providing immunization history and schedules to provider-MA teamlets is a relatively simple way to improve immunizations.5–7 The addition of using a patient registry to inform outreach for those patients not coming to clinic has also been shown effective.8–10 This study demonstrates that both methods used together significantly improve immunization rates in young children. This study also demonstrates the cost for a clinic to perform these activities regardless of the reimbursement model. Relatively few vaccines (n=36) would need to be administered to create a profit in a fee-for-service model. This information is crucial to know for implementation in varied and changing health care payment models. This study also illustrates that a family medicine resident can lead a CQI team and positively impact community health.
Break-even analysis for (A) number of immunizations administered and (B) per patient immunized.
Pre and post-intervention immunization rates at Madsen Health Center (MHC).
The authors set up the intervention with both letters and phone calls under the hypothesis that not many people would respond to the letters, and further personal contact would be needed for patients to show up. We were, therefore, surprised to find that letters were more productive than phone calls in bringing patients in for immunizations. One of the clinics even showed a much lower success rate with telephone calls than the other (figures 2 and 3). There were multiple wrong phone numbers among those called, which may imply that the address was also not accurate, and those who did not come in for immunization had moved and received neither the letter nor the phone call, but this was not clear.
Patients contacted were encouraged to make an appointment with their provider as opposed to a nursing visit strictly for immunizations. This was decided upon because many of these patients were due for well-child checks in addition to their immunizations, and any questions or concerns from the parents could be discussed with the provider at the same time. There were also certain local insurances that would not pay for immunizations given in a nursing visit, only providing reimbursement for vaccines given in a provider visit. In other areas or environments, this may not be a problem, and would change the cost-benefit analysis if immunizations were given strictly in a nursing visit. This can also impact access and the time frame needed to update immunizations, depending upon how quickly an established patient can get in to see their provider.
A majority of the patients who received immunizations post-intervention were on Medicaid. However, for ease of analysis, Utah Medicaid rates for reimbursement were utilized for all patients, despite it providing the lowest reimbursement among insurances in Utah. As this demonstrated a profit to the clinic with Medicaid, reimbursement from private insurers would provide even more revenue to clinics who utilize this approach.
Pre and post-intervention immunization rates at Sugarhouse Health Center (SHC).
There are multiple limitations to our study. The break-even analysis included only those immunizations updated based on the outreach intervention and not the audit/feedback portion of the project. As this was intended to be a clinical CQI project, a control group was not used, though a control group for comparison would have provided a preferable study design for research purposes. USIIS encourages every clinic in the state to document immunizations, although this was not always documented, and extra time was spent by the resident searching through the clinic’s EMR to ensure accurate immunization data for every patient. Our particular curriculum and clinic directors allow for a half day each month of clinic closure to assure time for CQI meetings (among other activities), whereas this may not be feasible for all clinics. This also represents a short time interval to follow, and longer term reporting is needed to ensure continuation of the improved proportion of immunizations. The revenue figures in this project may not be fully generalizable to different situations and locales around the United States.
Acknowledgments
The authors would like to acknowledge Susan Pohl, MD, Jennifer Leiser, MD, Sonja Van Hala, MD, MPH, Bernadette Kiraly, MD, and Richard Backman, MD for their contributions and support to the educational curriculum in general, and this CQI project in particular.
Footnotes
Financial Support: This study was funded through a Health Studies Fund from the University of Utah Department of Family and Preventive Medicine.
- Received March 10, 2014.
- Revision received June 6, 2014.
- Accepted June 13, 2014.
References
- 1↵ZhouFSheferAWengerJMessonnierMWangLYLopezAMooreMMurphyTVCorteseMRodewaldL. Economic evaluation of the routine childhood immunization program in the United States, 2009. Pediatrics 2014;133:577–585.
- 2↵van PanhuisWGGrefenstetteJJungSYChokNSCrossAEngHLeeBYZadorozhnyVBrownSCummingsDBurkeDS. Contagious diseases in the United States from 1888 to the present. N Engl J Med 2013;369:2152–2158.
- 3↵Healthy People 2020. Immunizations and Infectious Diseases Objectives. Available at: https://www.healthypeople.gov/2020/topics-objectives/topic/immunization-and-infectious-diseases. Accessed March 7, 2014.
- 4↵EpsteinD. High immunization rates versus missed immunization opportunities in a private pediatric office. Del Med J 1998;70:361–366.
- 5↵BordleyWCChelminskiAMargolisPAKrausRSzilagyiPGVannJJ. The effect of audit and feedback on immunization delivery: a systemic review. Am J Prev Med 2000; 18:343–350.
- 6MelinkovichPHammerAStaudenmaierABergM. Improving pediatric immunization rates in a safety-net delivery system. Jt Comm J Qual Patient Saf 2007;33:205–210.
- 7↵AuLOsterAYehGHMagnoJPaekHM. Utilizing an electronic health record system to improve vaccination coverage in children. Appl Clin Inform 2010;1:221–231.
- 8↵AlemiFAlemagnoSAGoldhagenJAshLFinkelsteinBLavinAButtsJGhadiriA. Computer reminders improve on-time immunization rates. Med Care 1996;34:OS45OS51.
- 9GaglaniMRiggsMKamenickyCGlezenWP. A computerized reminder strategy for annual influenza immunization of children with asthma or reactive airway disease. Pediatr Infect Dis J 2001;20:1155–1160.
- 10↵HambridgeSJPhibbsSLChandramouliVFaircloughDSteinerJF. A stepped intervention increases well-child care and immunization rates in a disadvantaged population. Pediatrics 2009;124:455–464.
- 11↵SzilagyiPGBordleyCVannJCChelminskiAKrausRMMargolisPARodewaldLE. Effect of patient reminder/recall interventions on immunization rates: a review. JAMA 2000;284:1820–1827.
- 12↵VoraSVerberLPottsSDozierTDaumRS. Effect of a novel birth intervention and reminder-recall on on-time immunization compliance in high-risk children. Hum Vaccin 2009;5:395–402.
- 13↵LieuTABlackSBRayPSchwalbeJALewisEMLavetterAMorozumiPAShinefieldHR. Computer-generated recall letters for underimmunized children: how cost-effective? Pediatr Infect Dis J 1997;16:28–33.
- 14↵LieuTACapraAMMakolJBlackSBShinefieldHR. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunized children in a health maintenance organization. Pediatrics 1998;101:E3.
- 15↵SinnJSMorrowALFinchAB. Improving immunization rates in private pediatric practices through physician leadership. Arch Pediatr Adolesc Med 1999;153:597–603.
- 16↵PatowCA. Making residents visible in quality improvement. Acad Med 2009;84:1642.
- 17↵WeissKBWagnerRNascaTJ. Development, Testing, and Implementation of the ACGME Clinical Learning Environment Review (CLER) Program. J Grad Med Educ 2012;4:396398.
- 18↵Van HalaSBackmanRJonesKBKiralyBLeiserJ. Continuous Quality Improvement (CQI) in a Family Medicine Residency Program: Leveraging Three Missions. Presented at the 2013 Society of Teachers of Family Medicine Annual Spring Conference, Baltimore, MD, May 1–5, 2013.
- 19↵Center for Medicare and Medicaid Management. FOCUS-PDSA Roadmap. Available at: http://www.health.state.mn.us/divs/fpc/cww/B01_PDSA.pdf. Accessed March 7, 2014.
- 20↵JonesKBGrenLHBackmanR. Improving pediatric immunization rates: description of a resident-led clinical continuous quality improvement project. Fam Med 2014;46:631–635.
- 21↵Centers for Disease Control and Prevention. National, state, and local area vaccination coverage among children aged 19–35 months—United States, 2012. MMWR Morb Mortal Wkly Rep 2013;62:733–740.
- 22↵DardenPMTaylorJASloraEJHasemeierCMAsmussenLRecknorJCWassermanRC. Methodological issues on determining rates of childhood immunization in office practice. Arch Pediatr Adolesc Med 1996;150:1027–1031.
- 23↵Centers for Medicare and Medicaid Services. Initial Core Set of Children’s Health Care Quality Measures: Technical Specifications and Resource Manual for Federal Fiscal Year 2012 Reporting. November 2012. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Downloads/InitialCoreSetResourceManual.pdf. Accessed May 29, 2014.
- 24Utah Medicaid Program. Coverage and Reimbursement Code Lookup. Available at: http://health.utah.gov/medicaid/stplan/lookup/CoverageLookup.php. Accessed March 7, 2014.
