Abstract
Objective In this study, health event capture is broadly defined as the degree to which a group of people use a particular provider network as their primary source of health care services. The Marshfield Epidemiologic Study Area (MESA) is a valuable resource for population-based health research, but the completeness of health event capture has not been validated in recent years. Our objective was to determine the current level of outpatient and inpatient health event capture by Marshfield Clinic (MC) facilities and affiliated hospitals for people living within MESA.
Design A stratified sample survey with strata defined by MESA region (Central or North) and age group (<18 years or ≥18 years).
Setting 24 ZIP codes in central and northern Wisconsin, USA.
Participants 2,485 individuals participated of the 4,313 sampled cohort members residing in MESA Central (N=61,041) and MESA North (N=25,906) on February 22, 2011.
Methods A health care utilization survey was mailed to a random sample stratified by age group and MESA region. Telephone interviews were attempted for nonrespondents. The survey requested information on sources of outpatient care and overnight hospital admissions. Population proportions representing health event capture metrics and corresponding 95% confidence intervals (CI) were estimated with analytic weights applied to account for the survey design.
Results Among those with an outpatient visit during the past 24 months, the most recent visit of an estimated 93% (95% CI, 91% – 94%) was at a MC facility. The most recent admission of an estimated 93% (95% CI, 90% – 96%) of those hospitalized in the past 24 months was at a hospital affiliated with MC. The proportion admitted to MC affiliated hospitals was higher for residents of MESA Central (97%) compared to MESA North (83%).
Conclusion A high proportion of outpatient visits and inpatient admissions in MESA Central and MESA North are accessible in the MC electronic health record. This pattern of high health event capture has been demonstrated since the inception of MESA in 1991. The results from this study validate and support the continued use of MESA for population-based epidemiologic and clinical research.
For over 20 years, the Marshfield Epidemiologic Study Area (MESA) has facilitated population-based health research for the Marshfield Clinic (MC) and collaborating partners.1,2 MESA is a geographic area in central and northern Wisconsin, USA defined by ZIP codes. It is divided into two regions: MESA Central includes 14 ZIP codes surrounding Marshfield, and MESA North includes 10 ZIP codes surrounding MC regional centers in Ladysmith and Park Falls (figure 1). These MESA ZIP codes represent a subset of the MC primary service area where a high proportion of health care utilization among residents has historically been captured in the MC electronic health record (EHR).1–3
MESA is unique in that it is a predominantly rural population-based cohort based on U.S. Census designations, and it is within an area that has been traditionally medically underserved and economically disadvantaged compared to urban or suburban counterparts.4 MESA represents an entire geographic population; in contrast to, for example, a typical managed care population that is defined by insurance type or employment status. In addition, through provision of primary, secondary, and tertiary care, the full clinical spectrum of disease is represented, not only the more severe cases often studied at hospital or referral-based research centers.
MESA has been used in a variety of studies on disease incidence, prevalence, risk factors, and clinical outcomes. From its inception, MESA was designed to support medical effectiveness and outcomes research, making it particularly well suited for comparative effectiveness research.5 To date, there have been over 75 published studies addressing a broad diversity of health issues. In the past 10 years, MESA was instrumental in the establishment of the MC Personalized Medicine Research Project personalized medicine cohort and biobank and the Wisconsin Rural Women’s Health Study cohort.6,7 MESA was essential in supporting the Centers for Disease Control and Prevention (CDC) sponsored annual studies of seasonal influenza vaccines effectiveness and influenza surveillance.8,9 In addition, MESA was the basis for rural population epidemiologic studies of Barrett’s esophagus, asthma in farm children, systemic lupus, and cardiovascular disease, among other health conditions.10–13
MESA differs from other established epidemiologic cohorts in several respects. Though similar in concept and size to the Mayo Clinic Rochester Epidemiology Project (REP), REP represents the population of Olmsted County Minnesota and is largely an urban cohort with less than 20% of members from rural areas.14 The Survey of the Health of Wisconsin (SHOW) began in 2008 and, modelled after the National Health and Nutrition Examination Survey (NHANES), samples over 3,000 Wisconsin adults annually.15,16 SHOW does not have a link to medical records/care data. Other large population-based research cohorts (eg, Framingham Heart Study) tend not to link with medical records and may be limited only to specialized disease states.17
Marshfield Epidemiologic Study Area (MESA) Central and North regions within the Marshfield Clinic (MC) primary service area in Wisconsin, USA.
The MESA cohort is a cumulative residency database designed to identify and track residents in MESA ZIP codes since its establishment in 1991. Membership in this cohort is updated daily based on automated review of MC computerized data systems, thus requiring members to have both an address within a MESA ZIP code and activity within the MC system. It is comprised of current MESA cohort members as well as past MESA cohort members who have moved outside of MESA. Although they remain in the MESA cohort, the follow-up time for the latter group is terminated when they die or move out of MESA. Researchers can link MESA residency with the MC EHR. The MC EHR is an extensive archive of clinical encounters, diagnoses, procedures, medications, and laboratory results from more than 50 MC patient care locations throughout central, northern, and western Wisconsin communities.
The utility of MESA as a resource for population-based health research depends heavily on the degree to which health events are captured in the MC EHR among the region’s residents. This study was conducted to determine the current level of inpatient and outpatient MC health event capture among people residing in MESA on February 22, 2011 according to MC records (referred to in this paper as ‘current MESA cohort members’). Specifically, we aimed to estimate: (1) the proportion of current MESA cohort members whose last outpatient encounter was at a MC facility, (2) how often MC locations are used by current MESA cohort members for outpatient health care, and (3) the proportion of current MESA cohort members whose most recent overnight hospital stay was at a MC affiliated hospital.
Response rates (total surveysa) for health event capture survey within the Marshfield Epidemiologic Study Area
Methods
Design
MESA is a defined geographic area comprised of 24 ZIP codes in central and northern Wisconsin, USA. The sampling frame consisted of people residing in MESA on February 22, 2011 according to MC records (ie, current MESA cohort members). A stratified random sample was selected from the frame where the four sampling strata were defined by each combination of MESA region (Central or North) and age group (<18 years or ≥18 years) on the date the sample was drawn.
Data Collection Procedures
A brief health care utilization survey was mailed to each sampled adult and to the parents of sampled children. The mailing included a cover letter, self-administered survey instrument, and prepaid return envelope. The survey packet was mailed with “return service requested” designation to identify individuals who were no longer living at the MESA address on file. Telephone interviews were attempted for those who did not return the survey within approximately 4 weeks. Each individual was contacted up to eight times over a 2-week period. This included up to one morning and afternoon call each week, and four evening calls.
Respondents were asked to report the timeframe (in the last 24 months, more than 24 months ago, never) of their most recent outpatient encounter and hospital admission (excluding when they were born). They were also asked whether care was provided at a MC facility. Individuals with hospital admissions were asked to select the hospital where they were most recently admitted. A list of hospitals affiliated with MC at the time of the survey and within close proximity to the MESA Central and MESA North regions was provided, in addition to general categories for Wisconsin hospitals and out-of-state hospitals not affiliated with MC. Respondents were also asked to report their frequency of utilizing MC locations for outpatient care using a 4-category scale: always/almost always, more than half the time, less than half the time, never/almost never.
Data from mailed surveys were entered by trained data entry staff into an electronic database. Data collected from telephone interviews were directly entered by interviewers into the electronic database via customized data entry scripts. For quality assurance checking, 10% of mailed surveys were entered in duplicate. The study protocol and all participant documents were approved by the Marshfield Clinic Research Foundation Institutional Review Board.
Sample Size and Analytic Approach
Sample size estimates were computed for each of the four sampling strata assuming known MESA cohort size, a value of 80% for a given metric of MC health event capture, and a desired precision of ±3% for the corresponding 95% confidence intervals (CI).18 Summing the stratum-specific estimates yielded a required total of 2,587 respondents to the survey. With an expected response rate of 60%, the total number of current MESA cohort members included in the sample was 4,313.
Proportions (reported as percentages) and 95% confidence intervals (CI) were estimated with analytic weights applied to account for the survey design. The weights had components for sampling probability, non-response (within strata defined by MESA region, gender and age group [<18, 18–39, 40–59 and ≥60 years]), and a post hoc adjustment for people in the initial sampling frame who were later found not to be current MESA residents. Estimates were generated using PROC SURVEYFREQ in SAS 9.3 (SAS Institute Inc., Cary, NC). Proportions were considered significantly different when their 95% confidence intervals did not overlap.
Results
Of the 4,313 sampled individuals, 2,485 (58%) completed the survey, 610 (14%) refused, and 1,218 (28%) did not respond (figure 2). There were 114 people identified as not being current MESA residents. Most of them had moved away from the area (n=101) or died (n=7) before the survey was mailed, with the remaining individuals being temporary residents (foreign exchange students). Of completed surveys, 48% were received by mail and 52% were obtained by telephone interview. Response rates for the survey ranged from 30% for males aged 18–39 years in MESA North to 79% for females aged 60 years or older in MESA North (table 1). The most common reasons for non-response were ‘unable to contact by phone’ and ‘moved with no new address available’. The most frequent reason given for refusing to participate in the survey was ‘not interested’.
Flow diagram of current MESA cohort members from random selection through initial outpatient and inpatient utilization survey response.
Of current MESA cohort members, 92% (CI: 91%–94%) were estimated to have at least one outpatient visit in the past 24 months. Among those, an estimated 93% (CI: 91%–94%) had their most recent outpatient encounter at a MC facility. This latter percentage was similar by MESA region, age group, and gender (table 2). Of current MESA cohort members with an outpatient visit in the past 24 months, 87% (CI: 86%–89%) were estimated to ‘always’ or ‘almost always’ receive outpatient care at MC facilities, with significantly higher percentages in MESA Central compared to MESA North (89% vs. 83%). The most recent outpatient encounter of an estimated 8% (CI: 6%–9%) of current MESA cohort members occurred more than 24 months before the survey. Both measures of outpatient health event capture were lower for this group without recent outpatient activity (table 2). Specifically, estimates for the demographic groups shown in table 2 were 4.9% to 7.1% lower for the outpatient measure of recent MC care and 8.1% to 16.5% lower for frequency of MC care.
An estimated 16% (CI: 14%–18%) of current MESA cohort members had an overnight hospital admission during the 24 months prior to the survey. Among those, the most recent admission for an estimated 93% (CI: 90%–96%) were at hospitals affiliated with MC (table 3). The percentage admitted to MC affiliated hospitals was significantly higher in MESA Central compared to MESA North (97% vs. 83%). Overnight stays in MESA Central were almost exclusively at Saint Joseph’s Hospital in Marshfield (99%). In MESA North, 85% of overnight stays were distributed among five hospitals with individual percentages for these hospitals ranging from 13% to 25%. The most recent overnight hospital stay of an estimated 45% (CI: 42%–47%) of current MESA cohort members occurred more than 24 months before the survey. Similar to the outpatient results, inpatient health event capture was lower for this group without recent hospital admissions (table 3). Specifically, estimates for the demographic groups shown in table 3 were 9.0% to 17.1% lower. Additional information is available in the Appendix, which includes flow diagrams for each MESA region (figures A1 and A2), and health event capture rates by age group and gender within each MESA region (tables A1 and A2).
Outpatient health event capture within the Marshfield Epidemiologic Study Areaa
Discussion
This study found that Marshfield Clinic continues to be the primary source of outpatient and inpatient care for MESA cohort members. The findings were consistent with previous studies on MESA health care utilization patterns, although direct comparisons are limited due to differences in study designs. One study conducted in the year 2000 indicated that 90% and 86% of outpatient visits in MESA Central and North were captured in the MC system.2 Unpublished validations of hospital discharges versus State of Wisconsin data for 2001–2005 exhibited consistent capture of 94% for MESA Central and 88% to 90% for MESA North. In 2003, another study based on random digit dialing of MESA residents showed 96% and 91% capture of outpatient visits occurring in the previous year in MESA Central and MESA North, respectively. This same study yielded inpatient capture rates of 95% in MESA Central and 87% in MESA North for stays occurring in the previous 2 years. Marshfield Clinic data systems have also exhibited high capture of deaths in MESA. Annual mortality capture for the most recently available 10-year period (2002 to 2011) was 98% to 100% for MESA Central and 95% to 99% for MESA North when compared to data from the State of Wisconsin (unpublished data).
The primary limitations of this study are reliance on self-reported data and potential non-response bias. The results from this study are based on self-reported data. We did not expect the bias associated with self-reported data to be substantial for the outcome measures we examined (in contrast to, for example, a study that required responders to recall specific details of a previous illness or medical encounter). Nevertheless, we attempted to verify health events to the extent possible. Of the 2,136 responders reporting that their most recent outpatient encounter was at an MC facility in the past 24 months, 97% had such an encounter in the MC EHR. The corresponding number for overnight stays was 82% of 286 responders. When attempting to verify overnight stays, we used a strict criterion requiring evidence that the patient was discharged from an MC affiliated hospital. Application of less restrictive criteria that searched the EHR for a broader class of hospital-related documents (some of which could be related to hospital-based outpatient activity) yielded a confirmation rate of 91%. Hence, the true rate of confirmation for overnight stays was between 82% and 91%. The above information supports the assertion that responders had the ability to recall basic facts relating to the receipt of health care. It is not possible to precisely quantify the impact of inaccurate recall by survey responders on estimates of health event capture.
Hospital admission capture within the Marshfield Epidemiologic Study Areaa
A standard assumption in studies employing sample survey methodology is that survey responders are representative of non-responders with respect to the outcome of interest. In this study, non-response bias exists to the extent that, within each of the 16 non-response weighting strata defined by MESA region, gender and age group (<18, 18–39, 40–59 and ≥60 years), those who responded to the survey are not representative of the other MESA cohort members with respect to their health event capture by Marshfield Clinic facilities.
We believe the results from this study are representative of MC health event capture in the general MESA population. MESA data systems were designed to track as many residents as possible within the target ZIP codes, and past validation studies suggest that the MESA Central and MESA North cohorts include at least 96% and 91% of all residents, respectively.1,2 Nevertheless, at a given point in time, the list of current MESA cohort members could be different from those actually residing in the MESA ZIP codes. For this difference to meaningfully distort the heath event capture estimates from this study with respect to their representativeness of the general MESA population, a notable disparity must have existed between the distribution of people in the current MESA cohort and the general MESA population across the subgroups used in the analytic weighting process. To evaluate whether such a disparity was present, we first compared the percentage of people in the current MESA cohort on February 22, 2011 versus the 2010 US Census population in the four primary sampling strata (combinations of MESA region and age [<18, ≥18 years]).19 The absolute differences (MESA percentage minus Census percentage) in the four strata were −2.2%, −0.3%, 0.5%, and 1.9%. We then compared MESA versus Census percentages within the 16 cells used in the nonresponse adjustment. These cells were defined by MESA region, gender, and age (<18, 18–39, 40–59, and ≥60 years). Eight of the 16 cells had absolute differences between −0.5% and 0.5%. The percentage from the MESA cohort exceeded the Census value by more than 0.5% in four cells with a maximum difference of 1.7%. The Census percentage exceeded the value from the MESA cohort by more than 0.5% in four cells with a maximum difference of 1.1%. The small magnitude of the differences in these comparisons support our assertion that estimates from this study are representative of MC health event capture in the general MESA population. The underlying assumption for this assertion is that within the weighting cells, MESA residents who completed the survey are representative of MESA residents in general with respect to health event capture. This assumption would be necessary even if the current MESA cohort were identical to those residents living in the MESA ZIP codes at a given point in time.
As the medical industry continues to consolidate, and reimbursement models emerge that emphasize systems-level accountability for patient outcomes, EHR-based platforms are needed that outline how care systems can systematically construct populations and track their health status over time. The consistently high overall MC capture of medical care in MESA permits computation of population-based estimates of disease incidence and prevalence. Burgeoning models are also being developed within MESA to conduct regional surveillance of ‘upstream’ preventive medical and dental therapies that can subsequently be used to identify gaps in care quality and monitor the longitudinal impact of intervention programs.20,21
Conclusion
A high proportion of outpatient visits and inpatient admissions in MESA Central and MESA North are accessible in the MC EHR. This pattern of high health event capture has been demonstrated since the inception of MESA in 1991. The results from this study validate and support the continued use of MESA for population-based epidemiologic and clinical research.
Acknowledgments
The authors thank the administrative support staff, data entry staff, interviewers, research coordinators and programmer/analysts of the Center for Clinical Epidemiology & Population Health at the Marshfield Clinic Research Foundation for their important contributions to this project.
Appendix
Flow diagram of current MESA Central cohort members from random selection through initial outpatient and inpatient utilization survey response.
Flow diagram of current MESA North cohort members from random selection through initial outpatient and inpatient utilization survey response.
| Region=Central (n=1,236) | Region=North (n=1,067) | ||||
|---|---|---|---|---|---|
| Unweighted frequency | Weighted percent (95% confidence interval) | Unweighted frequency | Weighted percent (95% confidence interval) | ||
| Most recent outpatient visit was at a Marshfield Clinic facility | |||||
| Age group | ≥18 years | 538 | 93.2 (91.0-95.3) | 476 | 89.9 (87.3-92.5) |
| <18 years | 621 | 94.2 (92.4-96.0) | 501 | 94.0 (92.0-96.0) | |
| Gender | Male | 550 | 92.6 (89.9-95.3) | 485 | 90.2 (86.9-93.4) |
| Female | 609 | 94.2 (92.1-96.3) | 492 | 91.1 (88.3-94.0) | |
| Always/almost always seek outpatient care at a Marshfield Clinic facility | |||||
| Age group | ≥18 years | 507 | 88.3 (85.6-91.0) | 431 | 82.0 (78.6-85.4) |
| <18 years | 600 | 91.0 (88.9-93.2) | 463 | 87.4 (84.5-90.2) | |
| Gender | Male | 524 | 87.6 (84.3-91.0) | 437 | 81.2 (76.9-85.5) |
| Female | 583 | 90.1 (87.4-92.8) | 457 | 84.9 (81.3-88.6) | |
Outpatient health event capture by region within the Marshfield Epidemiologic Study Areaa,b
| Region=Central (n=170) | Region=North (n=145) | ||||
|---|---|---|---|---|---|
| Unweighted frequency | Weighted percent (95% confidence interval) | Unweighted frequency | Weighted percent (95% confidence interval) | ||
| Most recent admission was at a hospital affiliated with the Marshfield Clinic | |||||
| Age group | ≥18 years | 119 | 97.1 (94.0-100) | 94 | 82.7 (75.3-90.1) |
| <18 years | 47 | 97.9 (93.7-100) | 26 | 81.2 (67.4-94.9) | |
| Gender | Male | 61 | 97.9 (93.7-100) | 59 | 81.7 (71.3-92.2) |
| Female | 105 | 96.8 (92.9-100) | 61 | 83.5 (74.2-92.7) | |
Hospital admission capture by region within the Marshfield Epidemiologic Study Areaa,b
Footnotes
↵† Current affiliation: Security Health Plan of Wisconsin, Marshfield, Wisconsin, USA
Sources of Support: The project described was supported by the Marshfield Clinic Research Foundation and the Clinical and Translational Science Award (CTSA) program, through the NIH National Center for Advancing Translational Sciences (NCATS), grant UL1TR000427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
- Received April 17, 2014.
- Revision received July 18, 2014.
- Accepted July 24, 2014.
References
- 1↵DeStefanoFEakerEDBrosteSKNordstromDLPeissigPLVierkantRAKonitzerKAGruberRLLaydePM. Epidemiologic research in an integrated regional medical care system: the Marshfield Epidemiologic Study Area. J Clin Epidemiol 1996;49:643–652.
- 2↵GreenleeRT. Measuring disease frequency in the Marshfield Epidemiologic Study Area (MESA). Clin Med Res 2003;1:273–280.
- 3↵GreenleeRTZentnerJKiekeBElliottJMarlengaB. Farm health surveillance in the Marshfield Epidemiologic Study Area: a pilot study. J Agric Saf Health 2005;11:211–218.
- 4↵HartleyD. Rural health disparities, population health, and rural culture. Am J Public Health 2004;94:1675–1678.
- 5↵SoxHCGoodmanSN. The methods of comparative effectiveness research. Annu Rev Public Health 2012;33:425–445.
- 6↵McCartyCAWilkeRAGiampietroPFWesbrookSDCaldwellMD. Marshfield Clinic Personalized Medicine Research Project (PMRP): design, methods and recruitment for a large population-based biobank. Personalized Medicine 2005;2:49–79.
- 7↵ChikaniVRedingDGundersonPMcCartyCA. Psychosocial work characteristics predict cardiovascular disease risk factors and health functioning in rural women: the Wisconsin Rural Women’s Health Study. J Rural Health 2005; 21:295–302.
- 8↵Centers for Disease Control and Prevention (CDC). Interim within-season estimate of the effectiveness of trivalent inactivated influenza vaccine--Marshfield, Wisconsin, 2007–08 influenza season. MMWR Morb Mortal Wkly Rep 2008;57:393–398.
- 9↵BelongiaEAIrvingSAWaringSCColemanLAMeeceJKVandermauseMLindstromSKempfDShayDK. Clinical characteristics and 30-day outcomes for influenza A 2009 (H1N1), 2008–2009 (H1N1), and 2007–2008 (H3N2) infections. JAMA 2010;304:1091–1098.
- 10↵MusanaAKResnickJMTorbeyCFMukeshBNGreenleeRT. Barrett’s esophagus: incidence and prevalence estimates in a rural Mid-Western population. Am J Gastroenterol 2008;103:516–524.
- 11GreenleeRTChyouPHKiekeANaikYGKirkhornS. Prevalence of asthma in a general population cohort of farm children: comparison of estimates based on parental report and medical record review. J Agromedicine 2008; 13:225–231.
- 12NalewayALDavisMEGreenleeRTWilsonDAMcCartyDJ. Epidemiology of systemic lupus erythematosus in rural Wisconsin. Lupus 2005;14:862–866.
- 13↵SharmaPPGreenleeRTAndersonKPChyouPHOsorioHJSmithPNHayesJHVidailletH. Prevalence and mortality of patients with myocardial infarction and reduced left ventricular ejection fraction in a defined community: relation to the second multicenter automatic defibrillator implantation trial. J Interv Card Electrophysiol 2007;19:157–164.
- 14↵RoccaWAYawnBPSt SauverJLGrossardtBRMeltonLJ3rd History of the Rochester Epidemiology Project: half a century of medical records linkage in a US population. Mayo Clin Proc 2012;87:1202–1213.
- 15↵NietoFJPeppardPEEngelmanCDMcElroyJAGalvaoLWFriedmanEMBerschAJMaleckiKC. The Survey of the Health of Wisconsin (SHOW), a novel infrastructure for population health research: rationale and methods. BMC Public Health 2010;10:785.
- 16↵National Health and Nutrition Examination Survey. Centers for Disease Control and Prevention Web site. Available at: http://www.cdc.gov/nhanes. Accessed July 7, 2014.
- 17↵MahmoodSSLevyDVasanRSWangTJ. The Framingham Heart Study and the epidemiology of cardiovascular disease: a historical perspective. Lancet 2014;383:999–1008.
- 18
- 19↵2010 Census Summary File 1 Wisconsin/prepared by the U.S. Census Bureau, 2011.
- 20↵VanWormerJJMillerAWRezkallaSH. Identifying opportunities to improve aspirin utilization for the primary prevention of cardiovascular disease in a regional healthcare system. WMJ 2014;113:190–195; quiz 196.
- 21↵AcharyaAVanWormerJJWaringSCMillerAWFuehrerJTNyczGR. Regional epidemiologic assessment of prevalent periodontitis using an electronic health record system. Am J Epidemiol 2013;177:700–707.
