Abstract
Objective Hospital readmissions are common and expensive. Risk factors for hospital readmission may include vital sign abnormalities (VSA) at the time of discharge. The study aimed to validate VSA at the time of discharge as a useful predictor of hospital readmission within 30 days of discharge. VSA was compared to the validated HOSPITAL score and LACE index readmission risk prediction models.
Design All adult medical patients discharged from internal medicine hospitalist service were studied retrospectively. Variables such as age, gender, diagnoses, vital signs at discharge, 30-day hospital readmission, and components for the HOSPITAL score and LACE index were extracted from the electronic health record for analysis.
Settings A 507-bed university-affiliated tertiary care center.
Participants During the 2-year study period, a cohort of 1,916 discharges for the hospitalist service were evaluated. The final analysis was based on the data from 1,781 hospital discharges that met the inclusion criteria.
Results VSA was found in 13% of the study population. Only one abnormal vital sign was present in a higher proportion readmitted to the hospital within 30 days of discharge. No discharges had three or more unstable vital signs. Receiver operating characteristic (ROC) comparisons of the HOSPITAL score (C statistic of 0.67, P < 0.001), LACE index (C statistic of 0.61, P < 0.001), and VSA (C statistic of 0.52, P = 0.318) indicated that VSA at time of discharge was not a useful predictor of hospital readmission within 30 days of discharge.
Conclusion Our study indicated that VSA at the time of discharge is not a useful predictor of 30-day hospital readmission at a university-affiliated teaching hospital. The more complex and validated HOSPITAL score and LACE index were useful predictors of hospital readmission in this patient population.
In recent years, intense efforts and rigorous risk-adjusted methods have been applied to reduce hospital readmission. Hospital readmissions are frequent, unpredictable, expensive, and associated with poor outcome.1 Nearly 20% of Medicare patients are readmitted to a hospital within 30 days of discharge.2 The Nationwide Readmission Database, which consisted of 2,006 non-federal hospitals in 21 states from January to November 2013, showed that the total cost for readmission was 50.7 billion USD, highest for Medicare (29.6 billion USD), with non-Medicare costs exceeding 21 billion USD. Medicaid was associated with the highest adjusted odds ratio for readmission in those ≥45 years of age.3 The frequency and cost of hospital readmissions have made this problem the target of the Medicare Hospital Readmission Reduction Program (HRRP). The HRRP provides a strong financial incentive to hospitals and health systems because higher than expected readmission rates will result in reduced reimbursement.4 The HRRP’s incentives have encouraged healthcare organizations to invest considerable resources into reducing the rates of hospital readmission.
Risk factors for hospital readmission are numerous and diverse, making it challenging to identify patients at increased risk for readmission so they can be targeted for intervention. Studies have identified many risk factors for hospital readmission within 30 days, including age, race, lack of a regular health care provider, major surgery, medical comorbidities, length of hospital stay, previous admission(s) in the past year, failure to transfer critical information to the outpatient setting, premature hospital discharge, and the higher number of medications prescribed at hospital discharge.4–7 Newer readmission risk models, such as the cancer specific model developed by Schmidt and colleagues8 have acceptable predictive ability (C statistic = 0.70) and are superior to the clinical judgement of health care providers in identifying patients at the highest risk of hospital readmission.8,9
General-purpose hospital readmission risk assessment models are diverse, ranging from in depth multidisciplinary patient interviews to more simplistic screening tool using a few variables.6,10–12 These risk assessment models use factors such as illness severity, hospitalization in the past year, emergency department visits, age, ethnicity, and socioeconomic status to predict hospital readmissions.13 Unfortunately, most readmission risk models perform poorly and have not been validated in multiple care settings.11
The HOSPITAL score and LACE index are two validated and useful readmission prediction models.14,15 The HOSPITAL score uses seven readily available clinical predictors (Hemoglobin level at discharge, Oncology discharge, Sodium level at discharge, Procedure during hospitalization, Index admission, number of hospital Admissions, Length of stay) to accurately identify patients with a higher risk of hospital readmission in the next 30 days. This score has been internationally validated in a population of over 100,000 patients at large academic medical centers (average size 975 beds) and has been shown to have superior discriminative ability over other prediction tools.11,14,16
The LACE index is more complex and uses a combination of administrative and clinical data to assess the risk of hospital readmission.17 The LACE index uses four variables to predict the risk of death or non-elective 30-day readmission: Length of hospital stay, Acuity of admission, medical Comorbidities in the form of a Charlson score, and the number of Emergency department visits in the last 6 months.17 This model has been internally validated using data collected from 4,812 patients discharged from 11 community hospitals in Ontario, and it was externally validated using administrative data collected randomly from 1,000,000 discharges also in Ontario.17 More variable results have been reported elsewhere. The LACE index has been useful and accurate in a study of 26,000 hospital admissions of Medicare patients,13 110,000 discharges from multiple hospitals in the Chicago, Illinois area,18 and 600 patients from a community hospital.19 The LACE index had fair discrimination in a study of 5,800 patients in Singapore20 and poor discrimination for 500 hospital discharges with an average age of 85 years in the UK.21
Direct comparison of the LACE index and HOSPITAL score in a nationwide Medicare sample failed to show a significant difference between the two risk-prediction models.13 In contrast, comparison with HOSPITAL score and LACE index from Denmark22 and Switzerland23 indicates that the HOSPITAL score has superior performance in predicting the risk of hospital readmission.
HOSPITAL Score16
Recent studies indicate that single risk factors, such as the number of prescribed medications at the time of hospital discharge4 or the presence of any abnormal vital signs,24 may be useful predictors of readmission risk. Nguyen and colleagues24 found abnormal vital signs at the time of hospital discharge in 20% of the study population of 33,000 individuals with any diagnosis discharged from a hospitalist service from Northern Texas, USA.24 VSA was defined as an elevated temperature (37.8°C or more), an elevated heart rate (100 beats per minute or more), a high respiratory rate (24 breaths per minute or more), a low systolic blood pressure (90 mmHg or less), or an oxygen saturation of 90% or less.24 Individuals with VSAs were found to have increased risk of death and hospital readmission within 30 days of discharge. The risk of readmission or death was 17% with one abnormal vital sign, 21% with two abnormal vital signs, and 26% with three or more abnormal vital signs. Another study that looked at the association between VSA and adverse clinical outcome in patients admitted with pneumonia showed similar correlations.25 The authors reviewed 680 patients admitted with pneumonia and found that 13.7% of patients discharged with one VSA and 46.2% of those with two or more VSAs were readmitted in 30 days, compared to 10.5% of patients without vital sign abnormalities (P < .003). A study on patients admitted with hip fractures had a higher risk of readmission if they had an active clinical issue at discharge (OR 1.7, CI: 1.2–2.3).26
LACE index20
Simple to use and accurate readmission risk models would be of great utility to many clinicians. The simplicity, ubiquity, and amenability to targeted interventions make VSA an interesting topic for further exploration. This investigation attempted to validate VSA as a readmission risk factor in a different patient care setting and compared performance to the validated HOSPITAL score and LACE-index readmission risk-assessment tools.
Materials and Methods
All adult medical patients discharged from the SIU-School of Medicine (SIU-SOM) Hospitalist service at Memorial Medical Center from January 1, 2015 to January 1, 2017 were studied retrospectively to determine if VSA at the time of discharge was a useful predictor of any cause (avoidable and unavoidable) hospital readmission within 30 days. Exclusion criteria were being transferred to another acute care hospital, leaving the hospital against medical advice, or in-hospital death. The study endpoint was all-cause readmission to the same hospital within 30 days. All-cause readmission was selected because it is the measure used by the Medicare Hospital Readmission Reduction Program.
Memorial Medical Center is a not-for-profit university-affiliated tertiary care center located in Springfield, Illinois, USA with 507 authorized beds. The Southern Illinois University School of Medicine Hospitalist service is the teaching service for the internal medicine residency. Faculty who staff the hospitalist service are board certified or board eligible internal medicine physicians with a hospital-based practice. Patients are generally admitted from the hospital emergency department or transferred from other regional hospitals with acute medical illnesses. Elective hospital admissions are extremely rare for the hospitalist service.
De-identified data for age, gender, vital signs, length of hospital stay, International Classification of Disease (ICD) codes, the number of emergency department (ED) visits in the last 6 months, readmission status at 30 days after discharge, and other information needed to calculate the HOSPITAL score (Table 1) and LACE index (Table 2) were extracted from the electronic health record. The data were de-identified for analysis. Missing laboratory data for the HOSPITAL score were coded in the normal range. The de-identified nature of the data makes accurate linkage to other data sources impossible.
One of the predictors in the HOSPITAL score is admission to an oncology service. However, the study hospital does not have an oncology admitting service. The local practice pattern is to have hospitalists admit oncology patients for inpatient care. To compensate for this local practice, patients with oncology-related ICD codes were considered to have been discharged from an oncology service. As a single center study, readmissions at other hospitals cannot be detected. The HOSPITAL score, Charlson score, and LACE index were calculated for each admission. Vital signs were then classified as normal or abnormal (Table 3) based on the criteria developed by Nguyen and colleagues.24
Institutional review board oversight for this study was from the Springfield Committee for Research Involving Human Subjects. Review determined the study does not to meet the criteria for research involving human subjects.
Vital sign abnormality definitions24
Statistical Analysis
VSA, the HOSPITAL score, and LACE index were investigated as predictors of all-cause hospital readmission. The Pearson Chi2 or Fisher’s exact test were used when evaluating qualitative variables and reported as frequency (%). The Mann–Whitney U or Kruskal–Wallis tests were used to evaluate quantitative variables and reported as the mean ± standard deviation. Variables from univariate analysis with a P value of 0.10 or less were evaluated using multivariate logistic regression with stepwise backwards variable selection (Table 4). SPSS version 22 (SPSS Inc., Chicago, IL, USA) was used for statistical analysis. Two-sided P values < 0.05 were considered significant.
Results
During the study period (2 years), 1916 discharges were recorded for the SIU-SOM Hospitalist service. The analysis included data for the 1781 discharges for 1410 individual patients that met inclusion criteria (Figure 1). Of these discharges, 456 (26%) were readmitted to the same hospital within 30 days. The overall study population was 47% female, had an average age of 63 years, and spent an average of 7.9 days in the hospital. Patients readmitted compared to those not readmitted had more frequent admissions in the past year, evaluations in the emergency department in the last 6 months, higher HOSPITAL scores, and higher LACE index values (Table 5).
Study Flow Diagram
One or more abnormal vital signs were seen in 13% of the study population at the time of discharge. The hospital readmission within 30 days of discharge rate did not differ between the group with any VSA when compared to the group with no VSA (15% vs. 12%, P = 0.084) (Table 5, Figure 2). Multivariate logistic regression analysis also demonstrated non-significance of VSA as a potential risk factor for hospital readmission (Table 4). Only one abnormal vital sign was present in a higher proportion of the study population readmitted to the hospital within 30 days of discharge (14.5% vs. 10.6%, P = 0.024). Two abnormal vital signs were present in the group not readmitted within 30 days of discharge, but this failed to reach statistical significance (1.4% vs. 0.06%, P = 0.196). No discharges had three or more abnormal vital signs.
Receiver operating characteristic (ROC) comparisons of the HOSPITAL score (C statistic of 0.67, 95% CI 0.65 – 0.71), LACE index (C statistic of 0.61, 95% CI 0.58 – 0.64), and VSA (C statistic of 0.52, 95% CI 0.49 – 0.55) are shown in Figure 3. The area under the curve (AUC) is statistically significant for the HOSPITAL Score (P < 0.001) and LACE Index (P < 0.001), but not for VSA (P = 0.318).
Multivariate logistic regression of potential risk factors for hospital readmission within 30 days of discharge
Discussion
We compared the utility of VSA to the established and validated HOSPITAL score and LACE index predictors for all-cause hospital readmission within 30 days of discharge in our study population. We found that VSA is not a significant predictor of hospital readmission in a not-for-profit university-affiliated tertiary care hospital in the Midwestern United States. Significant differences in the prevalence of VSA (13% vs. 20%) and patient demographics exist between our study and the results of Nguyen and colleagues.24 Patients in our study were more likely to be male (53% vs. 46%), had more emergency department visits (1.15 vs. 0.28) and hospital admissions (0.94 vs. 0.31) in the past year, and had a longer hospital length of stay (8 vs. 4 days). It is not clear if these differences reflect fundamental differences, such as severity of illness or access to outpatient medical care, between the studied populations or variations between local practice patterns on the clinical decision of when to discharge a patient from the hospital. The underlying patient-level factors associated with readmission risk may differ between study populations, which could account for the success of some models in specific populations and the lack of success of others as described by Kansagara, et al.11
Currently, hospital readmission is one of the measures used to assess the service provided to the patient and has become an important determinant of the quality of a patient’s outcome. Several variables and multicomponent interventions have helped reduce readmission rates, but readmission models lack predictive ability.27 Kansagara, et al11 described 26 readmission risk prediction models in a variety of settings and populations. No single variable and single component interventions were found to reduce readmissions significantly despite aggressive utilization of substantial resources for planning, implementation, and monitoring.
Multiple studies have examined specific clinical characteristics such as VSA and clinical frailty scale or have used predictive models for higher risk of readmission.28 Patients with psychiatric or substance abuse disorders (24%), sepsis (18%), and congestive heart failure (23%) have the highest readmission rates.3 Therefore, it is crucial to assess the index diagnosis in addition to including the effectiveness and sustainability of several risk factors for readmission. A recent study on 30-day hospital readmissions using Agency for Healthcare Research and Quality Nationwide Readmissions Database demonstrated that the presence of multiple chronic conditions had the highest odds of hospital readmission (for example, 3.67 (3.64 to 3.69) for six or more versus no chronic conditions).29 This study also determined that mental health is one of the most common reasons for index admissions that had high adjusted readmission rates (≥75th percentile). Furthermore, readmission rates vary significantly by type of insurance (private 7.0%, Medicaid 10.1%, and Medicare 16.4%). The authors found that home disposition had a lower readmission rate (9.4%) than patients discharged to home health (17.1%) or post-acute facility care (18.0%) or patients who left the hospital against medical advice (25.4%). As age at index admission increased from 0 to 90+ years, the unadjusted 30-day readmission rate increased from 2.4% to 15.3%. The most significant increase in unadjusted readmission rate (from 7.3% to 14.2%) occurred between ages 30 and 44 years.29 Failure to change care strategies while treating a readmitted patient may also contribute to further readmission.30
Baseline characteristics of the study population by 30 day readmission status
The frequency of vital sign abnormalities at discharge by 30-day readmission status
Most models for risk of readmission are derived from retrospective administrative data and are associated with limited generalizability, and the predictive ability has remained inaccurate. Our single-center retrospective study indicates that the HOSPITAL score and LACE index are superior to VSA at predicting all-cause hospital readmissions within 30 days for a medical hospitalist service at a university-affiliated hospital. The study population contains patients who were admitted more than one time during the study period. The inclusion of these patients is essential for this analysis, because it reflects the criteria used by the Medicare HRRP to assess readmission rates.31 The endpoint of all-cause readmissions is highly relevant, because it is a significant marker of hospital quality under the Medicare program for hospital reimbursement through the HRRP. Under this program, hospitals with high readmission rates can face financial penalties.
Unfortunately, VSA alone is not a sufficiently powerful readmission risk predictor to have utility in this setting. Reliance on more complex but useful risk prediction tools such as the LACE index and HOSPITAL score is essential for identifying patients at increased risk of hospital readmission. The performances of the HOSPITAL score and LACE index in our study were comparable to prior investigations at this center15,32 and validation studies of the HOSPITAL score13,16 and LACE index.13,17
Hospital readmissions are complex and multifactorial, underscored by the evidence presented in a recent meta-analysis showing that no single intervention could prevent readmission.33 Therefore, a combined approach, including the clinical judgment and insights from risk assessment models comprising several variables should guide identification of high-risk candidates for readmission. Readmission models should comprise social, environmental, and demographic determinants such as lack of access to health care, inadequate social support, marginal housing, drug abuse, and poor functional status that all contribute to readmission. The timing of post-discharge follow-up, coordination of care with the primary care physician, and the quality of medication reconciliation are also important attributes associated with readmission risk.34 Overall, patients with multiple comorbidities, advanced age, poor general health, and frailty are at high risk to be readmitted, as demonstrated in multiple studies done in various populations and with different index conditions, time frames, and reasons for readmission.35 Therefore, the inclusion of such factors would improve the predictive ability of models.
Our study shows that VSA alone is not useful to identify patients at high risk of readmission in a community-based tertiary care center. This study has several limitations; it is retrospective, single center, focused on medical patients, has a small sample size, and is shaped by local practice patterns (eg, no oncology admitting service, longer length of hospital stay, more frequent emergency department visits). But our study has opened a discussion for further research to identify readmission risk assessment tools by conducting multicenter studies using public database so the result would be more effective and generalizable over a wide range of patient populations and in different medical practice patterns.
Receiver operating characteristic curves for vital sign abnormalities at discharge, the HOSPITAL score, and LACE index.
Conclusions
The presence of VSA at the time of discharge did not appear to be a useful tool in a community hospital to identify high-risk patients for hospital readmission within 30 days. Repeatability of HOSPITAL score and LACE index reaffirms and further validates the usefulness of these methods in this studied population. Further research is needed to identify readmission risk assessment tools that are easy to implement at the point of care and effective over a wide range of patient populations and medical practice patterns.
- Received August 16, 2018.
- Received January 19, 2019.
- Received April 16, 2019.
- Accepted June 5, 2019.
References
- 1↵ZuckermanRBSheingoldSHOravEJRuhterJEpsteinAM. Readmissions, Observation, and the Hospital Readmissions Reduction Program. N Engl J Med. 2016;374(16):1543 1551.
- 2↵JencksSFWilliamsMVColemanEA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360(14):1418–1428.
- 3↵StromJBKramerDBWangY. Short-term rehospitalization across the spectrum of age and insurance types in the United States. PLoS One. 2017;12(7):e0180767.
- 4↵PickerDHeardKBaileyTCMartinNRLaRossaGNKollefMH. The number of discharge medications predicts thirty-day hospital readmission: a cohort study. BMC Health Serv Res. 2015;15(1):282.
- 5HasanOMeltzerDOShaykevichSA. Hospital readmission in general medicine patients: a prediction model. J Gen Intern Med. 2010;25(3):211–219.
- 6↵SilversteinMDQinHMercerSQFongJHaydarZ. Risk factors for 30-day hospital readmission in patients ≥65 years of age. Baylor University Medical Center Proceedings. 2008;21(4):363–372.
- 7↵AuerbachADKripalaniSVasilevskisEE. Preventability and Causes of Readmissions in a National Cohort of General Medicine Patients. JAMA Intern Med. 2016;176(4):484 493.
- 8↵SchmidtCRHefnerJMcAlearneyAS. Development and prospective validation of a model estimating risk of readmission in cancer patients. J Surg Oncol. 2018;117(6):1113 1118.
- 9↵AllaudeenNSchnipperJLOravEJWachterRMVidyarthiAR. Inability of providers to predict unplanned readmissions. J Gen Intern Med. 2011;26(7):771–776.
- 10↵ZhouHDellaPRRobertsPGohLDhaliwalSS. Utility of models to predict 28-day or 30-day unplanned hospital readmissions: an updated systematic review. BMJ Open. 2016;6(6):e011060.
- 11↵KansagaraDEnglanderHSalanitroA. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306(15):1688–1698.
- 12↵SmithDMGiobbie-HurderAWeinbergerM; Department of Veterans Affairs Cooperative Study Group on Primary Care and Readmissions Predicting non-elective hospital readmissions. J Clin Epidemiol. 2000;53(11):1113–1118.
- 13↵GarrisonGMRobeliaPMPecinaJLDawsonNL. Comparing performance of 30-day readmission risk classifiers among hospitalized primary care patients. J Eval Clin Pract. 2017;23(3):524–529.
- 14↵DonzéJAujeskyDWilliamsDSchnipperJL. Potentially avoidable 30-day hospital readmissions in medical patients: derivation and validation of a prediction model. JAMA Intern Med. 2013;173(8):632–638.
- 15↵RobinsonR. The HOSPITAL score as a predictor of 30 day readmission in a retrospective study at a university affiliated community hospital. PeerJ. 2016;4:e2441.
- 16↵DonzéJDWilliamsMVRobinsonEJ. International Validity of the HOSPITAL Score to Predict 30-Day Potentially Avoidable Hospital Readmissions. JAMA Intern Med. 2016;176(4):496–502.
- 17↵van WalravenCDhallaIABellC. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ. 2010;182(6):551–557.
- 18↵TongLErdmannCDaldalianMLiJEspositoT. Comparison of predictive modeling approaches for 30-day all-cause non-elective readmission risk. BMC Med Res Methodol. 2016;16:26.
- 19↵SpivaLHandMVanBrackleLMcVayF. Validation of a Predictive Model to Identify Patients at High Risk for Hospital Readmission. J Healthc Qual. 2016;38(1):34–41.
- 20↵LowLLLeeKHHock OngME. Predicting 30-Day Readmissions: Performance of the LACE Index Compared with a Regression Model among General Medicine Patients in Singapore. BioMed Res Int. 2015;2015:169870.
- 21↵CotterPEBhallaVKWallisSJBiramRW. Predicting readmissions: poor performance of the LACE index in an older UK population. Age Ageing. 2012;41(6):784–789.
- 22↵CooksleyTNanayakkaraPWBNickelCH. Readmissions of medical patients: an external validation of two existing prediction scores. QJM. 2016;109(4):245–248.
- 23↵AubertCEFollyAMancinettiMHayozDDonzéJ. Prospective validation and adaptation of the HOSPITAL score to predict high risk of unplanned readmission of medical patients. Swiss Med Wkly. 2016;146:w14335.
- 24↵NguyenOKMakamANClarkC. Vital Signs Are Still Vital: Instability on Discharge and the Risk of Post-Discharge Adverse Outcomes. J Gen Intern Med. 2017;32(1):42–48.
- 25↵HalmEAFineMJKapoorWNSingerDEMarrieTJSiuAL. Instability on hospital discharge and the risk of adverse outcomes in patients with pneumonia. Arch Intern Med. 2002;162(11):1278–1284.
- 26↵HalmEAMagazinerJHannanEL. Frequency and impact of active clinical issues and new impairments on hospital discharge in patients with hip fracture. Arch Intern Med. 2003;163(1):108–113.
- 27↵KripalaniSTheobaldCNAnctilBVasilevskisEE. Reducing hospital readmission rates: current strategies and future directions. Annu Rev Med. 2014;65(1):471–485.
- 28↵KahlonSPedersonJMajumdarSR. Association between frailty and 30-day outcomes after discharge from hospital. CMAJ. 2015;187(11):799–804.
- 29↵BerryJGGayJCJoynt MaddoxK. Age trends in 30 day hospital readmissions: US national retrospective analysis. BMJ. 2018;360:k497.
- 30↵PennathurPRAyresBS. A qualitative investigation of healthcare workers’ strategies in response to readmissions. BMC Health Serv Res. 2018;18(1):138.
- 31↵CMS.gov. Centers for Medicare and Medicaid Services. Hospital Readmissions Reduction Program (HRRP). Available at: https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. Accessed March 30, 2018.
- 32↵RobinsonRHudaliT. The HOSPITAL score and LACE index as predictors of 30 day readmission in a retrospective study at a university-affiliated community hospital. PeerJ. 2017;5:e3137.
- 33↵LeppinALGionfriddoMRKesslerM. Preventing 30-day hospital readmissions: a systematic review and meta-analysis of randomized trials. JAMA Intern Med. 2014;174(7):1095–1107.
- 34↵KripalaniSLeFevreFPhillipsCOWilliamsMVBasaviahPBakerDW. Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care. JAMA. 2007;297(8):831–841.
- 35↵VestJRGammLDOxfordBAGonzalezMISlawsonKM. Determinants of preventable readmissions in the United States: a systematic review. Implement Sci. 2010;5:88.
