Abstract
Background/Aims Early detection of Heart Failure (HF) could help change the natural history of one of the most costly and disabling conditions among older patients through early intervention with treatments. We describe natural language processing (NLP) methods and validation results for detection of major and minor Framingham criteria in electronic health record (EHR) clinical notes as a first step towards early detection of HF.
Methods Data for this study were from 6,355 primary care incident heart failure cases identified using Geisinger EHR data between 2001 and 2010 and from 26,052 controls. Text data were from Office Visit, Case Manager, and Radiology notes in the four-year period before diagnosis of HF in cases and a comparable period for controls. Text analysis comprised:
-
application-independent analytics for paragraph and sentence identification, tokenization, dictionary look-up, morphological analysis, and part-of-speech tagging;
-
dictionaries and grammars for 9 major and 6 minor Framingham criteria, and for the compound noun phrases, negated contexts, and counterfactual contexts in which criteria are mentioned; and
-
text analysis engines to disambiguate criteria mentions, to apply constraint rules (e.g., word co-occurrence), and to decide whether identified signs and symptoms are being affirmed or denied.
During training, NLP results were iteratively validated against the judgments of a cardiologist. The final system was tested against gold standard annotations reflecting the consensus judgments of four trained coding experts.
Results Over an 8-week period of iterative training, the NLP results on a dataset of 65 encounter notes improved from a positive predictive value (PPV) of 0.741 and a sensitivity (Se) of 0.504 to a final PPV of 0.931 and a Se of 0.939. On the consensus gold standard dataset of 200 cases and 200 controls, the fully-trained system achieved a PPV of 0.925 and a Se of 0.896.
Discussion The NLP system can accurately identify affirmations and denials of Framingham criteria, which can directly reduce physicians workload for HF-related chart reviews and for possible early detection of HF. In addition, the integrated NLP components and our validation methodology can be adapted for application beyond HF.
