Dataset Information

Novel methodology to measure pre-procedure antimicrobial prophylaxis: integrating text searches with structured data from the Veterans Health Administration's electronic medical record.

ABSTRACT: BACKGROUND:Antimicrobial prophylaxis is an evidence-proven strategy for reducing procedure-related infections; however, measuring this key quality metric typically requires manual review, due to the way antimicrobial prophylaxis is documented in the electronic medical record (EMR). Our objective was to electronically measure compliance with antimicrobial prophylaxis using both structured and unstructured data from the Veterans Health Administration (VA) EMR. We developed this methodology for cardiac device implantation procedures. METHODS:With clinician input and review of clinical guidelines, we developed a list of antimicrobial names recommended for the prevention of cardiac device infection. We trained the algorithm using existing fiscal year (FY) 2008-15 data from the VA Clinical Assessment Reporting and Tracking-Electrophysiology (CART-EP), which contains manually determined information about antimicrobial prophylaxis. We merged CART-EP data with EMR data and programmed statistical software to flag an antimicrobial orders or drug fills from structured data fields in the EMR and hits on text string searches of antimicrobial names documented in clinician's notes. We iteratively tested combinations of these data elements to optimize an algorithm to accurately classify antimicrobial use. The final algorithm was validated in a national cohort of VA cardiac device procedures from FY2016-2017. Discordant cases underwent expert manual review to identify reasons for algorithm misclassification. RESULTS:The CART-EP dataset included 2102 procedures at 38 VA facilities with manually identified antimicrobial prophylaxis in 2056 cases (97.8%). The final algorithm combining structured EMR fields and text note search results correctly classified 2048 of the CART-EP cases (97.4%). In the validation sample, the algorithm measured compliance with antimicrobial prophylaxis in 16,606 of 18,903 cardiac device procedures (87.8%). Misclassification was due to EMR documentation issues, such as antimicrobial prophylaxis documented only in hand-written clinician notes in a format that cannot be electronically searched. CONCLUSIONS:We developed a methodology with high accuracy to measure guideline concordant use of antimicrobial prophylaxis before cardiac device procedures using data fields present in modern EMRs. This method can replace manual review in quality measurement in the VA and other healthcare systems with EMRs; further, this method could be adapted to measure compliance in other procedural areas where antimicrobial prophylaxis is recommended.

SUBMITTER: Mull HJ

PROVIDER: S-EPMC6993312 | biostudies-literature | 2020 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Novel methodology to measure pre-procedure antimicrobial prophylaxis: integrating text searches with structured data from the Veterans Health Administration's electronic medical record.

Mull Hillary J HJ Stolzmann Kelly K Kalver Emily E Shin Marlena H MH Schweizer Marin L ML Asundi Archana A Mehta Payal P Stanislawski Maggie M Branch-Elliman Westyn W

BMC medical informatics and decision making 20200130 1

<h4>Background</h4>Antimicrobial prophylaxis is an evidence-proven strategy for reducing procedure-related infections; however, measuring this key quality metric typically requires manual review, due to the way antimicrobial prophylaxis is documented in the electronic medical record (EMR). Our objective was to electronically measure compliance with antimicrobial prophylaxis using both structured and unstructured data from the Veterans Health Administration (VA) EMR. We developed this methodology ...[more]

PMID: 32000780

Similar Datasets

Project description:ImportanceHealth care-associated infections (HAIs) are preventable, harmful, and costly; however, few resources are dedicated to infection surveillance of nonsurgical procedures, particularly cardiovascular implantable electronic device (CIED) procedures.ObjectiveTo develop a method that includes text mining of electronic clinical notes to reliably and efficiently measure HAIs for CIED procedures.Design, setting, and participantsIn this multicenter, national cohort study using electronic medical record data for patients undergoing CIED procedures in Veterans Health Administration (VA) facilities for fiscal years (FYs) 2016 and 2017, an algorithm to flag cases with a true CIED-related infection based on structured (eg, microbiology orders, vital signs) and free text diagnostic and therapeutic data (eg, procedure notes, discharge summaries, microbiology results) was developed and validated. Procedure data were divided into development and validation data sets. Criterion validity (ie, positive predictive validity [PPV], sensitivity, and specificity) was assessed via criterion-standard manual medical record review.ExposuresCIED procedure.Main outcomes and measuresThe concordance between medical record review and the study algorithm with respect to the presence or absence of a CIED infection. CIED infection in the algorithm included 90-day mortality, congestive heart failure and nonmetastatic tumor comorbidities, CIED or surgical site infection International Statistical Classification of Diseases and Related Health Problems, Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis codes, antibiotic treatment of Staphylococci, a microbiology test of a cardiac specimen, and text documentation of infection in specific clinical notes (eg, cardiology, infectious diseases, inpatient discharge summaries).ResultsThe algorithm sample consisted of 19 212 CIED procedures; 15 077 patients (78.5%) were White individuals, 1487 (15.5%) were African American; 18 766 (97.7%) were men. The mean (SD) age in our sample was 71.8 (10.6) years. The infection detection threshold of predicted probability was set to greater than 0.10 and the algorithm flagged 276 of 9606 (2.9%) cases in the development data set (9606 procedures); PPV in this group was 41.4% (95% CI, 31.6%-51.8%). In the validation set (9606 procedures), at predicted probability 0.10 or more the algorithm PPV was 43.5% (95% CI, 37.1%-50.2%), and overall sensitivity and specificity were 94.4% (95% CI, 88.2%-97.9%) and 48.8% (95% CI, 42.6%-55.1%), respectively.Conclusions and relevanceThe findings of this study suggest that the method of combining structured and text data in VA electronic medical records can be used to expand infection surveillance beyond traditional boundaries to include outpatient and procedural areas.

Project description:BackgroundThe electronic medical record (EMR) contains a rich source of information that could be harnessed for epidemic surveillance. We asked if structured EMR data could be coupled with computerized processing of free-text clinical entries to enhance detection of acute respiratory infections (ARI).MethodologyA manual review of EMR records related to 15,377 outpatient visits uncovered 280 reference cases of ARI. We used logistic regression with backward elimination to determine which among candidate structured EMR parameters (diagnostic codes, vital signs and orders for tests, imaging and medications) contributed to the detection of those reference cases. We also developed a computerized free-text search to identify clinical notes documenting at least two non-negated ARI symptoms. We then used heuristics to build case-detection algorithms that best combined the retained structured EMR parameters with the results of the text analysis.Principal findingsAn adjusted grouping of diagnostic codes identified reference ARI patients with a sensitivity of 79%, a specificity of 96% and a positive predictive value (PPV) of 32%. Of the 21 additional structured clinical parameters considered, two contributed significantly to ARI detection: new prescriptions for cough remedies and elevations in body temperature to at least 38°C. Together with the diagnostic codes, these parameters increased detection sensitivity to 87%, but specificity and PPV declined to 95% and 25%, respectively. Adding text analysis increased sensitivity to 99%, but PPV dropped further to 14%. Algorithms that required satisfying both a query of structured EMR parameters as well as text analysis disclosed PPVs of 52-68% and retained sensitivities of 69-73%.ConclusionStructured EMR parameters and free-text analyses can be combined into algorithms that can detect ARI cases with new levels of sensitivity or precision. These results highlight potential paths by which repurposed EMR information could facilitate the discovery of epidemics before they cause mass casualties.

Project description:UnlabelledOver one-third of patients treated in the Veterans Health Administration (VHA) are obese. VHA introduced the MOVE! Weight Management Program for Veterans in 2006 to provide comprehensive weight management services. An evolving, periodic evaluation using the RE-AIM framework (reach, effectiveness, adoption, implementation, and maintenance) has been conducted to gauge success and opportunities for improvement. Key metrics were identified in each RE-AIM dimension. Data were compiled over fiscal years (FY) 2006 through 2010 from a variety of sources including VHA administrative and clinical databases, electronic medical record reviews, and an annual, structured VHA facility self-report. REACH: Screening for obesity and offering weight management treatment to eligible patients increased from 66% to 95% over the past 3 years. MOVE! is currently provided at every VHA hospital facility and at over one-half of VHA community-based outpatient clinics. The percent of eligible patients who participate in at least one weight management visit has doubled since implementation began but has stabilized at 10 to 12%.EffectivenessAbout 18.6% of the 31,854 patients with available weight data who participated in at least two treatment visits between Jul 1, 2008 and Sep 30, 2009 had at least a 5% body weight loss by 6 months as did almost one-third of those who participated in more intense and sustained treatment. By contrast, only 12.5% of a comparison group of patients matched on age, gender, body mass index (BMI) class, and comorbidity status who were not treated with MOVE! had at least a 5% body weight loss. ADOPTION: The median full-time staff equivalent providing weight management services at each facility has increased over time and was 1.76 in FY 2010.ImplementationStaff from multiple disciplines typically provide MOVE!-related care although not all disciplines are involved with providing care at every facility. Group-based treatment has become increasingly utilized, and in FY 2010 it represented 72% of all MOVE!-related visits. Intensity of treatment has increased from an average of 3.6 visits per patient per year in FY 2007 to 4.6 in FY 2010, but more than half of patients have two visits or less. Almost all facilities now report the consistent use of key evidence-based behavioral strategies with patients. MAINTENANCE: While participation in MOVE! by patients continues to grow each year, facility self-reported program staffing and space/equipment challenges are potential barriers to long-term program maintenance. Evidence-based weight management treatment can be delivered at VHA medical centers and community-based outpatient clinics, but the REACH remains limited after several years of implementation. Intense and sustained treatment with MOVE! results in a modest positive impact on short-term weight loss outcomes, but a relatively small proportion of patients engage in this level of care. Increasing reach, improving effectiveness of care, and keeping patients engaged in treatment are areas for future policy, practice, and research.

Project description:IntroductionThe robustness of record linkage evaluation measures is of high importance since linkage techniques are assessed based on these. However, minimal research has been conducted to evaluate the suitability of existing evaluation measures in the context of linking groups of records. Linkage quality is generally evaluated based on traditional measures such as precision and recall. As we show, these traditional evaluation measures are not suitable for evaluating groups of linked records because they evaluate the quality of individual record pairs rather than the quality of records grouped into clusters.ObjectivesWe set out to highlight the shortcomings of traditional evaluation measures and then propose a novel method to evaluate clustering quality in the context of group-based record linkage.MethodsThe proposed linkage evaluation method assesses how well individual records have been allocated into predicted groups/clusters with respect to ground-truth data. We first identify the best representative predicted cluster for each ground-truth cluster and, based on the resulting mapping, each record in a ground-truth cluster is assigned to one of seven categories. These categories reflect how well the linkage technique assigned records into groups.ResultsWe empirically evaluated our proposed method using real-world data and showed that it better reflects the quality of clusters generated by three group-based record linkage techniques. We also showed that traditional measures such as precision and recall can produce ambiguous results whereas our method does not.ConclusionsThe proposed evaluation method provides unambiguous results regarding the assessed group-based record linkage approaches. The method comprises of seven categories which reflect how each record was predicted, providing more detailed information about the quality of the linkage result. This will help to make better-informed decisions about which linkage technique is best suited for a given linkage application.

Project description:BackgroundWhen complex health services interventions are implemented in real-world settings, adaptations are inevitable. Adaptations are changes made to an intervention, implementation strategy, or context prior to, during, and after implementation to improve uptake and fit. There is a growing interest in systematically documenting and understanding adaptations including what is changed, why, when, by whom, and with what impact. The rural Transitions Nurse Program (TNP) is a program in the Veterans Health Administration (VHA), designed to safely transition a rural veteran from a tertiary hospital back home. TNP has been implemented in multiple cohorts across 11 sites nationwide over 4 years. In this paper, we describe adaptations in five TNP sites from the first cohort of sites and implications for the scale-up of TNP and discuss lessons learned for the systematic documentation and analysis of adaptations.MethodsWe used the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) expanded version of the original Stirman framework to guide the rapid qualitative matrix analysis of adaptations. Adaptations were documented using multiple approaches: real-time database, semi-structured midpoint and exit interviews with implementors, and member checking with the implementation team. Interviews were recorded and transcribed. To combine multiple sources of adaptations, we used key domains from our framework and organized adaptations by time when the adaptation occurred (pre-, early, mid-, late implementation; sustainment) and categorized them as proactive or reactive.ResultsForty-one unique adaptations were reported during the study period. The most common type of adaptation was changes in target populations (patient enrollment criteria) followed by personnel changes (staff turnover). Most adaptations occurred during the mid-implementation time period and varied in number and type of adaptation. The reasons for this are discussed, and suggestions for future adaptation protocols are included.ConclusionsThis study demonstrates the feasibility of systematically documenting adaptations using multiple methods across time points. Implementors were able to track adaptations in real time across the course of an intervention, which provided timely and actionable feedback to the implementation team overseeing the national roll-out of the program. Longitudinal semi-structured interviews can complement the real-time database and elicit reflective adaptations.

Dataset Information

Novel methodology to measure pre-procedure antimicrobial prophylaxis: integrating text searches with structured data from the Veterans Health Administration's electronic medical record.

Publications

Novel methodology to measure pre-procedure antimicrobial prophylaxis: integrating text searches with structured data from the Veterans Health Administration's electronic medical record.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets