Dataset Information

Modeling physician variability to prioritize relevant medical record information.

ABSTRACT:

Objective

Patient information can be retrieved more efficiently in electronic medical record (EMR) systems by using machine learning models that predict which information a physician will seek in a clinical context. However, information-seeking behavior varies across EMR users. To explicitly account for this variability, we derived hierarchical models and compared their performance to nonhierarchical models in identifying relevant patient information in intensive care unit (ICU) cases.

Materials and methods

Critical care physicians reviewed ICU patient cases and selected data items relevant for presenting at morning rounds. Using patient EMR data as predictors, we derived hierarchical logistic regression (HLR) and standard logistic regression (LR) models to predict their relevance.

Results

In 73 pairs of HLR and LR models, the HLR models achieved an area under the receiver operating characteristic curve of 0.81, 95% confidence interval (CI) [0.80-0.82], which was statistically significantly higher than that of LR models (0.75, 95% CI [0.74-0.76]). Further, the HLR models achieved statistically significantly lower expected calibration error (0.07, 95% CI [0.06-0.08]) than LR models (0.16, 95% CI [0.14-0.17]).

Discussion

The physician reviewers demonstrated variability in selecting relevant data. Our results show that HLR models perform significantly better than LR models with respect to both discrimination and calibration. This is likely due to explicitly modeling physician-related variability.

Conclusion

Hierarchical models can yield better performance when there is physician-related variability as in the case of identifying relevant information in the EMR.

SUBMITTER: Tajgardoon M

PROVIDER: S-EPMC7886572 | biostudies-literature | 2020 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Modeling physician variability to prioritize relevant medical record information.

Tajgardoon Mohammadamin M Cooper Gregory F GF King Andrew J AJ Clermont Gilles G Hochheiser Harry H Hauskrecht Milos M Sittig Dean F DF Visweswaran Shyam S

JAMIA open 20201231 4

<h4>Objective</h4>Patient information can be retrieved more efficiently in electronic medical record (EMR) systems by using machine learning models that predict which information a physician will seek in a clinical context. However, information-seeking behavior varies across EMR users. To explicitly account for this variability, we derived hierarchical models and compared their performance to nonhierarchical models in identifying relevant patient information in intensive care unit (ICU) cases.<h ...[more]

PMID: 33623894

Similar Datasets

Project description:BackgroundUnderstanding association between factors related to clinical work environment and well-being can inform strategies to improve physicians' work experience.ObjectiveTo model and quantify what drivers of work composition, team structure, and dynamics are associated with well-being.DesignUtilizing social network modeling, this cohort study of physicians in an academic health center examined inbasket messaging data from 2018 to 2019 to identify work composition, team structure, and dynamics features. Indicators from a survey in 2019 were used as dependent variables to identify factors predictive of well-being.ParticipantsEHR data available for 188 physicians and their care teams from 18 primary care practices; survey data available for 163/188 physicians.Main measuresArea under the receiver operating characteristic curve (AUC) of logistic regression models to predict well-being dependent variables was assessed out-of-sample.Key resultsThe mean AUC of the model for the dependent variables of emotional exhaustion, vigor, and professional fulfillment was, respectively, 0.665 (SD 0.085), 0.700 (SD 0.082), and 0.669 (SD 0.082). Predictors associated with decreased well-being included physician centrality within support team (OR 3.90, 95% CI 1.28-11.97, P=0.01) and share of messages related to scheduling (OR 1.10, 95% CI 1.03-1.17, P=0.003). Predictors associated with increased well-being included higher number of medical assistants within close support team (OR 0.91, 95% CI 0.83-0.99, P=0.05), nurse-centered message writing practices (OR 0.89, 95% CI 0.83-0.95, P=0.001), and share of messages related to ambiguous diagnosis (OR 0.92, 95% CI 0.87-0.98, P=0.01).ConclusionsThrough integration of EHR data with social network modeling, the analysis highlights new characteristics of care team structure and dynamics that are associated with physician well-being. This quantitative methodology can be utilized to assess in a refined data-driven way the impact of organizational changes to improve well-being through optimizing team dynamics and work composition.

Project description:BackgroundPhysician notes routinely recorded during patient care represent a vast and underutilized resource for human disease studies on a population scale. Their use in research is primarily limited by the need to separate confidential patient information from clinical annotations, a process that is resource-intensive when performed manually. This study seeks to create an automated method for de-identifying physician notes that does not require large amounts of private information: in addition to training a model to recognize Protected Health Information (PHI) within private physician notes, we reverse the problem and train a model to recognize non-PHI words and phrases that appear in public medical texts.MethodsPublic and private medical text sources were analyzed to distinguish common medical words and phrases from Protected Health Information. Patient identifiers are generally nouns and numbers that appear infrequently in medical literature. To quantify this relationship, term frequencies and part of speech tags were compared between journal publications and physician notes. Standard medical concepts and phrases were then examined across ten medical dictionaries. Lists and rules were included from the US census database and previously published studies. In total, 28 features were used to train decision tree classifiers.ResultsThe model successfully recalled 98% of PHI tokens from 220 discharge summaries. Cost sensitive classification was used to weight recall over precision (98% F10 score, 76% F1 score). More than half of the false negatives were the word "of" appearing in a hospital name. All patient names, phone numbers, and home addresses were at least partially redacted. Medical concepts such as "elevated white blood cell count" were informative for de-identification. The results exceed the previously approved criteria established by four Institutional Review Boards.ConclusionsThe results indicate that distributional differences between private and public medical text can be used to accurately classify PHI. The data and algorithms reported here are made freely available for evaluation and improvement.

Dataset Information

Modeling physician variability to prioritize relevant medical record information.

Objective

Materials and methods

Results

Discussion

Conclusion

Publications

Modeling physician variability to prioritize relevant medical record information.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets