Dataset Information

Validation of an algorithm to estimate gestational age in electronic health plan databases.

ABSTRACT: To validate an algorithm that uses delivery date and diagnosis codes to define gestational age at birth in electronic health plan databases.Using data from 225,384 live born deliveries to women aged 15-45?years in 2001-2007 within eight of the 11 health plans participating in the Medication Exposure in Pregnancy Risk Evaluation Program, we compared (1) the algorithm-derived gestational age versus the "gold-standard" gestational age obtained from the infant birth certificate file and (2) the prenatal exposure status of two antidepressants (fluoxetine and sertraline) and two antibiotics (amoxicillin and azithromycin) as determined by the algorithm-derived versus the gold-standard gestational age.The mean algorithm-derived gestational age at birth was lower than the mean obtained from the birth certificate file among singleton deliveries (267.9 vs 273.5?days) but not among multiple-gestation deliveries (253.9 vs 252.6?days). The algorithm-derived prenatal exposure to the antidepressants had a sensitivity and a positive predictive value of ?95%, and a specificity and a negative predictive value of almost 100%. Sensitivity and positive predictive value were both ?90%, and specificity and negative predictive value were both >99% for the antibiotics.A gestational age algorithm based upon electronic health plan data correctly classified medication exposure status in most live born deliveries, but trimester-specific misclassification may be higher for drugs typically used for short durations.

SUBMITTER: Li Q

PROVIDER: S-EPMC3644383 | biostudies-literature | 2013 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Validation of an algorithm to estimate gestational age in electronic health plan databases.

Li Qian Q Andrade Susan E SE Cooper William O WO Davis Robert L RL Dublin Sascha S Hammad Tarek A TA Pawloski Pamala A PA Pinheiro Simone P SP Raebel Marsha A MA Scott Pamela E PE Smith David H DH Dashevsky Inna I Haffenreffer Katherine K Johnson Karin E KE Toh Sengwee S

Pharmacoepidemiology and drug safety 20130121 5

<h4>Purpose</h4>To validate an algorithm that uses delivery date and diagnosis codes to define gestational age at birth in electronic health plan databases.<h4>Methods</h4>Using data from 225,384 live born deliveries to women aged 15-45 years in 2001-2007 within eight of the 11 health plans participating in the Medication Exposure in Pregnancy Risk Evaluation Program, we compared (1) the algorithm-derived gestational age versus the "gold-standard" gestational age obtained from the infant birth c ...[more]

PMID: 23335117

Similar Datasets

Project description:ObjectivesBiological modelling of routinely collected newborn screening data has emerged as a novel method for deriving postnatal gestational age estimates. Validation of published models has previously been limited to cohorts largely consisting of infants of white Caucasian ethnicity. In this study, we sought to determine the validity of a published gestational age estimation algorithm among recent immigrants to Canada, where maternal landed immigrant status was used as a surrogate measure of infant ethnicity.DesignWe conducted a retrospective validation study in infants born in Ontario between April 2009 and September 2011.SettingProvincial data from Ontario, Canada were obtained from the Institute for Clinical Evaluative Sciences.ParticipantsThe dataset included 230?034 infants born to non-landed immigrants and 70?098 infants born to immigrant mothers. The five most common countries of maternal origin were India (n=10?038), China (n=7468), Pakistan (n=5824), The Philippines (n=5441) and Vietnam (n=1408). Maternal country of origin was obtained from Citizenship and Immigration Canada's Landed Immigrant Database.Primary and secondary outcome measuresPerformance of a postnatal gestational age algorithm was evaluated across non-immigrant and immigrant populations.ResultsRoot mean squared error (RMSE) of 1.05 weeks was observed for infants born to non-immigrant mothers, whereas RMSE ranged from 0.98 to 1.15 weeks among infants born to immigrant mothers. Area under the receiver operating characteristic curve for distinguishing term versus preterm infants (?37?vs?<37 weeks gestational age or >34?vs ?34 weeks gestational age) was 0.958 and 0.986, respectively, in the non-immigrant subgroup and ranged from 0.927 to 0.964 and 0.966 to 0.99 in the immigrant subgroups.ConclusionsAlgorithms for postnatal determination of gestational age may be further refined by development and validation of region or ethnicity-specific models. However, our results provide reassurance that an algorithm developed from Ontario-born infant cohorts performs well across a range of ethnicities and maternal countries of origin without modification.

Project description:BACKGROUND:Electronic health record (EHR) systems generate large datasets that can significantly enrich the development of medical predictive models. Several attempts have been made to investigate the effect of glycated hemoglobin (HbA1c) elevation on the prediction of diabetes onset. However, there is still a need for validation of these models using EHR data collected from different populations. OBJECTIVE:The aim of this study is to perform a replication study to validate, evaluate, and identify the strengths and weaknesses of replicating a predictive model that employed multiple logistic regression with EHR data to forecast the levels of HbA1c. The original study used data from a population in the United States and this differentiated replication used a population in Saudi Arabia. METHODS:A total of 3 models were developed and compared with the model created in the original study. The models were trained and tested using a larger dataset from Saudi Arabia with 36,378 records. The 10-fold cross-validation approach was used for measuring the performance of the models. RESULTS:Applying the method employed in the original study achieved an accuracy of 74% to 75% when using the dataset collected from Saudi Arabia, compared with 77% obtained from using the population from the United States. The results also show a different ranking of importance for the predictors between the original study and the replication. The order of importance for the predictors with our population, from the most to the least importance, is age, random blood sugar, estimated glomerular filtration rate, total cholesterol, non-high-density lipoprotein, and body mass index. CONCLUSIONS:This replication study shows that direct use of the models (calculators) created using multiple logistic regression to predict the level of HbA1c may not be appropriate for all populations. This study reveals that the weighting of the predictors needs to be calibrated to the population used. However, the study does confirm that replicating the original study using a different population can help with predicting the levels of HbA1c by using the predictors that are routinely collected and stored in hospital EHR systems.

Project description:BackgroundFinancial codes are often used to extract diagnoses from electronic health records. This approach is prone to false positives. Alternatively, queries are constructed, but these are highly center and language specific. A tantalizing alternative is the automatic identification of patients by employing machine learning on format-free text entries.ObjectiveThe aim of this study was to develop an easily implementable workflow that builds a machine learning algorithm capable of accurately identifying patients with rheumatoid arthritis from format-free text fields in electronic health records.MethodsTwo electronic health record data sets were employed: Leiden (n=3000) and Erlangen (n=4771). Using a portion of the Leiden data (n=2000), we compared 6 different machine learning methods and a naïve word-matching algorithm using 10-fold cross-validation. Performances were compared using the area under the receiver operating characteristic curve (AUROC) and the area under the precision recall curve (AUPRC), and F1 score was used as the primary criterion for selecting the best method to build a classifying algorithm. We selected the optimal threshold of positive predictive value for case identification based on the output of the best method in the training data. This validation workflow was subsequently applied to a portion of the Erlangen data (n=4293). For testing, the best performing methods were applied to remaining data (Leiden n=1000; Erlangen n=478) for an unbiased evaluation.ResultsFor the Leiden data set, the word-matching algorithm demonstrated mixed performance (AUROC 0.90; AUPRC 0.33; F1 score 0.55), and 4 methods significantly outperformed word-matching, with support vector machines performing best (AUROC 0.98; AUPRC 0.88; F1 score 0.83). Applying this support vector machine classifier to the test data resulted in a similarly high performance (F1 score 0.81; positive predictive value [PPV] 0.94), and with this method, we could identify 2873 patients with rheumatoid arthritis in less than 7 seconds out of the complete collection of 23,300 patients in the Leiden electronic health record system. For the Erlangen data set, gradient boosting performed best (AUROC 0.94; AUPRC 0.85; F1 score 0.82) in the training set, and applied to the test data, resulted once again in good results (F1 score 0.67; PPV 0.97).ConclusionsWe demonstrate that machine learning methods can extract the records of patients with rheumatoid arthritis from electronic health record data with high precision, allowing research on very large populations for limited costs. Our approach is language and center independent and could be applied to any type of diagnosis. We have developed our pipeline into a universally applicable and easy-to-implement workflow to equip centers with their own high-performing algorithm. This allows the creation of observational studies of unprecedented size covering different countries for low cost from already available data in electronic health record systems.

Project description:BackgroundStroke is an important clinical outcome in cardiovascular research. However, the ascertainment of incident stroke is typically accomplished via time-consuming manual chart abstraction. Current phenotyping efforts using electronic health records for stroke focus on case ascertainment rather than incident disease, which requires knowledge of the temporal sequence of events.ObjectiveThe aim of this study was to develop a machine learning-based phenotyping algorithm for incident stroke ascertainment based on diagnosis codes, procedure codes, and clinical concepts extracted from clinical notes using natural language processing.MethodsThe algorithm was trained and validated using an existing epidemiology cohort consisting of 4914 patients with atrial fibrillation (AF) with manually curated incident stroke events. Various combinations of feature sets and machine learning classifiers were compared. Using a heuristic rule based on the composition of concepts and codes, we further detected the stroke subtype (ischemic stroke/transient ischemic attack or hemorrhagic stroke) of each identified stroke. The algorithm was further validated using a cohort (n=150) stratified sampled from a population in Olmsted County, Minnesota (N=74,314).ResultsAmong the 4914 patients with AF, 740 had validated incident stroke events. The best-performing stroke phenotyping algorithm used clinical concepts, diagnosis codes, and procedure codes as features in a random forest classifier. Among patients with stroke codes in the general population sample, the best-performing model achieved a positive predictive value of 86% (43/50; 95% CI 0.74-0.93) and a negative predictive value of 96% (96/100). For subtype identification, we achieved an accuracy of 83% in the AF cohort and 80% in the general population sample.ConclusionsWe developed and validated a machine learning-based algorithm that performed well for identifying incident stroke and for determining type of stroke. The algorithm also performed well on a sample from a general population, further demonstrating its generalizability and potential for adoption by other institutions.

Project description:BackgroundPrecision oncology has the potential to leverage clinical and genomic data in advancing disease prevention, diagnosis, and treatment. A key research area focuses on the early detection of primary cancers and potential prediction of cancers of unknown primary in order to facilitate optimal treatment decisions.ObjectiveThis study presents a methodology to harmonize phenotypic and genetic data features to classify primary cancer types and predict cancers of unknown primaries.MethodsWe extracted genetic data elements from oncology genetic reports of 1011 patients with cancer and their corresponding phenotypical data from Mayo Clinic's electronic health records. We modeled both genetic and electronic health record data with HL7 Fast Healthcare Interoperability Resources. The semantic web Resource Description Framework was employed to generate the network-based data representation (ie, patient-phenotypic-genetic network). Based on the Resource Description Framework data graph, Node2vec graph-embedding algorithm was applied to generate features. Multiple machine learning and deep learning backbone models were compared for cancer prediction performance.ResultsWith 6 machine learning tasks designed in the experiment, we demonstrated the proposed method achieved favorable results in classifying primary cancer types (area under the receiver operating characteristic curve [AUROC] 96.56% for all 9 cancer predictions on average based on the cross-validation) and predicting unknown primaries (AUROC 80.77% for all 8 cancer predictions on average for real-patient validation). To demonstrate the interpretability, 17 phenotypic and genetic features that contributed the most to the prediction of each cancer were identified and validated based on a literature review.ConclusionsAccurate prediction of cancer types can be achieved with existing electronic health record data with satisfactory precision. The integration of genetic reports improves prediction, illustrating the translational values of incorporating genetic tests early at the diagnosis stage for patients with cancer.

Project description:BackgroundFractures as a result of osteoporosis and low bone mass are common and give rise to significant clinical, personal, and economic burden. Even after a fracture occurs, high fracture risk remains widely underdiagnosed and undertreated. Common fracture risk assessment tools utilize a subset of clinical risk factors for prediction, and often require manual data entry. Furthermore, these tools predict risk over the long term and do not explicitly provide short-term risk estimates necessary to identify patients likely to experience a fracture in the next 1-2 years.ObjectiveThe goal of this study was to develop and evaluate an algorithm for the identification of patients at risk of fracture in a subsequent 1- to 2-year period. In order to address the aforementioned limitations of current prediction tools, this approach focused on a short-term timeframe, automated data entry, and the use of longitudinal data to inform the predictions.MethodsUsing retrospective electronic health record data from over 1,000,000 patients, we developed Crystal Bone, an algorithm that applies machine learning techniques from natural language processing to the temporal nature of patient histories to generate short-term fracture risk predictions. Similar to how language models predict the next word in a given sentence or the topic of a document, Crystal Bone predicts whether a patient's future trajectory might contain a fracture event, or whether the signature of the patient's journey is similar to that of a typical future fracture patient. A holdout set with 192,590 patients was used to validate accuracy. Experimental baseline models and human-level performance were used for comparison.ResultsThe model accurately predicted 1- to 2-year fracture risk for patients aged over 50 years (area under the receiver operating characteristics curve [AUROC] 0.81). These algorithms outperformed the experimental baselines (AUROC 0.67) and showed meaningful improvements when compared to retrospective approximation of human-level performance by correctly identifying 9649 of 13,765 (70%) at-risk patients who did not receive any preventative bone-health-related medical interventions from their physicians.ConclusionsThese findings indicate that it is possible to use a patient's unique medical history as it changes over time to predict the risk of short-term fracture. Validating and applying such a tool within the health care system could enable automated and widespread prediction of this risk and may help with identification of patients at very high risk of fracture.

Project description:Objective Identifying the time of SARS-CoV-2 viral infection relative to specific gestational weeks is critical for delineating the role of viral infection timing in adverse pregnancy outcomes. However, this task is difficult when it comes to Electronic Health Records (EHR). In combating the COVID-19 pandemic for maternal health, we sought to develop and validate a clinical information extraction algorithm to detect the time of clinical events relative to gestational weeks. Materials and methods We used EHR from the National COVID Cohort Collaborative (N3C), in which the EHR are normalized by the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM). We performed EHR phenotyping, resulting in 270,897 pregnant women (June 1st, 2018 to May 31st, 2021). We developed a rule-based algorithm and performed a multi-level evaluation to test content validity and clinical validity, and extreme length of gestation (<150 or >300). Results The algorithm identified 296,194 pregnancies (16,659 COVID-19, 174,744 without COVID-19) in 270,897 pregnant women. For inferring gestational age, 95% cases (n = 40) have moderate-high accuracy (Cohen’s Kappa = 0.62); 100% cases (n = 40) have moderate-high granularity of temporal information (Cohen’s Kappa = 1). For inferring delivery dates, the accuracy is 100% (Cohen’s Kappa = 1). The accuracy of gestational age detection for the extreme length of gestation is 93.3% (Cohen’s Kappa = 1). Mothers with COVID-19 showed higher prevalence in obesity or overweight (35.1% vs. 29.5%), diabetes (17.8% vs. 17.0%), chronic obstructive pulmonary disease (0.2% vs. 0.1%), respiratory distress syndrome or acute respiratory failure (1.8% vs. 0.2%). Discussion We explored the characteristics of pregnant women by different gestational weeks of SARS-CoV-2 infection with our algorithm. TED-PC is the first to infer the exact gestational week linked with every clinical event from EHR and detect the timing of SARS-CoV-2 infection in pregnant women. Conclusion The algorithm shows excellent clinical validity in inferring gestational age and delivery dates, which supports multiple EHR cohorts on N3C studying the impact of COVID-19 on pregnancy.

Dataset Information

Validation of an algorithm to estimate gestational age in electronic health plan databases.

Publications

Validation of an algorithm to estimate gestational age in electronic health plan databases.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets