Dataset Information

Validation of diagnosis codes to identify side of colon in an electronic health record registry.

ABSTRACT:

Background

The use of real-world data to generate evidence requires careful assessment and validation of critical variables before drawing clinical conclusions. Prospective clinical trial data suggest that anatomic origin of colon cancer impacts prognosis and treatment effectiveness. As an initial step in validating this observation in routine clinical settings, we explored the feasibility and accuracy of obtaining information on tumor sidedness from electronic health records (EHR) billing codes.

Methods

Nine thousand four hundred three patients with metastatic colorectal cancer (mCRC) were selected from the Flatiron Health database, which is derived from de-identified EHR data. This study included a random sample of 200 mCRC patients. Tumor site data derived from International Classification of Diseases (ICD) codes were compared with data abstracted from unstructured documents in the EHR (e.g. surgical and pathology notes). Concordance was determined via observed agreement and Cohen's kappa coefficient (?). Accuracy of ICD codes for each tumor site (left, right, transverse) was determined by calculating the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), and corresponding 95% confidence intervals, using abstracted data as the gold standard.

Results

Study patients had similar characteristics and side of colon distribution compared with the full mCRC dataset. The observed agreement between the ICD codes and abstracted data for tumor site for all sampled patients was 0.58 (??=?0.41). When restricting to the 62% of patients with a side-specific ICD code, the observed agreement was 0.84 (??=?0.79). The specificity (92-98%) of structured data for tumor location was high, with lower sensitivity (49-63%), PPV (64-92%) and NPV (72-97%). Demographic and clinical characteristics were similar between patients with specific and non-specific side of colon ICD codes.

Conclusions

ICD codes are a highly reliable indicator of tumor location when the specific location code is entered in the EHR. However, non-specific side of colon ICD codes are present for a sizable minority of patients, and structured data alone may not be adequate to support testing of some research hypotheses. Careful assessment of key variables is required before determining the need for clinical abstraction to supplement structured data in generating real-world evidence from EHRs.

SUBMITTER: Luhn P

PROVIDER: S-EPMC6700780 | biostudies-literature | 2019 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Validation of diagnosis codes to identify side of colon in an electronic health record registry.

Luhn Patricia P Kuk Deborah D Carrigan Gillis G Nussbaum Nathan N Sorg Rachael R Rohrer Rebecca R Tucker Melisa G MG Arnieri Brandon B Taylor Michael D MD Meropol Neal J NJ

BMC medical research methodology 20190819 1

<h4>Background</h4>The use of real-world data to generate evidence requires careful assessment and validation of critical variables before drawing clinical conclusions. Prospective clinical trial data suggest that anatomic origin of colon cancer impacts prognosis and treatment effectiveness. As an initial step in validating this observation in routine clinical settings, we explored the feasibility and accuracy of obtaining information on tumor sidedness from electronic health records (EHR) billi ...[more]

PMID: 31426736

Similar Datasets

Project description:BACKGROUND:Accurate anesthesiology procedure code data are essential to quality improvement, research, and reimbursement tasks within anesthesiology practices. Advanced data science techniques, including machine learning and natural language processing, offer opportunities to develop classification tools for Current Procedural Terminology codes across anesthesia procedures. METHODS:Models were created using a Train/Test dataset including 1,164,343 procedures from 16 academic and private hospitals. Five supervised machine learning models were created to classify anesthesiology Current Procedural Terminology codes, with accuracy defined as first choice classification matching the institutional-assigned code existing in the perioperative database. The two best performing models were further refined and tested on a Holdout dataset from a single institution distinct from Train/Test. A tunable confidence parameter was created to identify cases for which models were highly accurate, with the goal of at least 95% accuracy, above the reported 2018 Centers for Medicare and Medicaid Services (Baltimore, Maryland) fee-for-service accuracy. Actual submitted claim data from billing specialists were used as a reference standard. RESULTS:Support vector machine and neural network label-embedding attentive models were the best performing models, respectively, demonstrating overall accuracies of 87.9% and 84.2% (single best code), and 96.8% and 94.0% (within top three). Classification accuracy was 96.4% in 47.0% of cases using support vector machine and 94.4% in 62.2% of cases using label-embedding attentive model within the Train/Test dataset. In the Holdout dataset, respective classification accuracies were 93.1% in 58.0% of cases and 95.0% among 62.0%. The most important feature in model training was procedure text. CONCLUSIONS:Through application of machine learning and natural language processing techniques, highly accurate real-time models were created for anesthesiology Current Procedural Terminology code classification. The increased processing speed and a priori targeted accuracy of this classification approach may provide performance optimization and cost reduction for quality improvement, research, and reimbursement tasks reliant on anesthesiology procedure codes.

Project description:ObjectivesHospitalisations for serious infections are common among middle age and older adults and frequently used as study outcomes. Yet, few studies have evaluated the performance of diagnosis codes to identify serious infections in this population. We sought to determine the positive predictive value (PPV) of diagnosis codes for identifying hospitalisations due to serious infections among middle age and older adults.Setting and participantsWe identified hospitalisations for possible infection among adults >=50 years enrolled in the Tennessee Medicaid healthcare programme (2008-2012) using International Classifications of Diseases, Ninth Revision diagnosis codes for pneumonia, meningitis/encephalitis, bacteraemia/sepsis, cellulitis/soft-tissue infections, endocarditis, pyelonephritis and septic arthritis/osteomyelitis.DesignMedical records were systematically obtained from hospitals randomly selected from a stratified sampling framework based on geographical region and hospital discharge volume.MeasuresTwo trained clinical reviewers used a standardised extraction form to abstract information from medical records. Predefined algorithms served as reference to adjudicate confirmed infection-specific hospitalisations. We calculated the PPV of diagnosis codes using confirmed hospitalisations as reference. Sensitivity analyses determined the robustness of the PPV to definitions that required radiological or microbiological confirmation. We also determined inter-rater reliability between reviewers.ResultsThe PPV of diagnosis codes for hospitalisations for infection (n=716) was 90.2% (95% CI 87.8% to 92.2%). The PPV was highest for pneumonia (96.5% (95% CI 93.9% to 98.0%)) and cellulitis (91.1% (95% CI 84.7% to 94.9%)), and lowest for meningitis/encephalitis (50.0% (95% CI 23.7% to 76.3%)). The adjudication reliability was excellent (92.7% agreement; first agreement coefficient: 0.91). The overall PPV was lower when requiring microbiological confirmation (45%) and when requiring radiological confirmation for pneumonia (79%).ConclusionsDischarge diagnosis codes have a high PPV for identifying hospitalisations for common, serious infections among middle age and older adults. PPV estimates for rare infections were imprecise.

Project description:BACKGROUND:Electronic health record (EHR) based chronic kidney disease (CKD) registries are central to population health strategies to improve CKD care. In 2015, Partners Healthcare System (PHS), encompassing multiple academic and community hospitals and outpatient care facilities in Massachusetts, developed an EHR-based CKD registry to identify opportunities for quality improvement, defined as improvement on both process measures and outcomes measures associated with clinical care. METHODS:Patients are included in the registry based on the following criteria: 1) two estimated glomerular filtration rate (eGFR) results < 60 ml/min/1.73m2 separated by 90 days, including the most recent eGFR being < 60 ml/min/1.73m2; or 2) the most recent two urine protein values > 300 mg protein/g creatinine on either urine total protein/creatinine ratio or urine albumin/creatinine ratio; or 3) an EHR problem list diagnosis of end stage renal disease (ESRD). The registry categorizes patients by CKD stage and includes rates of annual testing for eGFR and proteinuria, blood pressure control, use of angiotensin converting enzyme inhibitors (ACE-Is) or angiotensin receptor blockers (ARBs), nephrotoxic medication use, hepatitis B virus (HBV) immunization, vascular access placement, transplant status, CKD progression risk; number of outpatient nephrology visits, and hospitalizations. RESULTS:The CKD registry includes 60,503 patients and has revealed several opportunities for care improvement including 1) annual proteinuria testing performed for 17% (stage 3) and 31% (stage 4) of patients; 2) ACE-I/ARB used in 41% (stage 3) and 46% (stage 4) of patients; 3) nephrotoxic medications used among 23% of stage 4 patients; and 4) 89% of stage 4 patients lack HBV immunity. For advanced CKD patients there are opportunities to improve vascular access placement, transplant referrals and outpatient nephrology contact. CONCLUSIONS:A CKD registry can identify modifiable care gaps across the spectrum of CKD care and enable population health strategy implementation. No linkage to Social Security Death Master File or US Renal Data System (USRDS) databases limits our ability to track mortality and progression to ESRD.

Project description:BackgroundNursing care is a critical determinant of patient outcomes in the intensive care unit (ICU). Most studies of nursing care have focused on nursing characteristics aggregated across the ICU (eg, unit-wide nurse-to-patient ratios, education, and working environment). In contrast, relatively little work has focused on the influence of individual nurses and their characteristics on patient outcomes. Such research could provide granular information needed to create evidence-based nurse assignments, where a nurse's unique skills are matched to each patient's needs. To date, research in this area is hindered by an inability to link individual nurses to specific patients retrospectively and at scale.ObjectiveThis study aimed to determine the feasibility of using nurse metadata from the electronic health record (EHR) to retrospectively determine nurse-patient assignments in the ICU.MethodsWe used EHR data from 38 ICUs in 18 hospitals from 2018 to 2020. We abstracted data on the time and frequency of nurse charting of clinical assessments and medication administration; we then used those data to iteratively develop a deterministic algorithm to identify a single ICU nurse for each patient shift. We examined the accuracy and precision of the algorithm by performing manual chart review on a randomly selected subset of patient shifts.ResultsThe analytic data set contained 5,479,034 unique nurse-patient charting times; 748,771 patient shifts; 87,466 hospitalizations; 70,002 patients; and 8,134 individual nurses. The final algorithm identified a single nurse for 97.3% (728,533/748,771) of patient shifts. In the remaining 2.7% (20,238/748,771) of patient shifts, the algorithm either identified multiple nurses (4,755/748,771, 0.6%), no nurse (14,689/748,771, 2%), or the same nurse as the prior shift (794/748,771, 0.1%). In 200 patient shifts selected for chart review, the algorithm had a 93% accuracy (ie, correctly identifying the primary nurse or correctly identifying that there was no primary nurse) and a 94.4% precision (ie, correctly identifying the primary nurse when a primary nurse was identified). Misclassification was most frequently due to patient transitions in care location, such as ICU transfers, discharges, and admissions.ConclusionsMetadata from the EHR can accurately identify individual nurse-patient assignments in the ICU. This information enables novel studies of ICU nurse staffing at the individual nurse-patient level, which may provide further insights into how nurse staffing can be leveraged to improve patient outcomes.

Dataset Information

Validation of diagnosis codes to identify side of colon in an electronic health record registry.

Background

Methods

Results

Conclusions

Publications

Validation of diagnosis codes to identify side of colon in an electronic health record registry.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets