Dataset Information

An Automated Line-of-Therapy Algorithm for Adults With Metastatic Non-Small Cell Lung Cancer: Validation Study Using Blinded Manual Chart Review.

ABSTRACT:

Background

Extraction of line-of-therapy (LOT) information from electronic health record and claims data is essential for determining longitudinal changes in systemic anticancer therapy in real-world clinical settings.

Objective

The aim of this retrospective cohort analysis is to validate and refine our previously described open-source LOT algorithm by comparing the output of the algorithm with results obtained through blinded manual chart review.

Methods

We used structured electronic health record data and clinical documents to identify 500 adult patients treated for metastatic non-small cell lung cancer with systemic anticancer therapy from 2011 to mid-2018; we assigned patients to training (n=350) and test (n=150) cohorts, randomly divided proportional to the overall ratio of simple:complex cases (n=254:246). Simple cases were patients who received one LOT and no maintenance therapy; complex cases were patients who received more than one LOT and/or maintenance therapy. Algorithmic changes were performed using the training cohort data, after which the refined algorithm was evaluated against the test cohort.

Results

For simple cases, 16 instances of discordance between the LOT algorithm and chart review prerefinement were reduced to 8 instances postrefinement; in the test cohort, there was no discordance between algorithm and chart review. For complex cases, algorithm refinement reduced the discordance from 68 to 62 instances, with 37 instances in the test cohort. The percentage agreement between LOT algorithm output and chart review for patients who received one LOT was 89% prerefinement, 93% postrefinement, and 93% for the test cohort, whereas the likelihood of precise matching between algorithm output and chart review decreased with an increasing number of unique regimens. Several areas of discordance that arose from differing definitions of LOTs and maintenance therapy could not be objectively resolved because of a lack of precise definitions in the medical literature.

Conclusions

Our findings identify common sources of discordance between the LOT algorithm and clinician documentation, providing the possibility of targeted algorithm refinement.

SUBMITTER: Meng W

PROVIDER: S-EPMC8548977 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundFalls are common adverse events in hospitals, frequently leading to additional health costs due to prolonged stays and extra care. Therefore, reliable fall detection is vital to develop and test fall prevention strategies. However, conventional methods-voluntary incident reports and manual chart reviews-are error-prone and time consuming, respectively. Using a search algorithm to examine patients' electronic health record data and flag fall indicators offers an inexpensive, sensitive, cost-effective alternative.ObjectiveThis study's purpose was to develop a fall detection algorithm for use with electronic health record data, then to evaluate it alongside the Global Trigger Tool, incident reports, a manual chart review, and patient-reported falls.MethodsConducted on 2 campuses of a large hospital system in Switzerland, this retrospective diagnostic accuracy study consisted of 2 substudies: the first, targeting 240 patients, for algorithm development and the second, targeting 298 patients, for validation. In the development study, we compared the new algorithm's in-hospital fall rates with those indicated by the Global Trigger Tool and incident reports; in the validation study, we compared the algorithm's in-hospital fall rates with those from patient-reported falls and manual chart review. We compared the various methods by calculating sensitivity, specificity, and predictive values.ResultsTwenty in-hospital falls were discovered in the development study sample. Of these, the algorithm detected 19 (sensitivity 95%), the Global Trigger Tool detected 18 (90%), and incident reports detected 14 (67%). Of the 15 falls found in the validation sample, the algorithm identified all 15 (100%), the manual chart review identified 14 (93%), and the patient-reported fall measure identified 5 (33%). Owing to relatively high numbers of false positives based on falls present on admission, the algorithm's positive predictive values were 50% (development sample) and 47% (validation sample). Instead of requiring 10 minutes per case for a full manual review or 20 minutes to apply the Global Trigger Tool, the algorithm requires only a few seconds, after which only the positive results (roughly 11% of the full case number) require review.ConclusionsThe newly developed electronic health record algorithm demonstrated very high sensitivity for fall detection. Applied in near real time, the algorithm can record in-hospital falls events effectively and help to develop and test fall prevention measures.

Project description:BackgroundMuscular dystrophies (MDs) are a group of inherited conditions characterized by progressive muscle degeneration and weakness. The rarity and heterogeneity of the population with MD have hindered therapeutic developments as well as epidemiological and health outcomes research. The objective of the study was to develop and validate a case-finding algorithm utilizing administrative claims data to identify and characterize patients with MD.MethodsThis retrospective cohort study used medical chart validation to evaluate an ICD-9/10 coding algorithm in a large commercial claims database. Patients were identified who had ≥2 office visits with a diagnosis of hereditary progressive MDs from January 1, 2013 through December 31, 2016, were male, and younger than 18 years at the time of first MD diagnosis. Cases who met the algorithm were then validated against medical charts. Diagnoses of MD and specific type (Duchenne, Becker, or other MD) were confirmed by medical chart review by trained reviewers. Positive predictive value (PPV) and 95% confidence intervals (CI) were calculated using a 2 × 2 contingence table. Patient demographic, clinical, and health utilization characteristics were summarized using basic descriptive statistics.ResultsCharts were obtained and reviewed for 109 patients who met the algorithm. The PPV of the case-identifying algorithm for MD was 95% (95% CI 88-98%). Of the 103 confirmed MD cases, 87 patients (85%, 95% CI 76-91%) had Duchenne or Becker MD; 76 patients (74%, 95% CI 64-82%) had Duchenne MD, and 11 patients (11%, 95% CI 5-18%) had Becker MD. A total of 74 (67.9%) patients had ≥1 pediatric complex chronic condition (other than neurologic/neuromuscular disease); 54 (49.5%) had cardiovascular conditions; 14 (12.8%) had respiratory conditions; 50 (45.9%) had bone-related issues; 11 (10.1%) had impaired growth; and 6 (5.5%) had puberty delay.ConclusionsThe results of this study demonstrate that the case-finding algorithm accurately identified patients with MD, primarily Duchenne MD, within a large administrative database. The algorithm, which was constructed using a few items easily accessible from claims, can be used to facilitate epidemiological and health outcomes research in the Duchenne patient population.

Project description:Automated microscopy is currently the only method to non-invasively and label-free observe complex multi-cellular processes, such as cell migration, cell cycle, and cell differentiation. Extracting biological information from a time-series of micrographs requires each cell to be recognized and followed through sequential microscopic snapshots. Although recent attempts to automatize this process resulted in ever improving cell detection rates, manual identification of identical cells is still the most reliable technique. However, its tedious and subjective nature prevented tracking from becoming a standardized tool for the investigation of cell cultures. Here, we present a novel method to accomplish automated cell tracking with a reliability comparable to manual tracking. Previously, automated cell tracking could not rival the reliability of manual tracking because, in contrast to the human way of solving this task, none of the algorithms had an independent quality control mechanism; they missed validation. Thus, instead of trying to improve the cell detection or tracking rates, we proceeded from the idea to automatically inspect the tracking results and accept only those of high trustworthiness, while rejecting all other results. This validation algorithm works independently of the quality of cell detection and tracking through a systematic search for tracking errors. It is based only on very general assumptions about the spatiotemporal contiguity of cell paths. While traditional tracking often aims to yield genealogic information about single cells, the natural outcome of a validated cell tracking algorithm turns out to be a set of complete, but often unconnected cell paths, i.e. records of cells from mitosis to mitosis. This is a consequence of the fact that the validation algorithm takes complete paths as the unit of rejection/acceptance. The resulting set of complete paths can be used to automatically extract important biological parameters with high reliability and statistical significance. These include the distribution of life/cycle times and cell areas, as well as of the symmetry of cell divisions and motion analyses. The new algorithm thus allows for the quantification and parameterization of cell culture with unprecedented accuracy. To evaluate our validation algorithm, two large reference data sets were manually created. These data sets comprise more than 320,000 unstained adult pancreatic stem cells from rat, including 2592 mitotic events. The reference data sets specify every cell position and shape, and assign each cell to the correct branch of its genealogic tree. We provide these reference data sets for free use by others as a benchmark for the future improvement of automated tracking methods.