Dataset Information

Automated Comparison of Pathology Reports for On-the-job Assessment of Residents.

ABSTRACT: The clinical competency of residents at teaching hospitals is always under scrutiny. Ideally, assessment should reflect competency on-the-job, under realistic circumstances, and include evaluating their medical reports. Currently, the assessment is done manually by the attending physicians, which adds to the cognitive load. In this study, we developed an automated system for assessing medical resident's pathology reports. Our system used natural language processing (NLP) techniques to identify different lexical and semantic similarity scores at sentence level as well as chunk level. We then used supervised learning to classify the reports into three categories- No Change (NC), Minor Changes (MiC), and major changes (MaC), reflecting how much the attending physician's report differs from that of the resident. Our system was able to classify the reports with an accuracy of 73.6%. Although moderately successful, our work shows the potential and future of automated assessment systems in the biomedical domain.

SUBMITTER: Chandrashekar PB

PROVIDER: S-EPMC6568070 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Automated Comparison of Pathology Reports for On-the-job Assessment of Residents.

Chandrashekar Pramod Bharadwaj PB Montone Kathleen K Feldman Michael D MD Gonzalez-Heranndez Graciela G

AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science 20190506

The clinical competency of residents at teaching hospitals is always under scrutiny. Ideally, assessment should reflect competency on-the-job, under realistic circumstances, and include evaluating their medical reports. Currently, the assessment is done manually by the attending physicians, which adds to the cognitive load. In this study, we developed an automated system for assessing medical resident's pathology reports. Our system used natural language processing (NLP) techniques to identify d ...[more]

PMID: 31259004

Similar Datasets

Project description:Background and objectivesThe growing interest in the impact of lifetime occupational exposures on later-life health underscores the need to expand and evaluate the quality of data resources. The present study took advantage of a retrospective life history survey fielded within the context of the Health and Retirement Study to assess the accuracy of retrospectively obtained information on job history. We evaluated hypotheses related to job history and respondent characteristics to understand more about factors associated with recall accuracy.Research design and methodsWe used data from the Life History Mail Survey (LHMS), a self-administered survey conducted in 2015 and 2017. We compared the match rate of work status collected in the LHMS questionnaire with data collected concurrently during HRS core face-to-face or phone interviews from 1992 through 2016 with respect to jobs held at the time of the interview. We also conducted a limited set of comparisons of occupation and industry match.ResultsThe sample was 61.79% women, 82.12% White, and 8.57% Hispanic with a mean age of 74.70 years. The overall work status match rate was 83%. Jobs held longer ago were recalled with less accuracy. Jobs held for longer durations and that were full-time rather than part-time were recalled with greater accuracy. More complex job histories that involved a larger number of jobs were also associated with a lower match rate. Higher levels of conscientiousness and cognitive functioning were both associated with a higher match between the two sources of work status information. The occupation match rate was 69%, and the industry match rate was 77%.Discussion and implicationsA self-administered, paper-and-pencil questionnaire attempting to measure decades-long histories of autobiographically important dimensions of life can provide reasonably accurate historical employment information. Several factors are likely to influence the relative accuracy of recalled information.

Project description:ObjectivesAutomatic job coding tools were developed to reduce the laborious task of manually assigning job codes based on free-text job descriptions in census and survey data sources, including large occupational health studies. The objective of this study is to provide a case study of comparative performance of job coding and JEM (Job-Exposure Matrix)-assigned exposures agreement using existing coding tools.MethodsWe compared three automatic job coding tools [AUTONOC, CASCOT (Computer-Assisted Structured Coding Tool), and LabourR], which were selected based on availability, coding of English free-text into coding systems closely related to the 1988 version of the International Standard Classification of Occupations (ISCO-88), and capability to perform batch coding. We used manually coded job histories from the AsiaLymph case-control study that were translated into English prior to auto-coding to assess their performance. We applied two general population JEMs to assess agreement at exposure level. Percent agreement and PABAK (Prevalence-Adjusted Bias-Adjusted Kappa) were used to compare the agreement of results from manual coders and automatic coding tools.ResultsThe coding per cent agreement among the three tools ranged from 17.7 to 26.0% for exact matches at the most detailed 4-digit ISCO-88 level. The agreement was better at a more general level of job coding (e.g. 43.8-58.1% in 1-digit ISCO-88), and in exposure assignments (median values of PABAK coefficient ranging 0.69-0.78 across 12 JEM-assigned exposures). Based on our testing data, CASCOT was found to outperform others in terms of better agreement in both job coding (26% 4-digit agreement) and exposure assignment (median kappa 0.61).ConclusionsIn this study, we observed that agreement on job coding was generally low for the three tools but noted a higher degree of agreement in assigned exposures. The results indicate the need for study-specific evaluations prior to their automatic use in general population studies, as well as improvements in the evaluated automatic coding tools.

Project description:ObjectiveThis study compares the intraoperative phase times in laparoscopic cholecystectomy performed by an attending surgeon and supervised residents over 10-years to assess operative times as a marker of performance and any impact of case severity on times.DesignLaparoscopic cholecystectomy videos were uploaded to Touch Surgery™ Enterprise, a combined software and hardware solution for securely recording, storing, and analysing surgical videos, which provide analytics of intraoperative phase times. Case severity and visualisation of the critical view of safety (CVS) were manually assessed using modified 10-point intraoperative gallbladder scoring system (mG10) and CVS scores, respectively. Attending and residents' times were compared unmatched and matched by mG10.SettingSecondary analysis of anonymized laparoscopic cholecystectomy video, recorded as standard of care.ParticipantsAdult patients who underwent elective laparoscopic cholecystectomy a single UK hospital. Cases were performed by one attending and their residents.Results159 (attending=96, resident=63) laparoscopic cholecystectomy videos and intraoperative phase times were reviewed on Touch Surgery™ Enterprise and analyzed. Attending cases were more challenging (p=0.037). Residents achieved higher CVS scores (p=0.034) and showed longer dissection of hepatocystic triangle (HCT) times (p=0.012) in more challenging cases. Residents' total operative time (p=0.001) and dissection of HCT (p=0.002) times exceeded the attending's in low-severity matched cases (mG10=1). Residents' total operative times (p<0.001), port insertion/gallbladder exposure (p=0.032), and dissection of HCT (p<0.001) exceeded the attending's in matched cases (mG10=2). Residents' total operative (p<0.001), dissection of HCT (p<0.001), and gallbladder dissection (p=0.010) times exceeded the attendings in unmatched cases.ConclusionsResidents' total operative and dissection of HCT times significantly exceeded the attending's unmatched cases and low-severity matched cases which could suggest training need, however, also reflects an expected assessment of competence, and validates time as a marker of performance.

Dataset Information

Automated Comparison of Pathology Reports for On-the-job Assessment of Residents.

Publications

Automated Comparison of Pathology Reports for On-the-job Assessment of Residents.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets