Dataset Information

Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study.

ABSTRACT: BACKGROUND:Community-acquired pneumonia is a leading cause of pediatric morbidity. Administrative data are often used to conduct comparative effectiveness research (CER) with sufficient sample sizes to enhance detection of important outcomes. However, such studies are prone to misclassification errors because of the variable accuracy of discharge diagnosis codes. OBJECTIVE:The aim of this study was to develop an automated, scalable, and accurate method to determine the presence or absence of pneumonia in children using chest imaging reports. METHODS:The multi-institutional PHIS+ clinical repository was developed to support pediatric CER by expanding an administrative database of children's hospitals with detailed clinical data. To develop a scalable approach to find patients with bacterial pneumonia more accurately, we developed a Natural Language Processing (NLP) application to extract relevant information from chest diagnostic imaging reports. Domain experts established a reference standard by manually annotating 282 reports to train and then test the NLP application. Findings of pleural effusion, pulmonary infiltrate, and pneumonia were automatically extracted from the reports and then used to automatically classify whether a report was consistent with bacterial pneumonia. RESULTS:Compared with the annotated diagnostic imaging reports reference standard, the most accurate implementation of machine learning algorithms in our NLP application allowed extracting relevant findings with a sensitivity of .939 and a positive predictive value of .925. It allowed classifying reports with a sensitivity of .71, a positive predictive value of .86, and a specificity of .962. When compared with each of the domain experts manually annotating these reports, the NLP application allowed for significantly higher sensitivity (.71 vs .527) and similar positive predictive value and specificity . CONCLUSIONS:NLP-based pneumonia information extraction of pediatric diagnostic imaging reports performed better than domain experts in this pilot study. NLP is an efficient method to extract information from a large collection of imaging reports to facilitate CER.

SUBMITTER: Meystre S

PROVIDER: S-EPMC5447826 | biostudies-literature | 2017 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study.

Meystre Stephane S Gouripeddi Ramkiran R Tieder Joel J Simmons Jeffrey J Srivastava Rajendu R Shah Samir S

Journal of medical Internet research 20170515 5

<h4>Background</h4>Community-acquired pneumonia is a leading cause of pediatric morbidity. Administrative data are often used to conduct comparative effectiveness research (CER) with sufficient sample sizes to enhance detection of important outcomes. However, such studies are prone to misclassification errors because of the variable accuracy of discharge diagnosis codes.<h4>Objective</h4>The aim of this study was to develop an automated, scalable, and accurate method to determine the presence or ...[more]

PMID: 28506958

Similar Datasets

Project description:The National Institutes of Health (NIH) established the Building University Infrastructure Leading to Diversity (BUILD) initiative to increase engagement and retention of undergraduates from diverse backgrounds in biomedical research. Portland State University, in partnership with ten other academic institutions, received a BUILD award and developed the BUILD EXITO (Enhancing Cross-Disciplinary Infrastructure and Training at Oregon) project. The EXITO program offers a three-year research and mentorship experience for undergraduates in biomedical, behavioral, social science, clinical, and bioengineering disciplines. The BUILD initiative also emphasizes enhancing research capacity and infrastructure through institutional change and faculty development. A key piece of EXITO's program to enhance research capacity is offering faculty an opportunity to apply for up to $50,000 of funding to carry out a one-year pilot study. We conducted two separate RFAs for this purpose, closely modeled on NIH's Small Grant Program (R03), over two years. Principal Investigators of pilot projects were encouraged to include EXITO students, or other undergraduate students, on their research teams. Students then worked on these research projects as part of EXITO's intensive mentored research program. This paper reports on methods to conduct and implement a pilot project program intended to train primarily junior faculty members to write and submit an NIH proposal and fund successful applicants to gather pilot project data to aid in applying for future proposals. We provided a step-by-step rigorous submission and review process. We provided proposal writing and revising workshops, technical support, and helped pilot project Principal Investigators (PIs) with biosketches, IRB applications, IUCUC documents, budgets, and other proposal sections. We secured at least three external (not at any BUILD EXITO institution) reviewers for each proposal. PIs revised proposals before resubmitting and receiving their final scores. Across two RFAs, we provided funds twenty PIs to conduct pilot projects; these projects included at least 21 students working on them who received mentoring in research methods and in disseminating results. This paper describes important lessons learned, including the importance of: allotting sufficient time to recruit reviewers; recruiting reviewers through a variety of sources and methods; and assisting PIs in engaging with research administration staff at Portland State University and partner institutions. Challenges included: finding an optimal timeline that was neither too compressed nor too stretched out; encouraging applicants from distant partner institutions to apply and keeping them engaged and retained through the entire process; and assisting PIs from partner institutions to efficiently utilize Portland State University's sponsored projects department if similar resources were not available at their home institutions. Our goal is to provide guidance and insights to faculty and research-administration staff at other institutions interested in replicating or adapting EXITO's program to enhance institutional research capacity.

Dataset Information

Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study.

Publications

Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure