Dataset Information

Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study.

ABSTRACT: BACKGROUND:Community-acquired pneumonia is a leading cause of pediatric morbidity. Administrative data are often used to conduct comparative effectiveness research (CER) with sufficient sample sizes to enhance detection of important outcomes. However, such studies are prone to misclassification errors because of the variable accuracy of discharge diagnosis codes. OBJECTIVE:The aim of this study was to develop an automated, scalable, and accurate method to determine the presence or absence of pneumonia in children using chest imaging reports. METHODS:The multi-institutional PHIS+ clinical repository was developed to support pediatric CER by expanding an administrative database of children's hospitals with detailed clinical data. To develop a scalable approach to find patients with bacterial pneumonia more accurately, we developed a Natural Language Processing (NLP) application to extract relevant information from chest diagnostic imaging reports. Domain experts established a reference standard by manually annotating 282 reports to train and then test the NLP application. Findings of pleural effusion, pulmonary infiltrate, and pneumonia were automatically extracted from the reports and then used to automatically classify whether a report was consistent with bacterial pneumonia. RESULTS:Compared with the annotated diagnostic imaging reports reference standard, the most accurate implementation of machine learning algorithms in our NLP application allowed extracting relevant findings with a sensitivity of .939 and a positive predictive value of .925. It allowed classifying reports with a sensitivity of .71, a positive predictive value of .86, and a specificity of .962. When compared with each of the domain experts manually annotating these reports, the NLP application allowed for significantly higher sensitivity (.71 vs .527) and similar positive predictive value and specificity . CONCLUSIONS:NLP-based pneumonia information extraction of pediatric diagnostic imaging reports performed better than domain experts in this pilot study. NLP is an efficient method to extract information from a large collection of imaging reports to facilitate CER.

SUBMITTER: Meystre S

PROVIDER: S-EPMC5447826 | biostudies-literature | 2017 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study.

Meystre Stephane S Gouripeddi Ramkiran R Tieder Joel J Simmons Jeffrey J Srivastava Rajendu R Shah Samir S

Journal of medical Internet research 20170515 5

<h4>Background</h4>Community-acquired pneumonia is a leading cause of pediatric morbidity. Administrative data are often used to conduct comparative effectiveness research (CER) with sufficient sample sizes to enhance detection of important outcomes. However, such studies are prone to misclassification errors because of the variable accuracy of discharge diagnosis codes.<h4>Objective</h4>The aim of this study was to develop an automated, scalable, and accurate method to determine the presence or ...[more]

PMID: 28506958

Similar Datasets

Project description:The National Institutes of Health (NIH) established the Building University Infrastructure Leading to Diversity (BUILD) initiative to increase engagement and retention of undergraduates from diverse backgrounds in biomedical research. Portland State University, in partnership with ten other academic institutions, received a BUILD award and developed the BUILD EXITO (Enhancing Cross-Disciplinary Infrastructure and Training at Oregon) project. The EXITO program offers a three-year research and mentorship experience for undergraduates in biomedical, behavioral, social science, clinical, and bioengineering disciplines. The BUILD initiative also emphasizes enhancing research capacity and infrastructure through institutional change and faculty development. A key piece of EXITO's program to enhance research capacity is offering faculty an opportunity to apply for up to $50,000 of funding to carry out a one-year pilot study. We conducted two separate RFAs for this purpose, closely modeled on NIH's Small Grant Program (R03), over two years. Principal Investigators of pilot projects were encouraged to include EXITO students, or other undergraduate students, on their research teams. Students then worked on these research projects as part of EXITO's intensive mentored research program. This paper reports on methods to conduct and implement a pilot project program intended to train primarily junior faculty members to write and submit an NIH proposal and fund successful applicants to gather pilot project data to aid in applying for future proposals. We provided a step-by-step rigorous submission and review process. We provided proposal writing and revising workshops, technical support, and helped pilot project Principal Investigators (PIs) with biosketches, IRB applications, IUCUC documents, budgets, and other proposal sections. We secured at least three external (not at any BUILD EXITO institution) reviewers for each proposal. PIs revised proposals before resubmitting and receiving their final scores. Across two RFAs, we provided funds twenty PIs to conduct pilot projects; these projects included at least 21 students working on them who received mentoring in research methods and in disseminating results. This paper describes important lessons learned, including the importance of: allotting sufficient time to recruit reviewers; recruiting reviewers through a variety of sources and methods; and assisting PIs in engaging with research administration staff at Portland State University and partner institutions. Challenges included: finding an optimal timeline that was neither too compressed nor too stretched out; encouraging applicants from distant partner institutions to apply and keeping them engaged and retained through the entire process; and assisting PIs from partner institutions to efficiently utilize Portland State University's sponsored projects department if similar resources were not available at their home institutions. Our goal is to provide guidance and insights to faculty and research-administration staff at other institutions interested in replicating or adapting EXITO's program to enhance institutional research capacity.

Project description:BackgroundCurrent automated planning solutions are calibrated using trial and error or machine learning on historical datasets. Neither method allows for the intuitive exploration of differing trade-off options during calibration, which may aid in ensuring automated solutions align with clinical preference. Pareto navigation provides this functionality and offers a potential calibration alternative. The purpose of this study was to validate an automated radiotherapy planning solution with a novel multi-dimensional Pareto navigation calibration interface across two external institutions for prostate cancer.MethodsThe implemented 'Pareto Guided Automated Planning' (PGAP) methodology was developed in RayStation using scripting and consisted of a Pareto navigation calibration interface built upon a 'Protocol Based Automatic Iterative Optimisation' planning framework. 30 previous patients were randomly selected by each institution (IA and IB), 10 for calibration and 20 for validation. Utilising the Pareto navigation interface automated protocols were calibrated to the institutions' clinical preferences. A single automated plan (VMATAuto) was generated for each validation patient with plan quality compared against the previously treated clinical plan (VMATClinical) both quantitatively, using a range of DVH metrics, and qualitatively through blind review at the external institution.ResultsPGAP led to marked improvements across the majority of rectal dose metrics, with Dmean reduced by 3.7 Gy and 1.8 Gy for IA and IB respectively (p < 0.001). For bladder, results were mixed with low and intermediate dose metrics reduced for IB but increased for IA. Differences, whilst statistically significant (p < 0.05) were small and not considered clinically relevant. The reduction in rectum dose was not at the expense of PTV coverage (D98% was generally improved with VMATAuto), but was somewhat detrimental to PTV conformality. The prioritisation of rectum over conformality was however aligned with preferences expressed during calibration and was a key driver in both institutions demonstrating a clear preference towards VMATAuto, with 31/40 considered superior to VMATClinical upon blind review.ConclusionsPGAP enabled intuitive adaptation of automated protocols to an institution's planning aims and yielded plans more congruent with the institution's clinical preference than the locally produced manual clinical plans.

Dataset Information

Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study.

Publications

Enhancing Comparative Effectiveness Research With Automated Pediatric Pneumonia Detection in a Multi-Institutional Clinical Repository: A PHIS+ Pilot Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets