Data extraction methods for systematic review (semi)automation: A living review protocol.
Ontology highlight
ABSTRACT:
SUBMITTER: Schmidt L
PROVIDER: S-EPMC7338918 | biostudies-literature | 2020
REPOSITORIES: biostudies-literature
Publications
Data extraction methods for systematic review (semi)automation: A living review protocol.
Schmidt Lena L Olorisade Babatunde K BK McGuinness Luke A LA Thomas James J Higgins Julian P T JPT
F1000Research 20200325
<b>Background:</b> Researchers in evidence-based medicine cannot keep up with the amounts of both old and newly published primary research articles. Support for the early stages of the systematic review process - searching and screening studies for eligibility - is necessary because it is currently impossible to search for relevant research with precision. Better automated data extraction may not only facilitate the stage of review traditionally labelled 'data extraction', but also change earlie ...[more]
Similar Datasets
Project description:Background: The reliable and usable (semi)automation of data extraction can support the field of systematic review by reducing the workload required to gather information about the conduct and results of the included studies. This living systematic review examines published approaches for data extraction from reports of clinical studies. Methods: We systematically and continually search MEDLINE, Institute of Electrical and Electronics Engineers (IEEE), arXiv, and the dblp computer science bibliography databases. Full text screening and data extraction are conducted within an open-source living systematic review application created for the purpose of this review. This iteration of the living review includes publications up to a cut-off date of 22 April 2020. Results: In total, 53 publications are included in this version of our review. Of these, 41 (77%) of the publications addressed extraction of data from abstracts, while 14 (26%) used full texts. A total of 48 (90%) publications developed and evaluated classifiers that used randomised controlled trials as the main target texts. Over 30 entities were extracted, with PICOs (population, intervention, comparator, outcome) being the most frequently extracted. A description of their datasets was provided by 49 publications (94%), but only seven (13%) made the data publicly available. Code was made available by 10 (19%) publications, and five (9%) implemented publicly available tools. Conclusions: This living systematic review presents an overview of (semi)automated data-extraction literature of interest to different types of systematic review. We identified a broad evidence base of publications describing data extraction for interventional reviews and a small number of publications extracting epidemiological or diagnostic accuracy data. The lack of publicly available gold-standard data for evaluation, and lack of application thereof, makes it difficult to draw conclusions on which is the best-performing system for each data extraction target. With this living review we aim to review the literature continually.
| S-EPMC8361807 | biostudies-literature
Project description:BackgroundThe exponential growth of the biomedical literature necessitates investigating strategies to reduce systematic reviewer burden while maintaining the high standards of systematic review validity and comprehensiveness.MethodsWe compared the traditional systematic review screening process with (1) a review-of-reviews (ROR) screening approach and (2) a semi-automation screening approach using two publicly available tools (RobotAnalyst and AbstrackR) and different types of training sets (randomly selected citations subjected to dual-review at the title-abstract stage, highly curated citations dually reviewed at the full-text stage, and a combination of the two). We evaluated performance measures of sensitivity, specificity, missed citations, and workload burden RESULTS: The ROR approach for treatments of early-stage prostate cancer had a poor sensitivity (0.54) and studies missed by the ROR approach tended to be of head-to-head comparisons of active treatments, observational studies, and outcomes of physical harms and quality of life. Title and abstract screening incorporating semi-automation only resulted in a sensitivity of 100% at high levels of reviewer burden (review of 99% of citations). A highly curated, smaller-sized, training set (n = 125) performed similarly to a larger training set of random citations (n = 938).ConclusionTwo approaches to rapidly update SRs-review-of-reviews and semi-automation-failed to demonstrate reduced workload burden while maintaining an acceptable level of sensitivity. We suggest careful evaluation of the ROR approach through comparison of inclusion criteria and targeted searches to fill evidence gaps as well as further research of semi-automation use, including more study of highly curated training sets.
| S-EPMC7574591 | biostudies-literature
Project description:BackgroundWith the accumulation of evidence regarding potential harms of cancer screening in recent years, researchers, policy-makers, and the public are becoming more critical of population-based cancer screening. Consequently, a high-quality cancer screening program should consider individuals’ values and preferences when determining recommendations. In cervical cancer screening, offering women autonomy is considered a “person-centered” approach to health care services; however, it may impact the effectiveness of the program should women choose to not participate. As part of a larger project to investigate women’s cervical screening preferences and correlates of these preferences, this systematic review will capture quantitative and qualitative investigations of women’s cervical screening preferences and the methods used to elicit them.Design and methodsThis mixed methods synthesis will use a thematic analysis approach to synthesize qualitative, quantitative, and mixed methods evidence. This protocol describes the methods that will be used in this investigation. A search strategy has been developed with a health librarian and peer reviewed using PRESS. Based on this strategy, five databases and the gray literature will be searched for studies that meet the inclusion criteria. The quality of the included individual studies will be examined using the Mixed Methods Appraisal Tool. Three reviewers will extract data from the primary studies on the tools or instruments used to elicit women’s preferences regarding cervical cancer screening, theoretical frameworks used, outcomes measured, the outstanding themes from quantitative and qualitative evidence, and the identified preferences for cervical cancer screening. We will describe the relationships between study results and the study population, “intervention” (e.g., tool or instrument), and context. We will follow the PRISMA reporting guideline. We will compare findings across studies and between study methods (e.g., qualitative versus quantitative study designs). The strength of the synthesized findings will be assessed using the validated GRADE and CERQual tool.DiscussionThis review will inform the development of a tool to elicit women’s cervical screening preferences. Understanding the methods used to elicit women’s preferences and what is known about women’s cervical screening preferences will be useful for guideline developers who wish to incorporate a woman-centered approach specifically for cervical screening guidelines.Systematic review registrationPROSPERO CRD42016035737Electronic supplementary materialThe online version of this article (doi:10.1186/s13643-016-0310-9) contains supplementary material, which is available to authorized users.
| S-EPMC4982264 | biostudies-other
Project description:Although many protocols have been previously developed for genomic DNA (gDNA) extraction from S. cerevisiae, to take advantage of recent advances in laboratory automation and DNA-barcode sequencing, there is a need for automated methods that can provide high-quality gDNA at high efficiency. Here, we describe and demonstrate a fully automated protocol that includes five basic steps: cell wall and RNA digestion, cell lysis, DNA binding to magnetic beads, washing with ethanol, and elution. Our protocol avoids the use of hazardous reagents (e.g., phenol, chloroform), glass beads for mechanical cell disruption, or incubation of samples at 100°C (i.e., boiling). We show that our protocol can extract gDNA with high efficiency both from cells grown in liquid culture and from colonies grown on agar plates. We also show results from gel electrophoresis that demonstrate that the resulting gDNA is of high quality.
| S-EPMC10581484 | biostudies-literature
Project description:Background: The living systematic review (LSR) is an emerging approach for improved evidence synthesis that uses continual updating to include relevant new evidence as soon as it is published. The objectives of this study are to: 1) assess the methods of conduct and reporting of living systematic reviews using a living study approach; and 2) describe the life cycle of living systematic reviews, i.e., describe the changes over time to their methods and findings. Methods: For objective 1, we will begin by conducting a cross-sectional survey and then update its findings every 6 months by including newly published LSRs. For objective 2, we will conduct a prospective longitudinal follow-up of the cohort of included LSRs. To identify LSRs, we will continually search the following electronic databases: Medline, EMBASE and the Cochrane library. We will also contact groups conducting LSRs to identify eligible studies that we might have missed. We will follow the standard systematic review methodology for study selection and data abstraction. For each LSR update, we will abstract information on the following: 1) general characteristics, 2) systematic review methodology, 3) living approach methodology, 4) results, and 5) editorial and publication processes. We will update the findings of both the surveys and the longitudinal follow-up of included LSRs every 6 months. In addition, we will identify articles addressing LSR methods to be included in an 'LSR methods repository'. Conclusion: The proposed living methodological survey will allow us to monitor how the methods of conduct, and reporting as well as the findings of LSRs change over time. Ultimately this should help with ensuring the quality and transparency of LSRs.
| S-EPMC6556985 | biostudies-literature
Project description:BackgroundOverviews of systematic reviews attempt to systematically retrieve and summarise the results of multiple systematic reviews. Methods for conducting, interpreting and reporting overviews are in their infancy. To date, there has been no evidence map of the methods used in overviews, thus making it difficult to determine the gaps and priorities for methods research. Our objectives were to develop and populate a comprehensive framework of methods for conducting, interpreting and reporting overviews (stage I) and to create an evidence map by mapping studies that have evaluated overview methods to the framework (stage II).MethodsWe searched methods collections (e.g. Cochrane Methodology Register, Meth4ReSyn library, AHRQ Effective Health Care Program) to identify eligible studies for both stages of this research. In stage I, cross-sectional studies, guidance documents and commentaries that described methods proposed for, or used in, overviews were used to develop and populate the framework of methods. Drafts and multiple iterations of the framework were discussed and refined by all authors. In stage II, we identified and described studies evaluating overview methods and mapped these evaluations to the framework.ResultsIn this paper, we present results for the four initial steps of conducting an overview: (a) specification of the purpose, objectives and scope, (b) specification of the eligibility criteria, (c) search methods and (d) data extraction. Twenty-nine studies mentioned or described methods relevant to one or more of these steps. In the developed framework, identified methods and approaches were grouped according to the steps an overview author would need to undertake. Fifteen studies evaluated identified methods, all of which mapped to the search methods step. These studies either reported the development and evaluation of a new search filter to retrieve systematic reviews or compared the performance of multiple filters.ConclusionGaps in the evaluation of methods were found for the majority of steps in the framework. More empirical studies are needed to evaluate the methods outlined and provide a comprehensive evidence map. The framework is useful for planning these evaluations and for planning methods required to deal with challenges that arise when conducting an overview.
| S-EPMC5698938 | biostudies-literature
Project description:BACKGROUND: The PCR-based detection of archaea DNA in human specimens relies on efficient DNA extraction. We previously designed one such protocol involving only manual steps. In an effort to reduce the workload involved, we compared this manual protocol to semi-automated and automated protocols for archaea DNA extraction from human specimens. FINDINGS: We tested 110 human stool specimens using each protocol. An automated protocol using the EZ1 Advanced XL extractor with the V 1.066069118 Qiagen DNA bacteria card and the EZ1® DNA Tissue Kit (Qiagen, Courtaboeuf, France) yielded 35/110 (32%) positives for the real-time PCR detection of the Methanobrevibacter smithii 16S rRNA gene, with average Ct values of 36.1. A semi-automated protocol combining glass-powder crushing, overnight proteinase K digestion and lysis in the buffer from the EZ1 kit yielded 90/110 (82%) positive specimens (P?=?0.001) with an average Ct value of 27.4 (P?=?0.001). The manual protocol yielded 100/110 (91%) positive specimens (P?=?0.001) with an average Ct value of 30.33 (P?=?0.001). However, neither the number of positive specimens nor the Ct values were significantly different between the manual protocol and the semi-automated protocol (P?>?0.1 and P?>?0.1). CONCLUSION: Proteinase K digestion and glass powder crushing dramatically increase the extraction yield of archaea DNA from human stools. The semi-automated protocol described here was more rapid than the manual protocol and yielded significantly more archaeal DNA. It could be applied for extracting total stool DNA for further PCR amplification.
| S-EPMC3662635 | biostudies-literature
Project description:Introduction:While potentially reducing decision errors, decision support systems can introduce new types of errors. Automation bias (AB) happens when users become overreliant on decision support, which reduces vigilance in information seeking and processing. Most research originates from the human factors literature, where the prevailing view is that AB occurs only in multitasking environments. Objectives:This review seeks to compare the human factors and health care literature, focusing on the apparent association of AB with multitasking and task complexity. Data sources:EMBASE, Medline, Compendex, Inspec, IEEE Xplore, Scopus, Web of Science, PsycINFO, and Business Source Premiere from 1983 to 2015. Study selection:Evaluation studies where task execution was assisted by automation and resulted in errors were included. Participants needed to be able to verify automation correctness and perform the task manually. Methods:Tasks were identified and grouped. Task and automation type and presence of multitasking were noted. Each task was rated for its verification complexity. Results:Of 890 papers identified, 40 met the inclusion criteria; 6 were in health care. Contrary to the prevailing human factors view, AB was found in single tasks, typically involving diagnosis rather than monitoring, and with high verification complexity. Limitations:The literature is fragmented, with large discrepancies in how AB is reported. Few studies reported the statistical significance of AB compared to a control condition. Conclusion:AB appears to be associated with the degree of cognitive load experienced in decision tasks, and appears to not be uniquely associated with multitasking. Strategies to minimize AB might focus on cognitive load reduction.
| S-EPMC7651899 | biostudies-literature
Project description:BackgroundPediatric epilepsy, including treatment-resistant forms, has a major effect on the quality of life, morbidity, and mortality of affected children. Interest has been growing in the use of medical cannabis as a treatment for pediatric epilepsy, yet there has been no comprehensive review of the benefits and harms of cannabis use in this population. In this systematic review, we will search for, synthesize, and assess the published and gray literature in order to provide usable and relevant information to parents, clinicians, and policy makers.MethodsWe will perform a living systematic review of studies involving the use of cannabis to treat pediatric epilepsy. We will search the published and gray literature for studies involving children with any type of epilepsy taking any form of cannabis. Studies will be selected for inclusion by two independent reviewers. The primary outcome is seizure freedom. Secondary outcomes are seizure frequency, quality of life (child, caregiver), quality and quantity of sleep, status epilepticus, tonic-clonic seizures, death (all-cause, sudden unexpected death in epilepsy), gastrointestinal adverse events (diarrhea, vomiting), and visits to the emergency room. The quality of each included study will be assessed. If data are sufficient in quantity and sufficiently similar, we will conduct pairwise random-effects meta-analysis. We will repeat the literature search every 6 months to identify studies published after the previous search date. Sequential meta-analysis will be performed as necessary to update the review findings.DiscussionOur review aims to provide a comprehensive and up-to-date summary of the available evidence to inform decisions about the use of cannabis in children with treatment-resistant epilepsy. The results of this review will be of use to parents, clinicians, and policy makers as they navigate this rapidly evolving area.Systematic review registrationPROSPERO CRD42018084755.
| S-EPMC6052624 | biostudies-literature
Project description:IntroductionHealthcare is increasingly challenged to meet the demands of user involvement and knowledge mobilisation required by the 21st-century patient-centred and knowledge-based economies. Innovations are needed to reduce problematic barriers to knowledge exchange and improve collaborative problem solving. Living labs, as open knowledge systems, have the potential to address these gaps but are underexplored in healthcare.Methods and analysisWe will conduct the first systematic review of living labs across healthcare contexts. We will comprehensively search the following online databases from inception to 31 December 2020: Scopus, the Cochrane Library (Wiley), Medline (OVID), Embase (OVID), Web of Science, PsycINFO (OVID) and EBSCOhost databases including Academic Search Complete, Business Source Premier, Canadian Reference Centre, CINAHL, MasterFILE Premier, SPORTDiscus, Library & Information Science Source, Library, Information Science & Technology Abstracts, AgeLine, EconLit, Art Full Text, Women's Studies International and Social Work Abstracts. We will search for grey literature using Google advanced techniques and books/book chapters through scholarly and bibliographical databases. We will use a dual-reviewer, two-step selection process with pre-established inclusion criteria and limit to English language publications. Empirical studies of any design examining living lab development, implementation or evaluation in health or healthcare will be included. We will use the Mixed Methods Appraisal Tool (MMAT) for methodological quality appraisal and Covidence software for review management, and we will extract data on pre-established variables such as lab context and technological platforms. We will create evidence tables and analyse across variables such as focal aim and achievement of living lab principles, such as the use of cocreation and multimethod approaches. We will tabulate data for descriptive reporting and narrative synthesis to identify current applications, approaches and promising areas for living lab development across health contexts.Ethics and disseminationEthical approval was not required for this review. This review will inform research into living labs in health environments, including guidance for a living lab in paediatric rehabilitation. Academic publications shared through collaborative networks and social media channels will provide substantive knowledge to the growing tech-health development sector and to researchers, practitioners and organisations seeking enhanced patient/stakeholder engagement and innovations in knowledge translation and evidence-based practice.Prospero registration numberCRD42020175275.
| S-EPMC7925912 | biostudies-literature