Increasing value and reducing waste in data extraction for systematic reviews: tracking data in data extraction forms.
Ontology highlight
ABSTRACT:
SUBMITTER: Shokraneh F
PROVIDER: S-EPMC5544999 | biostudies-other | 2017 Aug
REPOSITORIES: biostudies-other
Similar Datasets
Project description:The methods and results of health research are documented in study protocols, full study reports (detailing all analyses), journal reports, and participant-level datasets. However, protocols, full study reports, and participant-level datasets are rarely available, and journal reports are available for only half of all studies and are plagued by selective reporting of methods and results. Furthermore, information provided in study protocols and reports varies in quality and is often incomplete. When full information about studies is inaccessible, billions of dollars in investment are wasted, bias is introduced, and research and care of patients are detrimentally affected. To help to improve this situation at a systemic level, three main actions are warranted. First, academic institutions and funders should reward investigators who fully disseminate their research protocols, reports, and participant-level datasets. Second, standards for the content of protocols and full study reports and for data sharing practices should be rigorously developed and adopted for all types of health research. Finally, journals, funders, sponsors, research ethics committees, regulators, and legislators should endorse and enforce policies supporting study registration and wide availability of journal reports, full study reports, and participant-level datasets.
| S-EPMC4533904 | biostudies-literature
Project description:After identification of an important research question and selection of an appropriate study design, waste can arise from the regulation, governance, and management of biomedical research. Obtaining regulatory and governance approval has become increasingly burdensome and disproportionate to the conceivable risks to research participants. Regulation and governance involve interventions that are assumed to be justified in the interests of patients and the public, but they can actually compromise these interests. Inefficient management of the procedural conduct of research is wasteful, especially if it results in poor recruitment and retention of participants in well designed studies addressing important questions. These sources of waste can be minimised if the following four recommendations are addressed. First, regulators should use their influence to reduce other causes of waste and inefficiency in research. Second, regulators and policy makers should work with researchers, patients, and health professionals to streamline and harmonise the laws, regulations, guidelines, and processes that govern whether and how research can be done, and ensure that they are proportionate to the plausible risks associated with the research. Third, researchers and research managers should increase the efficiency of recruitment, retention, data monitoring, and data sharing in research through use of research designs known to reduce inefficiencies, and further research should be done to learn how efficiency can be increased. Finally, everyone, particularly those responsible for health-care systems, should promote integration of research into everyday clinical practice. Regulators and researchers should monitor adherence to each of these recommendations and publish metrics.
| S-EPMC3952153 | biostudies-literature
Project description:Correctable weaknesses in the design, conduct, and analysis of biomedical and public health research studies can produce misleading results and waste valuable resources. Small effects can be difficult to distinguish from bias introduced by study design and analyses. An absence of detailed written protocols and poor documentation of research is common. Information obtained might not be useful or important, and statistical precision or power is often too low or used in a misleading way. Insufficient consideration might be given to both previous and continuing studies. Arbitrary choice of analyses and an overemphasis on random extremes might affect the reported findings. Several problems relate to the research workforce, including failure to involve experienced statisticians and methodologists, failure to train clinical researchers and laboratory scientists in research methods and design, and the involvement of stakeholders with conflicts of interest. Inadequate emphasis is placed on recording of research decisions and on reproducibility of research. Finally, reward systems incentivise quantity more than quality, and novelty more than reliability. We propose potential solutions for these problems, including improvements in protocols and documentation, consideration of evidence from studies in progress, standardisation of research efforts, optimisation and training of an experienced and non-conflicted scientific workforce, and reconsideration of scientific reward systems.
| S-EPMC4697939 | biostudies-literature
Project description:BackgroundData extraction forms link systematic reviews with primary research and provide the foundation for appraising, analysing, summarising and interpreting a body of evidence. This makes their development, pilot testing and use a crucial part of the systematic reviews process. Several studies have shown that data extraction errors are frequent in systematic reviews, especially regarding outcome data.MethodsWe reviewed guidance on the development and pilot testing of data extraction forms and the data extraction process. We reviewed four types of sources: 1) methodological handbooks of systematic review organisations (SRO); 2) textbooks on conducting systematic reviews; 3) method documents from health technology assessment (HTA) agencies and 4) journal articles. HTA documents were retrieved in February 2019 and database searches conducted in December 2019. One author extracted the recommendations and a second author checked them for accuracy. Results are presented descriptively.ResultsOur analysis includes recommendations from 25 documents: 4 SRO handbooks, 11 textbooks, 5 HTA method documents and 5 journal articles. Across these sources the most common recommendations on form development are to use customized or adapted standardised extraction forms (14/25); provide detailed instructions on their use (10/25); ensure clear and consistent coding and response options (9/25); plan in advance which data are needed (9/25); obtain additional data if required (8/25); and link multiple reports of the same study (8/25). The most frequent recommendations on piloting extractions forms are that forms should be piloted on a sample of studies (18/25); and that data extractors should be trained in the use of the forms (7/25). The most frequent recommendations on data extraction are that extraction should be conducted by at least two people (17/25); that independent parallel extraction should be used (11/25); and that procedures to resolve disagreements between data extractors should be in place (14/25).ConclusionsOverall, our results suggest a lack of comprehensiveness of recommendations. This may be particularly problematic for less experienced reviewers. Limitations of our method are the scoping nature of the review and that we did not analyse internal documents of health technology agencies.
| S-EPMC7574308 | biostudies-literature
Project description:BackgroundCompelling evidence has demonstrated that a large proportion of investment in biomedical research is wasted; this waste is avoidable. Academic institutions have, thus far, shown limited response to recommendations for increasing value and reducing waste. We formulated an academic response by (i) achieving consensus across a wide range of stakeholder groups on a comprehensive framework for quality of patient-oriented clinical research and (ii) highlighting first successful examples of its operationalization to facilitate waste-reducing strategies at academic institutions.Methods and findingsBased on a systematic review of quality definitions, concepts, and criteria in the medical literature (systematic MEDLINE search up to February 15, 2015, with independent and in duplicate article selection) and on stakeholder websites from 13 countries (Australia, Austria, Canada, France, Germany, Italy, Japan, Norway, Spain, Sweden, Switzerland, United Kingdom, and United States), we systematically developed a comprehensive framework for the quality of clinical research. We identified websites through personal contacts with experts in clinical research or public health who also suggested, for each country, websites of the following 7 stakeholder groups: patient organizations; academic research infrastructures; governmental bodies; regulatory agencies; ethics committees; the pharmaceutical industry; and funding agencies. In addition, we searched websites of inter- or supranational bodies involved in clinical research until no further insights emerged. After consolidation of the identified definitions, concepts, and criteria of quality in a basic framework structure, we conducted 4 rounds of an adapted online Delphi process among the same 7 stakeholder groups from 16 countries. The Delphi process ultimately achieved consensus on structure and content. The framework addresses 5 study stages (concept, planning and feasibility, conduct, analysis and interpretation, and reporting and knowledge translation) and includes the following dimensions: (i) protection of patient safety and rights, (ii) relevance/patient centeredness and involvement, (iii) minimization of bias (internal validity), (iv) precision, (v) transparency/access to data, and (vi) generalizability (external validity) of study results. These dimensions interact with 2 promoters-infrastructure and sustainability through education-that include a set of factors that may enhance all listed quality dimensions. Each quality dimension contains specific questions and explanatory items that guide quality assessment at each research stage from conceptualization of the research question through reporting and knowledge translation of study results. In the last survey round, Delphi participants from 9 countries (Austria, Australia, Canada, Germany, Italy, the Netherlands, Switzerland, UK, and US) agreed on the structure, content, and wording of the research stages, quality dimensions, specific questions, and descriptive examples of the final framework. In Switzerland, INQUIRE has resulted in a roadmap that guides initiatives to increase value within the Swiss Clinical Trial Organization network and through affiliated researchers.ConclusionsWe present a framework based on a consensus of different stakeholder groups guiding the practical assessment of clinical research quality at all stages of a research project. Operationalization of this common structure will support the increase of value by guiding academic institutions and researchers in developing quality enhancement initiatives, from posing the right research question to the transparent publication of results.
| S-EPMC5991651 | biostudies-literature
Project description:Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling studies. Please see later in the article for the Editors' Summary.
| S-EPMC4196729 | biostudies-literature
Project description:Citrus fruits are extensively grown and much consumed around the world. Eighteen percent of total citrus cultivars are destined for industrial processes, and as a consequence, large amounts of waste are generated. Citrus waste is a potential source of high biological value compounds, which can be used in the food, pharmaceutical, and cosmetic industries but whose final disposal may pose a problem due to economic and environmental factors. At the same time, the emerging need to reduce the environmental impact of citrus waste and its responsible management has increased. For these reasons, the study of the use of non-conventional methods to extract high biological value compounds such as carotenoids, polyphenols, essential oils, and pectins from this type of waste has become more urgent in recent years. In this review, the effectiveness of technologies such as ultrasound assisted extraction, microwave assisted extraction, supercritical fluid extraction, pressurized water extraction, pulsed electric field, high-voltage electric discharges, and high hydrostatic pressures is described and assessed. A wide range of information concerning the principal non-conventional methods employed to obtain high-biological-value compounds from citrus waste as well as the most influencing factors about each technology are considered.
| S-EPMC7353614 | biostudies-literature
Project description:BackgroundPrevious research on data extraction methods in systematic reviews has focused on single aspects of the process. We aimed to provide a deeper insight into these methods by analysing a current sample of reviews.MethodsWe included systematic reviews of health interventions in humans published in English. We analysed 75 Cochrane reviews from May and June 2020 and a random sample of non-Cochrane reviews published in the same period and retrieved from Medline. We linked reviews with protocols and study registrations. We collected information on preparing, piloting, and performing data extraction and on use of software to assist review conduct (automation tools). Data were extracted by one author, with 20% extracted in duplicate. Data were analysed descriptively.ResultsOf the 152 included reviews, 77 reported use of a standardized extraction form (51%); 42 provided information on the type of form used (28%); 24 on piloting (16%); 58 on what data was collected (38%); 133 on the extraction method (88%); 107 on resolving disagreements (70%); 103 on methods to obtain additional data or information (68%); 52 on procedures to avoid data errors (34%); and 47 on methods to deal with multiple study reports (31%). Items were more frequently reported in Cochrane than non-Cochrane reviews. The data extraction form used was published in 10 reviews (7%). Use of software was rarely reported except for statistical analysis software and use of RevMan and GRADEpro GDT in Cochrane reviews. Covidence was the most frequent automation tool used: 18 reviews used it for study selection (12%) and 9 for data extraction (6%).ConclusionsReporting of data extraction methods in systematic reviews is limited, especially in non-Cochrane reviews. This includes core items of data extraction such as methods used to manage disagreements. Few reviews currently use software to assist data extraction and review conduct. Our results can serve as a baseline to assess the uptake of such tools in future analyses.
| S-EPMC8571672 | biostudies-literature
Project description:BACKGROUND:Systematic reviews are critical for obtaining accurate estimates of diagnostic test accuracy, yet these require extracting information buried in free text articles, an often laborious process. OBJECTIVE:We create a dataset describing the data extraction and synthesis processes in 63 DTA systematic reviews, and demonstrate its utility by using it to replicate the data synthesis in the original reviews. METHOD:We construct our dataset using a custom automated extraction pipeline complemented with manual extraction, verification, and post-editing. We evaluate using manual assessment by two annotators and by comparing against data extracted from source files. RESULTS:The constructed dataset contains 5,848 test results for 1,354 diagnostic tests from 1,738 diagnostic studies. We observe an extraction error rate of 0.06-0.3%. CONCLUSIONS:This constitutes the first dataset describing the later stages of the DTA systematic review process, and is intended to be useful for automating or evaluating the process.
| S-EPMC6371350 | biostudies-literature
Project description:BackgroundData extraction (DE) is a challenging step in systematic reviews (SRs). Complex SRs can involve multiple interventions and/or outcomes and encompass multiple research questions. Attempts have been made to clarify DE aspects focusing on the subsequent meta-analysis; there are, however, no guidelines for DE in complex SRs. Comparing datasets extracted independently by pairs of reviewers to detect discrepancies is also cumbersome, especially when the number of extracted variables and/or studies is colossal. This work aims to provide a set of practical steps to help SR teams design and build DE tools and compare extracted data for complex SRs.MethodsWe provided a 10-step guideline, from determining data items and structure to data comparison, to help identify discrepancies and solve data disagreements between reviewers. The steps were organised into three phases: planning and building the database and data manipulation. Each step was described and illustrated with examples, and relevant references were provided for further guidance. A demonstration example was presented to illustrate the application of Epi Info and R in the database building and data manipulation phases. The proposed guideline was also summarised and compared with previous DE guidelines.ResultsThe steps of this guideline are described generally without focusing on a particular software application or meta-analysis technique. We emphasised determining the organisational data structure and highlighted its role in the subsequent steps of database building. In addition to the minimal programming skills needed, creating relational databases and data validation features of Epi info can be utilised to build DE tools for complex SRs. However, two R libraries are needed to facilitate data comparison and solve discrepancies.ConclusionsWe hope adopting this guideline can help review teams construct DE tools that suit their complex review projects. Although Epi Info depends on proprietary software for data storage, it can still be a potential alternative to other commercial DE software for completing complex reviews.
| S-EPMC10691069 | biostudies-literature