Dataset Information

Electronic search strategies to identify reports of cluster randomized trials in MEDLINE: low precision will improve with adherence to reporting standards.

ABSTRACT:

Background

Cluster randomized trials (CRTs) present unique methodological and ethical challenges. Researchers conducting systematic reviews of CRTs (e.g., addressing methodological or ethical issues) require efficient electronic search strategies (filters or hedges) to identify trials in electronic databases such as MEDLINE. According to the CONSORT statement extension to CRTs, the clustered design should be clearly identified in titles or abstracts; however, variability in terminology may make electronic identification challenging. Our objectives were to (a) evaluate sensitivity ("recall") and precision of a well-known electronic search strategy ("randomized controlled trial" as publication type) with respect to identifying CRTs, (b) evaluate the feasibility of new search strategies targeted specifically at CRTs, and (c) determine whether CRTs are appropriately identified in titles or abstracts of reports and whether there has been improvement over time.

Methods

We manually examined a wide range of health journals to identify a gold standard set of CRTs. Search strategies were evaluated against the gold standard set, as well as an independent set of CRTs included in previous systematic reviews.

Results

The existing strategy (randomized controlled trial.pt) is sensitive (93.8%) for identifying CRTs, but has relatively low precision (9%, number needed to read 11); the number needed to read can be halved to 5 (precision 18.4%) by combining with cluster design-related terms using the Boolean operator AND; combining with the Boolean operator OR maximizes sensitivity (99.4%) but would require 28.6 citations read to identify one CRT. Only about 50% of CRTs are clearly identified as cluster randomized in titles or abstracts; approximately 25% can be identified based on the reported units of randomization but are not amenable to electronic searching; the remaining 25% cannot be identified except through manual inspection of the full-text article. The proportion of trials clearly identified has increased from 28% between the years 2000-2003, to 60% between 2004-2007 (absolute increase 32%, 95% CI 17 to 47%).

Conclusions

CRTs should include the phrase "cluster randomized trial" in titles or abstracts; this will facilitate more accurate indexing of the publication type by reviewers at the National Library of Medicine, and efficient textword retrieval of the subset employing cluster randomization.

SUBMITTER: Taljaard M

PROVIDER: S-EPMC2833170 | biostudies-literature | 2010 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Electronic search strategies to identify reports of cluster randomized trials in MEDLINE: low precision will improve with adherence to reporting standards.

Taljaard Monica M McGowan Jessie J Grimshaw Jeremy M JM Brehaut Jamie C JC McRae Andrew A Eccles Martin P MP Donner Allan A

BMC medical research methodology 20100216

<h4>Background</h4>Cluster randomized trials (CRTs) present unique methodological and ethical challenges. Researchers conducting systematic reviews of CRTs (e.g., addressing methodological or ethical issues) require efficient electronic search strategies (filters or hedges) to identify trials in electronic databases such as MEDLINE. According to the CONSORT statement extension to CRTs, the clustered design should be clearly identified in titles or abstracts; however, variability in terminology m ...[more]

PMID: 20158899

Similar Datasets

Project description:BackgroundCluster randomized trials (CRTs) are becoming an increasingly important design. However, authors of CRTs do not always adhere to requirements to explicitly identify the design as cluster randomized in titles and abstracts, making retrieval from bibliographic databases difficult. Machine learning algorithms may improve their identification and retrieval. Therefore, we aimed to develop machine learning algorithms that accurately determine whether a bibliographic citation is a CRT report.MethodsWe trained, internally validated, and externally validated two convolutional neural networks and one support vector machine (SVM) algorithm to predict whether a citation is a CRT report or not. We exclusively used the information in an article citation, including the title, abstract, keywords, and subject headings. The algorithms' output was a probability from 0 to 1. We assessed algorithm performance using the area under the receiver operating characteristic (AUC) curves. Each algorithm's performance was evaluated individually and together as an ensemble. We randomly selected 5000 from 87,633 citations to train and internally validate our algorithms. Of the 5000 selected citations, 589 (12%) were confirmed CRT reports. We then externally validated our algorithms on an independent set of 1916 randomized trial citations, with 665 (35%) confirmed CRT reports.ResultsIn internal validation, the ensemble algorithm discriminated best for identifying CRT reports with an AUC of 98.6% (95% confidence interval: 97.8%, 99.4%), sensitivity of 97.7% (94.3%, 100%), and specificity of 85.0% (81.8%, 88.1%). In external validation, the ensemble algorithm had an AUC of 97.8% (97.0%, 98.5%), sensitivity of 97.6% (96.4%, 98.6%), and specificity of 78.2% (75.9%, 80.4%)). All three individual algorithms performed well, but less so than the ensemble.ConclusionsWe successfully developed high-performance algorithms that identified whether a citation was a CRT report with high sensitivity and moderately high specificity. We provide open-source software to facilitate the use of our algorithms in practice.

Project description:BackgroundTreatment non-adherence in randomised trials refers to situations where some participants do not receive their allocated treatment as intended. For cluster randomised trials, where the unit of randomisation is a group of participants, non-adherence may occur at the cluster or individual level. When non-adherence occurs, randomisation no longer guarantees that the relationship between treatment receipt and outcome is unconfounded, and the power to detect the treatment effects in intention-to-treat analysis may be reduced. Thus, recording adherence and estimating the causal treatment effect adequately are of interest for clinical trials.ObjectivesTo assess the extent of reporting of non-adherence issues in published cluster trials and to establish which methods are currently being used for addressing non-adherence, if any, and whether clustering is accounted for in these.MethodsWe systematically reviewed 132 cluster trials published in English in 2011 previously identified through a search in PubMed.ResultsOne-hundred and twenty three cluster trials were included in this systematic review. Non-adherence was reported in 56 cluster trials. Among these, 19 reported a treatment efficacy estimate: per protocol in 15 and as treated in 4. No study discussed the assumptions made by these methods, their plausibility or the sensitivity of the results to deviations from these assumptions.LimitationsThe year of publication of the cluster trials included in this review (2011) could be considered a limitation of this study; however, no new guidelines regarding the reporting and the handling of non-adherence for cluster trials have been published since. In addition, a single reviewer undertook the data extraction. To mitigate this, a second reviewer conducted a validation of the extraction process on 15 randomly selected reports. Agreement was satisfactory (93%).ConclusionDespite the recommendations of the Consolidated Standards of Reporting Trials statement extension to cluster randomised trials, treatment adherence is under-reported. Among the trials providing adherence information, there was substantial variation in how adherence was defined, handled and reported. Researchers should discuss the assumptions required for the results to be interpreted causally and whether these are scientifically plausible in their studies. Sensitivity analyses to study the robustness of the results to departures from these assumptions should be performed.

Project description:BackgroundPatients with COVID-19 may feel under pressure to participate in research during the pandemic. Safeguards to protect research participants include ethical guidelines [e.g. Declaration of Helsinki and good clinical practice (GCP)], legislation to protect participants' privacy, research ethics committees (RECs) and informed consent. The International Committee of Medical Journal Editors (ICMJE) advises researchers to document compliance with these safeguards. Adherence to publication guidelines has been suboptimal in other specialty fields. The aim of this rapid review was to determine whether COVID-19 human research publications report compliance with these ethical safeguards.MethodsA rapid systematic literature review was conducted in MEDLINE using the search term 'COVID-19'. The search was performed in April 2020 with no start date and repeated to include articles published in November 2020. Filters were 'Full free text available' and 'English Language'. Two reviewers assessed article title, abstracts and full texts. Non-COVID-19 articles and non-clinical studies were excluded. Independent reviewers conducted a second assessment of a random 20% of articles. The outcomes included reporting of compliance with the Declaration of Helsinki and GCP, REC approval, informed consent and participant privacy.ResultsThe searches yielded 1275 and 1942 articles of which 247 and 717 were deemed eligible, from the April search and November respectively. The majority of journals had editorial policies which purported to comply with ICMJE ethical standards. Reporting of compliance with ethical guidelines was low across all study types but was higher in the November search for case series and observational studies. Reporting of informed consent for case studies and observational studies was higher in the November search, but similar for case series. Overall, participant confidentiality was maintained but some case studies included a combination of details which would have enabled participant identification. Reporting of REC approval was higher in the November search for observational studies.ConclusionsWhile the majority of journal's editorial policies purported to support the ethical safeguards, many COVID-19 clinical research publications identified in this rapid review lacked documentation of these important safeguards for research participants. In order to promote public trust, ethical declarations should be included consistently.

Project description:BackgroundTo implement standardized machine-processable clinical sequencing reports in an electronic health record (EHR) system, the International Organization for Standardization Technical Specification (ISO/TS) 20428 international standard was proposed for a structured template. However, there are no standard implementation guidelines for data items from the proposed standard at the clinical site and no guidelines or references for implementing gene sequencing data results for clinical use. This is a significant challenge for implementation and application of these standards at individual sites.ObjectiveThis study examines the field utilization of genetic test reports by designing the Health Level 7 (HL7) Fast Healthcare Interoperability Resources (FHIR) for genomic data elements based on the ISO/TS 20428 standard published as the standard for genomic test reports. The goal of this pilot is to facilitate the reporting and viewing of genomic data for clinical applications. FHIR Genomics resources predominantly focus on transmitting or representing sequencing data, which is of less clinical value.MethodsIn this study, we describe the practical implementation of ISO/TS 20428 using HL7 FHIR Genomics implementation guidance to efficiently deliver the required genomic sequencing results to clinicians through an EHR system.ResultsWe successfully administered a structured genomic sequencing report in a tertiary hospital in Korea based on international standards. In total, 90 FHIR resources were used. Among 41 resources for the required fields, 26 were reused and 15 were extended. For the optional fields, 28 were reused and 21 were extended.ConclusionsTo share and apply genomic sequencing data in both clinical practice and translational research, it is essential to identify the applicability of the standard-based information system in a practical setting. This prototyping work shows that reporting data from clinical genomics sequencing can be effectively implemented into an EHR system using the existing ISO/TS 20428 standard and FHIR resources.

Dataset Information

Electronic search strategies to identify reports of cluster randomized trials in MEDLINE: low precision will improve with adherence to reporting standards.

Background

Methods

Results

Conclusions

Publications

Electronic search strategies to identify reports of cluster randomized trials in MEDLINE: low precision will improve with adherence to reporting standards.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets