Searching for unpublished data for Cochrane reviews: cross sectional study.
Ontology highlight
ABSTRACT: Objective
To describe the experiences of authors of Cochrane reviews in searching for, getting access to, and using unpublished data.Design
Cross sectional study.Setting
Cochrane reviews.Participants
2184 corresponding authors of Cochrane reviews as of May 2012.Main outcome measure
Frequencies of responses to open ended and closed questions in an online survey.Results
Of 5915 authors contacted by email, 2184 replied (36.9% response rate). Of those, 1656 (75.8%) had searched for unpublished data. In 913 cases (55.1% of 1656), new data were obtained and we received details about these data for 794 data sources. The most common data source was "trialists/investigators," accounting for 73.9% (n=587) of the 794 data sources. Most of the data were used in the review (82.0%, 651/794) and in 53.4% (424/794) of cases data were provided in less than a month. Summary data were most common, provided by 50.8% (403/794) of the data sources, whereas 20.5% (163/794) provided individual patient data. In only 6.3% (50/794) of cases were data reported to have been obtained from the manufacturers, and this group waited longer and had to make more contacts to get the data. The data from manufacturers were less likely to be for individual patients and less likely to be used in the review. Data from regulatory agencies accounted for 3.0% (24/794) of the obtained data.Conclusions
Most authors of Cochrane reviews who searched for unpublished data received useful information, primarily from trialists. Our response rate was low and the authors who did not respond were probably less likely to have searched for unpublished data. Manufacturers and regulatory agencies were uncommon sources of unpublished data.
SUBMITTER: Schroll JB
PROVIDER: S-EPMC3633324 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ObjectivesTo investigate the degree to which Cochrane reviews of drug interventions published in 2010 reported conflicts of interest from included trials and, among reviews that reported this information, where it was located in the review documents.DesignCross sectional study.Data sourcesCochrane Database of Systematic Reviews.Selection criteriaSystematic reviews of drug interventions published in 2010 in the Cochrane Database of Systematic Reviews, with review content classified as up to date in 2008 or later and with results from one or more randomised controlled trials.ResultsOf 151 included Cochrane reviews, 46 (30%, 95% confidence interval 24% to 38%) reported information on the funding sources of included trials, including 30 (20%, 14% to 27%) that reported information on trial funding for all included trials and 16 (11%, 7% to 17%) that reported for some, but not all, trials. Only 16 of the 151 Cochrane reviews (11%, 7% to 17%) provided any information on trial author-industry financial ties or trial author-industry employment. Information on trial funding and trial author-industry ties was reported in one to seven locations within each review, with no consistent reporting location observed.ConclusionsMost Cochrane reviews of drug trials published in 2010 did not provide information on trial funding sources or trial author-industry financial ties or employment. When this information was reported, location of reporting was inconsistent across reviews.
| S-EPMC3423635 | biostudies-literature
Project description:BackgroundDiscrepancies in outcome reporting (DOR) between protocol and published studies include inclusions of new outcomes, omission of prespecified outcomes, upgrade and downgrade of secondary and primary outcomes, and changes in definitions of prespecified outcomes. DOR can result in outcome reporting bias (ORB) when changes in outcomes occur after knowledge of results. This has potential to overestimate treatment effects and underestimate harms. This can also occur at the level of systematic reviews when changes in outcomes occur after knowledge of results of included studies. The prevalence of DOR and ORB in systematic reviews is unknown in systematic reviews published post-2007.ObjectiveTo estimate the prevalence of DOR and risk of ORB in all Cochrane reviews between the years 2007 and 2014.MethodsA stratified random sampling approach was applied to collect a representative sample of Cochrane systematic reviews from each Cochrane review group. DOR was assessed by matching outcomes in each systematic review with their respective protocol. When DOR occurred, reviews were further assessed if there was a risk of ORB (unclear, low or high risk). We classified DOR as a high risk for ORB if the discrepancy occurred after knowledge of results in the systematic review.Results150 of 350 (43%) review and protocol pairings contained DOR. When reviews were further scrutinised, 23% (35 of 150) of reviews with DOR contained a high risk of ORB, with changes being made after knowledge of results from individual trials.ConclusionsIn our study, we identified just under a half of Cochrane reviews with at least one DOR. Of these, a fifth were at high risk of ORB. The presence of DOR and ORB in Cochrane reviews is of great concern; however, a solution is relatively simple. Authors are encouraged to be transparent where outcomes change and to describe the legitimacy of changing outcomes in order to prevent suspicion of bias.
| S-EPMC7076244 | biostudies-literature
Project description:BACKGROUND:Despite the growing reputation and subject coverage of the Cochrane Database of Systematic Reviews, many systematic reviews continue to be published solely in paper-based health care journals. This study was designed to determine why authors choose to publish their systematic reviews outside of the Cochrane Collaboration and if they might be interested in converting their reviews to Cochrane format for publication in the Cochrane Database of Systematic Reviews. METHODS:Cross-sectional survey of Australian primary authors of systematic reviews not published on the Cochrane Database of Systematic Reviews identified from the Database of Abstracts of Reviews of Effectiveness. RESULTS:We identified 88 systematic reviews from the Database of Abstracts of Reviews of Effectiveness with an Australian as the primary author. We surveyed 52 authors for whom valid contact information was available. The response rate was 88 per cent (46/52). Ten authors replied without completing the survey, leaving 36 valid surveys for analysis. The most frequently cited reasons for not undertaking a Cochrane review were: lack of time (78%), the need to undergo specific Cochrane training (46%), unwillingness to update reviews (36%), difficulties with the Cochrane process (26%) and the review topic already registered with the Cochrane Collaboration (21%). (Percentages based on completed responses to individual questions.) Nearly half the respondents would consider converting their review to Cochrane format. Dedicated time emerged as the most important factor in facilitating the potential conversion process. Other factors included navigating the Cochrane system, assistance with updating and financial support. Eighty-six per cent were willing to have their review converted to Cochrane format by another author. CONCLUSION:Time required to complete a Cochrane review and the need for specific training are the primary reasons why some authors publish systematic reviews outside of the Cochrane Collaboration. Encouragingly, almost half of the authors would consider converting their review to Cochrane format. Based on the current number of reviews in the Database of Abstracts of Reviews of Effectiveness, this could result in more than 700 additional Cochrane reviews. Ways of supporting these authors and how to provide dedicated time to convert systematic reviews needs further consideration.
| S-EPMC149345 | biostudies-literature
Project description:INTRODUCTION:Many diagnostic accuracy studies are never reported in full in a peer-reviewed journal. Searching for unpublished studies may avoid bias due to selective publication, enrich the power of systematic reviews, and thereby help to reduce research waste. We assessed searching practices among recent systematic reviews of diagnostic accuracy. METHODS:We extracted data from 100 non-Cochrane systematic reviews of diagnostic accuracy indexed in MEDLINE and published between October 2017 and January 2018 and from all 100 Cochrane systematic reviews of diagnostic accuracy published by December 2018, irrespective of whether meta-analysis had been performed. RESULTS:Non-Cochrane and Cochrane reviews searched a median of 4 (IQR 3-5) and 6 (IQR 5-9) databases, respectively; most often MEDLINE/PubMed (n = 100 and n = 100) and EMBASE (n = 81 and n = 100). Additional efforts to identify studies beyond searching bibliographic databases were performed in 76 and 98 reviews, most often through screening reference lists (n = 71 and n = 96), review/guideline articles (n = 18 and n = 52), or citing articles (n = 3 and n = 42). Specific sources of unpublished studies were searched in 22 and 68 reviews, for example, conference proceedings (n = 4 and n = 18), databases only containing conference abstracts (n = 2 and n = 33), or trial registries (n = 12 and n = 39). At least one unpublished study was included in 17 and 23 reviews. Overall, 39 of 2082 studies (1.9%) included in non-Cochrane reviews were unpublished, and 64 of 2780 studies (2.3%) in Cochrane reviews, most often conference abstracts (97/103). CONCLUSION:Searching practices vary considerably across systematic reviews of diagnostic accuracy. Unpublished studies are a minimal fraction of the evidence included in recent reviews.
| S-EPMC7317757 | biostudies-literature
Project description:BackgroundOne of the most important formats to disseminate the evidence in health to different populations are Cochrane Plain Language Summaries (PLSs). PLSs should be written in a simplified language, easily understandable and providing clear message for the consumer. The aim of this study was to examine the extent to which PLSs are customized for lay persons, specifically by providing conclusive, comprehensible, and readable messages.MethodsThe study analyzed Cochrane PLSs of interventional studies (N = 4360) in the English language published from 1995 to 2019. We categorized the conclusiveness into one of the following categories: "positive", "positive inconclusive", "no evidence", "no opinion", "negative", "negative inconclusive", "unclear", "equal", "equal inconclusive". Language characteristics were analyzed using Linguistic Inquiry and Word Count (LIWC) software. The level of readability was measured by SMOG (Simple Measure of Gobbledygook) index, indicating the number of years of education required to read the text. For each PLS, we also collected the following data: Cochrane Review Network, year of publication and number of authors.ResultsMost of the PLSs (80%) did not have a conclusive message. In 53% PLSs there was no concluding opinion about the studied intervention or the conclusion was unclear. The most frequent conclusiveness category was "no opinion" (30%), and its frequency increased over time. The conclusiveness categories were similarly dispersed across Cochrane Networks. PLSs were written in an objective style, with high levels of analytical tone and clout above neutral, but a lower relation to authenticity and tone. The median number of years of non-specific education needed to read the PLSs was 14.9 (IQR 13.8 to 16.1), indicating that the person needs almost 15 years of general education to read the content with ease.ConclusionMost of the Cochrane PLSs provided no concluding opinion or unclear conclusion regarding the effects of analyzed intervention. Analysis of readability indicated that they may be difficult to read for the lay population without medical education. Our results indicate that PLSs may not be so plain, and that the writing of Cochrane PLSs requires more effort. Tools used in this study could improve PLSs and make them better suited for lay audiences.
| S-EPMC9464378 | biostudies-literature
Project description:BackgroundAlthough systematic reviews of health care interventions are an invaluable tool for health care providers and researchers, many potential authors never publish reviews. This study attempts to determine why some people with interest in performing systematic reviews do not subsequently publish a review; and what steps could possibly increase review completion.MethodsCross-sectional survey by email and facsimile of the 179 participants in Australasian Cochrane Centre training events between 1998 and 2000.ResultsNinety-two participants responded to the survey (51 percent). Response rate of deliverable surveys was 82 percent (92/112). The remainder of the participants had invalid or no contact information on file. More than 75 percent of respondents felt that the current workshops met their needs for training. The most critical barriers to completion of a Cochrane review were: lack of time (80 percent), lack of financial support (36 percent), methodological problems (23 percent) and problems with group dynamics (10 percent).ConclusionsStrategies to protect reviewer time and increase the efficiency of the review process may increase the numbers of trained reviewers completing a systematic review.
| S-EPMC116427 | biostudies-literature
Project description:BackgroundIt is possible that cross-over studies included in current systematic reviews are being inadequately assessed, because the current risk of bias tools do not consider possible biases specific to cross-over design. We performed this study to evaluate whether this was being done in cross-over studies included in Cochrane Systematic Reviews (CSRs).MethodsWe searched the Cochrane Library (up to 2013 issue 5) for CSRs that included at least one cross-over trial. Two authors independently undertook the study selection and data extraction. A random sample of the CSRs was selected and we evaluated whether the cross-over trials in these CSRs were assessed according to criteria suggested by the Cochrane handbook. In addition we reassessed the risk of bias of these cross-over trials by a checklist developed form the Cochrane handbook.ResultsWe identified 688 CSRs that included one or more cross-over studies. We chose a random sample of 60 CSRs and these included 139 cross-over studies. None of these CSRs undertook a risk of bias assessment specific for cross-over studies. In fact items specific for cross-over studies were seldom considered anywhere in quality assessment of these CSRs. When we reassessed the risk of bias, including the 3 items specific to cross-over trials, of these 139 studies, a low risk of bias was judged for appropriate cross-over design in 110(79%), carry-over effects in 48(34%) and for reporting data in all stages of the trial in 114(82%).Assessment of biases in cross-over trials could affect the GRADE assessment of a review's findings.ConclusionThe current Cochrane risk of bias tool is not adequate to assess cross-over studies. Items specific to cross-over trials leading to potential risk of bias are generally neglected in CSRs. A proposed check list for the evaluation of cross-over trials is provided.
| S-EPMC4395015 | biostudies-literature
Project description:BackgroundPrevious research on data extraction methods in systematic reviews has focused on single aspects of the process. We aimed to provide a deeper insight into these methods by analysing a current sample of reviews.MethodsWe included systematic reviews of health interventions in humans published in English. We analysed 75 Cochrane reviews from May and June 2020 and a random sample of non-Cochrane reviews published in the same period and retrieved from Medline. We linked reviews with protocols and study registrations. We collected information on preparing, piloting, and performing data extraction and on use of software to assist review conduct (automation tools). Data were extracted by one author, with 20% extracted in duplicate. Data were analysed descriptively.ResultsOf the 152 included reviews, 77 reported use of a standardized extraction form (51%); 42 provided information on the type of form used (28%); 24 on piloting (16%); 58 on what data was collected (38%); 133 on the extraction method (88%); 107 on resolving disagreements (70%); 103 on methods to obtain additional data or information (68%); 52 on procedures to avoid data errors (34%); and 47 on methods to deal with multiple study reports (31%). Items were more frequently reported in Cochrane than non-Cochrane reviews. The data extraction form used was published in 10 reviews (7%). Use of software was rarely reported except for statistical analysis software and use of RevMan and GRADEpro GDT in Cochrane reviews. Covidence was the most frequent automation tool used: 18 reviews used it for study selection (12%) and 9 for data extraction (6%).ConclusionsReporting of data extraction methods in systematic reviews is limited, especially in non-Cochrane reviews. This includes core items of data extraction such as methods used to manage disagreements. Few reviews currently use software to assist data extraction and review conduct. Our results can serve as a baseline to assess the uptake of such tools in future analyses.
| S-EPMC8571672 | biostudies-literature
Project description:In 1996, shortly after the founding of The Cochrane Collaboration, leading figures in test evaluation research established a Methods Group to focus on the relatively new and rapidly evolving methods for the systematic review of studies of diagnostic tests. Seven years later, the Collaboration decided it was time to develop a publication format and methodology for Diagnostic Test Accuracy (DTA) reviews, as well as the software needed to implement these reviews in The Cochrane Library. A meeting hosted by the German Cochrane Centre in 2004 brought together key methodologists in the area, many of whom became closely involved in the subsequent development of the methodological framework for DTA reviews. DTA reviews first appeared in The Cochrane Library in 2008 and are now an integral part of the work of the Collaboration.
| S-EPMC3851548 | biostudies-literature
Project description:BackgroundSystematic reviews with meta-analyses often contain many statistical tests. This multiplicity may increase the risk of type I error. Few attempts have been made to address the problem of statistical multiplicity in systematic reviews. Before the implications are properly considered, the size of the issue deserves clarification. Because of the emphasis on bias evaluation and because of the editorial processes involved, Cochrane reviews may contain more multiplicity than their non-Cochrane counterparts. This study measured the quantity of statistical multiplicity present in a population of systematic reviews and aimed to assess whether this quantity is different in Cochrane and non-Cochrane reviews.Methods/principal findingsWe selected all the systematic reviews published by the Cochrane Anaesthesia Review Group containing a meta-analysis and matched them with comparable non-Cochrane reviews. We counted the number of statistical tests done in each systematic review. The median number of tests overall was 10 (interquartile range (IQR) 6 to 18). The median was 12 in Cochrane and 8 in non-Cochrane reviews (difference in medians 4 (95% confidence interval (CI) 2.0-19.0). The proportion that used an assessment of risk of bias as a reason for doing extra analyses was 42% in Cochrane and 28% in non-Cochrane reviews (difference in proportions 14% (95% CI -8 to 36). The issue of multiplicity was addressed in 6% of all the reviews.Conclusion/significanceStatistical multiplicity in systematic reviews requires attention. We found more multiplicity in Cochrane reviews than in non-Cochrane reviews. Many of the reasons for the increase in multiplicity may well represent improved methodological approaches and greater transparency, but multiplicity may also cause an increased risk of spurious conclusions. Few systematic reviews, whether Cochrane or non-Cochrane, address the issue of multiplicity.
| S-EPMC3229598 | biostudies-literature