Dataset Information

Newcastle-Ottawa Scale: comparing reviewers' to authors' assessments.

ABSTRACT: BACKGROUND: Lack of appropriate reporting of methodological details has previously been shown to distort risk of bias assessments in randomized controlled trials. The same might be true for observational studies. The goal of this study was to compare the Newcastle-Ottawa Scale (NOS) assessment for risk of bias between reviewers and authors of cohort studies included in a published systematic review on risk factors for severe outcomes in patients infected with influenza. METHODS: Cohort studies included in the systematic review and published between 2008-2011 were included. The corresponding or first authors completed a survey covering all NOS items. Results were compared with the NOS assessment applied by reviewers of the systematic review. Inter-rater reliability was calculated using kappa (K) statistics. RESULTS: Authors of 65/182 (36%) studies completed the survey. The overall NOS score was significantly higher (p < 0.001) in the reviewers' assessment (median = 6; interquartile range [IQR] 6-6) compared with those by authors (median = 5, IQR 4-6). Inter-rater reliability by item ranged from slight (K = 0.15, 95% confidence interval [CI]?=?-0.19, 0.48) to poor (K = -0.06, 95% CI = -0.22, 0.10). Reliability for the overall score was poor (K = -0.004, 95% CI = -0.11, 0.11). CONCLUSIONS: Differences in assessment and low agreement between reviewers and authors suggest the need to contact authors for information not published in studies when applying the NOS in systematic reviews.

SUBMITTER: Lo CK

PROVIDER: S-EPMC4021422 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Newcastle-Ottawa Scale: comparing reviewers' to authors' assessments.

Lo Carson Ka-Lok CK Mertz Dominik D Loeb Mark M

BMC medical research methodology 20140401

<h4>Background</h4>Lack of appropriate reporting of methodological details has previously been shown to distort risk of bias assessments in randomized controlled trials. The same might be true for observational studies. The goal of this study was to compare the Newcastle-Ottawa Scale (NOS) assessment for risk of bias between reviewers and authors of cohort studies included in a published systematic review on risk factors for severe outcomes in patients infected with influenza.<h4>Methods</h4>Coh ...[more]

PMID: 24690082

Similar Datasets

Project description:OBJECTIVE:Many journals permit authors to submit supplementary material for publication alongside the article. We explore the value, use and role of this material in biomedical journal articles from the perspectives of authors, peer reviewers and readers. DESIGN AND SETTING:We conducted online surveys (November-December 2016) of corresponding authors and peer reviewers at 17 BMJ Publishing Group journals in a range of specialities. PARTICIPANTS:Participants were asked to respond to one of three surveys: as authors, peer reviewers or readers. RESULTS:We received 2872/20340 (14%) responses: authors 819/6892 (12%), peer reviewers 1142/6682 (17%) and readers 911/6766 (14%). Most authors submitted (711/819, 87%) and 80% (724/911) of readers reported reading supplementary material with their last article, while 95% (1086/1142) of reviewers reported seeing these materials sometimes. Additional data tables were the most common supplementary material reported (authors: 74%; reviewers: 89%; readers: 67%). A majority in each group indicated additional tables were most useful to readers (61%-77%); 20%-36%?and 3%-4% indicated they were most useful to peer reviewers and journal editors, respectively. Checklists and reporting guidelines showed the opposite: higher proportions of each group regarded these as most useful to journal editors. All three groups favoured the publication of additional tables and figures on the journal's website (80%-83%), with <4% of each group responding that these do not need to be available. Approximately one-fifth (16%-23%) responded that raw study data should be available on the journal's website, while 24%-33% said that these materials should not be made available anywhere. CONCLUSIONS:Authors, peer reviewers and readers agree that supplementary materials are useful. Supplementary tables and figures were favoured over reporting checklists or raw data for reading but not for study replication. Journals should consider the roles, resource costs and strategic placement of supplementary materials to ensure optimal usage and minimise waste. TRIAL REGISTRATION NUMBER:NCT02961036.

Project description:Synthesizing outcomes of underreported primary studies can pose a serious threat to the validity of outcomes and conclusions of systematic reviews. To address this problem, the Cochrane Collaboration recommends reviewers to contact authors of eligible primary studies to obtain additional information on poorly reported items. In this protocol, we present a cross-sectional study and a survey to assess (1) how reviewers of new Cochrane intervention reviews report on procedures and outcomes of contacting of authors of primary studies to obtain additional data, (2) how authors reply, and (3) the consequences of these additional data on the outcomes and quality scores in the review. All research questions and methods were pilot tested on 2 months of Cochrane reviews and were subsequently fine-tuned.Eligibility criteria are (1) all new (not-updates) Cochrane intervention reviews published in 2016, (2) reviews that included one or more primary studies, and (3) eligible interventions refer to contacting of authors of the eligible primary studies included in the review to obtain additional research data (e.g., information on unreported or missing data, individual patient data, research methods, and bias issues). Searching for eligible reviews and data extraction will be conducted by two authors independently. The cross-sectional study will primarily focus on how contacting of authors is conducted and reported, how contacted authors reply, and how reviewers report on obtained additional data and their consequences for the review.The same eligible reviews for the cross-sectional study will also be eligible for the survey. Surveys will be sent to the contact addresses of these reviews according to a pre-defined protocol. We will use Google Forms as our survey platform. Surveyees are asked to answer eight questions. The survey will primarily focus on the consequences of contacting authors of eligible primary studies for the risk of bias and Grading of Recommendations, Assessment, Development and Evaluation scores and the primary and secondary outcomes of the review.The findings of this study could help improve methods of contacting authors and reporting of these procedures and their outcomes. Patients, clinicians, researchers, guideline developers, research sponsors, and the general public will all be beneficiaries.

Dataset Information

Newcastle-Ottawa Scale: comparing reviewers' to authors' assessments.

Publications

Newcastle-Ottawa Scale: comparing reviewers' to authors' assessments.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets