Risk of Bias in Reports of In Vivo Research: A Focus for Improvement.
Ontology highlight
ABSTRACT:
SUBMITTER: Macleod MR
PROVIDER: S-EPMC4603955 | biostudies-literature | 2015 Oct
REPOSITORIES: biostudies-literature
Publications
Risk of Bias in Reports of In Vivo Research: A Focus for Improvement.
Macleod Malcolm R MR Lawson McLean Aaron A Kyriakopoulou Aikaterini A Serghiou Stylianos S de Wilde Arno A Sherratt Nicki N Hirst Theo T Hemblade Rachel R Bahor Zsanett Z Nunes-Fonseca Cristina C Potluru Aparna A Thomson Andrew A Baginskaite Julija J Egan Kieren K Vesterinen Hanna H Currie Gillian L GL Churilov Leonid L Howells David W DW Sena Emily S ES
PLoS biology 20151013 10
The reliability of experimental findings depends on the rigour of experimental design. Here we show limited reporting of measures to reduce the risk of bias in a random sample of life sciences publications, significantly lower reporting of randomisation in work published in journals of high impact, and very limited reporting of measures to reduce the risk of bias in publications from leading United Kingdom institutions. Ascertainment of differences between institutions might serve both as a meas ...[more]
Similar Datasets
Project description:BackgroundThe Cochrane risk of bias tool is a prominent instrument used to evaluate potential biases in clinical trials. In three updates of our Cochrane review on neuraminidase inhibitors, we assessed risk of bias on the same trials using different levels of detail: the trials in journal publications, in core reports, and in full clinical study reports. Here we analyse whether progressively greater amounts of information and detail in full clinical study reports (including trial protocols, statistical analysis plans, certificates of analyses, individual participant data listings and randomisation lists) affected our risk of bias assessments.Methods and findingsWe used the Cochrane risk of bias tool to assess and compare risk of bias in 14 oseltamivir trials (reported in 10 clinical study reports) obtained from the European Medicines Agency (EMA) and the manufacturer, Roche. With more detailed information, reported in clinical study reports, no previous assessment of 'high' risk of bias was reclassified as 'low' or 'unclear' in the main analysis, and over half (55%, 34/62) of the previous assessments of 'low' risk of bias were reclassified as 'high'. Most assessments of 'unclear' risk of bias (67%, or 28/42) were reclassified as 'high' risk of bias when our judgements were based on full clinical study reports. The limits of our study were our relative inexperience in dealing with large information sets, sometimes subjective bias judgements and focus on industry trials. Comparison with journal publications was not possible because of the low number of trials published.ConclusionsWe found that as information increased in the document, this increased our assessment of bias. This may mean that risk of bias has been insufficiently assessed in Cochrane reviews based on journal publications.
| S-EPMC4179405 | biostudies-literature
Project description:PurposeThis retrospective cross-sectional study aimed to identify opportunities for improvement in food and nutrition research by examining risk of bias (ROB) domains.MethodsRatings were extracted from critical appraisal records for 5675 studies used in systematic reviews conducted by three organizations. Variables were as follows: ROB domains defined by the Cochrane Collaboration (Selection, Performance, Detection, Attrition, and Reporting), publication year, research type (intervention or observation) and specific design, funder, and overall quality rating (positive, neutral, or negative). Appraisal instrument questions were mapped to ROB domains. The kappa statistic was used to determine consistency when multiple ROB ratings were available. Binary logistic regression and multinomial logistic regression were used to predict overall quality and ROB domains.FindingsStudies represented a wide variety of research topics (clinical nutrition, food safety, dietary patterns, and dietary supplements) among 15 different research designs with a balance of intervention (49%) and observation (51%) types, published between 1930 and 2015 (64% between 2000-2009). Duplicate ratings (10%) were consistent (κ = 0.86-0.94). Selection and Performance domain criteria were least likely to be met (57.9% to 60.1%). Selection, Detection, and Performance ROB ratings predicted neutral or negative quality compared to positive quality (p<0.001). Funder, year, and research design were significant predictors of ROB. Some sources of funding predicted increased ROB (p<0.001) for Selection (interventional: industry only and none/not reported; observational: other only and none/not reported) and Reporting (observational: university only and other only). Reduced ROB was predicted by combined and other-only funding for intervention research (p<0.005). Performance ROB domain ratings started significantly improving in 2000; others improved after 1990 (p<0.001). Research designs with higher ROB were nonrandomized intervention and time series designs compared to RCT and prospective cohort designs respectively (p<0.001).ConclusionsOpportunities for improvement in food and nutrition research are in the Selection, Performance, and Detection ROB domains.
| S-EPMC6033375 | biostudies-literature
Project description:BackgroundRandomized Controlled Trials (RCTs) represent the most valuable study design to evaluate the effectiveness of therapeutic interventions. However, flaws in design, conduct, analysis, and reporting of RCTs can cause the effect of an intervention to be under- or overestimated. These biased RCTs may be included in literature reviews. To make the assessment of Risk of Bias (RoB) consistent and transparent, Cochrane published a RoB tool, with which RoB is assessed per item as "low", "unclear" or "high". Our objective was to provide an overview of RoB assessments of RCTs in otorhinolaryngology over time, and to identify items where improvement is still warranted.MethodsWe retrieved Cochrane reviews in the otorhinolaryngologic research field published in 2012 and 2013. We used all judgments per item as assessed by the review authors of the included RCTs. We evaluated the association between "low RoB" vs. "unclear and high RoB" and the year of publication (time strata: '<1990', '1990-1995', '1996-2000', '2001-2005', '2006-2012') per item using binary logistic regression.ResultsWe extracted the RoB assessments from 42 Cochrane reviews that had included 402 RCTs (median number of RCTs per review: 7, range 1-40). In total 2,356 items were assessed (mean number of assessed items per RCT: 5.9, standard deviation 1.8). On binary logistic regression, RCTs published in 2006-2012, compared with those published before 1990, were more likely to have a low RoB for two items: random sequence generation (odds ratio 6.09 [95% confidence interval: 3.11-11.95]) and allocation concealment (3.59 [1.87-6.90]). On all other items, there was no significant increase in the proportion of low RoB when comparing RCTs published in 2006-2012 with RCTs published before 1990.ConclusionAlthough there were some positive developments in the RoB assessments in otorhinolaryngology, a further decrease in RoB is still warranted on several items. Currently, biased RCTs are included in Cochrane reviews and effects of therapeutic interventions can be under- or overestimated, with implications for clinical patient care.
| S-EPMC5395869 | biostudies-literature
Project description:OBJECTIVE:To provide an overview of qualitative methods, particularly for reviewers and authors who may be less familiar with qualitative research. METHODS:A question and answer format is used to address considerations for writing and evaluating qualitative research. RESULTS AND CONCLUSIONS:When producing qualitative research, individuals are encouraged to address the qualitative research considerations raised and to explicitly identify the systematic strategies used to ensure rigor in study design and methods, analysis, and presentation of findings. Increasing capacity for review and publication of qualitative research within pediatric psychology will advance the field's ability to gain a better understanding of the specific needs of pediatric populations, tailor interventions more effectively, and promote optimal health.
| S-EPMC4888120 | biostudies-literature
Project description:BackgroundTo assess the completeness of reporting, research transparency practices, and risk of selection and immortal bias in observational studies using routinely collected data for comparative effectiveness research.MethodWe performed a meta-research study by searching PubMed for comparative effectiveness observational studies evaluating therapeutic interventions using routinely collected data published in high impact factor journals from 01/06/2018 to 30/06/2020. We assessed the reporting of the study design (i.e., eligibility, treatment assignment, and the start of follow-up). The risk of selection bias and immortal time bias was determined by assessing if the time of eligibility, the treatment assignment, and the start of follow-up were synchronized to mimic the randomization following the target trial emulation framework.ResultSeventy-seven articles were identified. Most studies evaluated pharmacological treatments (69%) with a median sample size of 24,000 individuals. In total, 20% of articles inadequately reported essential information of the study design. One-third of the articles (n = 25, 33%) raised some concerns because of unclear reporting (n = 6, 8%) or were at high risk of selection bias and/or immortal time bias (n = 19, 25%). Only five articles (25%) described a solution to mitigate these biases. Six articles (31%) discussed these biases in the limitations section.ConclusionReporting of essential information of study design in observational studies remained suboptimal. Selection bias and immortal time bias were common methodological issues that researchers and physicians should be aware of when interpreting the results of observational studies using routinely collected data.
| S-EPMC8608432 | biostudies-literature
Project description:AimsCompleteness of procedure reports is an important quality indicator in endoscopy. A dictation template was developed to ensure key elements were included in colonoscopy and esophagogastroduodenoscopy (EGD) reports. Endoscopy reports were reviewed prior to and following implementation of the dictation templates to determine whether report completeness improved.MethodsKey elements in an endoscopic report were identified from published guidelines and posted at dictation stations. Colonoscopy and EGD reports were reviewed for the nine physicians performing endoscopy at St. Paul's Hospital prior to and following implementation of dictation templates. Dictation completeness was defined as inclusion of all key elements. Dictation completeness and inclusion of individual key elements at the two time points were compared using the t-test and Chi-square test.ResultsReports for 4648 procedures undertaken by nine endoscopists were reviewed for completeness at each time point (2008 and 2014). Colonoscopy report completeness increased from 65.8% to 83.2% (P < 0.001). Items that improved included documentation of consent, endoscope used, complications, withdrawal time and rectal retroflexion. EGD report completeness increased from 72.7% to 77.3% (P < 0.001) with improvement in documentation of consent and complications. Items consistently underreported for colonoscopy and EGD at both time points included: patient age, comorbidities, current medications and patient comfort.ConclusionThere was an association between the use of a posted dictation template at dictation stations and the improved completeness of endoscopic reports.
| S-EPMC7898381 | biostudies-literature
Project description:Few methods estimate the prevalence of child maltreatment in the general population due to concerns about socially desirable responding and mandated reporting laws. Innovative methods, such as interactive voice response (IVR), may obtain better estimates that address these concerns. This study examined the utility of interactive voice response (IVR) for child maltreatment behaviors by assessing differences between respondents who completed and did not complete a survey using IVR technology. A mixed-mode telephone survey was conducted in English and Spanish in 50 cities in California during 2009. Caregivers (n=3,023) self-reported abusive and neglectful parenting behaviors for a focal child under the age of 13 using computer-assisted telephone interviewing and IVR. We used hierarchical generalized linear models to compare survey completion by caregivers nested within cities for the full sample and age-specific ranges. For demographic characteristics, caregivers born in the United States were more likely to complete the survey when controlling for covariates. Parenting stress, provision of physical needs, and provision of supervisory needs were not associated with survey completion in the full multivariate model. For caregivers of children 0-4 years (n=838), those reporting they could often or always hear their child from another room had a higher likelihood of survey completion. The findings suggest IVR could prove to be useful for future surveys that aim to estimate abusive and/or neglectful parenting behaviors given the limited bias observed for demographic characteristics and problematic parenting behaviors. Further research should expand upon its utility to advance estimation rates.
| S-EPMC4225177 | biostudies-literature
Project description:BackgroundThe Patient-Centered Outcomes Research Institute (PCORI) is obligated to peer review and to post publicly "Final Research Reports" of all funded projects. PCORI peer review emphasizes adherence to PCORI's Methodology Standards and principles of ethical scientific communication. During the peer review process, reviewers and editors seek to ensure that results are presented objectively and interpreted appropriately, e.g., free of spin.MethodsTwo independent raters assessed PCORI peer review feedback sent to authors. We calculated the proportion of reports in which spin was identified during peer review, and the types of spin identified. We included reports submitted by April 2018 with at least one associated journal article. The same raters then assessed whether authors addressed reviewers' comments about spin. The raters also assessed whether spin identified during PCORI peer review was present in related journal articles.ResultsWe included 64 PCORI-funded projects. Peer reviewers or editors identified spin in 55/64 (86%) submitted research reports. Types of spin included reporting bias (46/55; 84%), inappropriate interpretation (40/55; 73%), inappropriate extrapolation of results (15/55; 27%), and inappropriate attribution of causality (5/55; 9%). Authors addressed comments about spin related to 47/55 (85%) of the reports. Of 110 associated journal articles, PCORI comments about spin were potentially applicable to 44/110 (40%) articles, of which 27/44 (61%) contained the same spin that was identified in the PCORI research report. The proportion of articles with spin was similar for articles accepted before and after PCORI peer review (63% vs 58%).DiscussionJust as spin is common in journal articles and press releases, we found that most reports submitted to PCORI included spin. While most spin was mitigated during the funder's peer review process, we found no evidence that review of PCORI reports influenced spin in journal articles. Funders could explore interventions aimed at reducing spin in published articles of studies they support.
| S-EPMC8638354 | biostudies-literature
Project description:The identification of modifiable risk factors for the development of rheumatic conditions and their sequelae is crucial for reducing the substantial worldwide burden of these diseases. However, the validity of such research can be threatened by sources of bias, including confounding, measurement and selection biases. In this Review, we discuss potentially major issues of selection bias--a type of bias frequently overshadowed by other bias and feasibility issues, despite being equally or more problematic--in key areas of rheumatic disease research. We present index event bias (a type of selection bias) as one of the potentially unifying reasons behind some unexpected findings, such as the 'risk factor paradox'--a phenomenon exemplified by the discrepant effects of certain risk factors on the development versus the progression of osteoarthritis (OA) or rheumatoid arthritis (RA). We also discuss potential selection biases owing to differential loss to follow-up in RA and OA research, as well as those due to the depletion of susceptibles (prevalent user bias) and immortal time bias. The lesson remains that selection bias can be ubiquitous and, therefore, has the potential to lead the field astray. Thus, we conclude with suggestions to help investigators avoid such issues and limit the impact on future rheumatology research.
| S-EPMC4260806 | biostudies-literature
Project description:We found tremendous inequality across gene and protein annotation resources. We observed that this bias leads biomedical researchers to focus on richly annotated genes instead of those with the strongest molecular data. We advocate that researchers reduce these biases by pursuing data-driven hypotheses.
| S-EPMC5778030 | biostudies-literature