Transparent reporting of hypotheses and analyses in behavioral medicine research: An audit of publications in 2018 and 2008.
Ontology highlight
ABSTRACT: Objective
We aimed to document the use of transparent reporting of hypotheses and analyses in behavioral medicine journals in 2018 and 2008.Design
We examined a randomly selected portion of articles published in 2018 and 2008 by behavioral medicine journals with the highest impact factor, excluding manuscripts that were reviews or purely descriptive.Main outcome measures
We coded whether articles explicitly stated if the hypotheses/outcomes/analyses were primary or secondary; if study was registered/pre-registered; if "exploratory" or a related term was used to describe analyses/aims; and if power analyses were reported.Results
We coded 162 manuscripts published in 2018 (87% observational and 12% experimental). Sixteen percent were explicit in describing hypotheses/outcomes/analyses as primary or secondary, 51% appeared to report secondary hypotheses/outcomes/analyses but did not use term "secondary," and 33% were unclear. Registration occurred in 14% of studies, but 91% did not report which analyses were registered. "Exploratory" or related term was used in 31% of studies. Power analyses were reported in 8% of studies. Compared to 2008 (n=120), studies published in 2018 were more likely to be registered and less likely to have explicitly stated if outcomes were primary or secondary.Conclusions
Behavioral medicine stakeholders should consider strategies to increase clarity of reporting, and particularly details that will inform readers if analyses were pre-planned or post-hoc.
SUBMITTER: McVay MA
PROVIDER: S-EPMC8145985 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundCancer rehabilitation research has accelerated over the last decade. However, closer examination of the published literature reveals that the majority of this work has focused on psychological interventions and cognitive and behavioral therapies. Recent initiatives have aggregated expert consensus around research priorities, highlighting a dearth in research regarding measurement of and interventions for physical function. Increasingly loud calls for the need to address the myriad of physical functional impairments that develop in people living with and beyond cancer have been published in the literature. A detailed survey of the landscape of published research has not been reported to our knowledge.PurposeThis scoping review systematically identified literature published between 2008 and 2018 related to the screening, assessment, and interventions associated with physical function in people living with and beyond cancer.Data sourcesPubMed and CINAHL were searched up to September 2018.Study selectionStudy selection included articles of all levels of evidence on any disease stage and population. A total of 11,483 articles were screened for eligibility, 2507 full-text articles were reviewed, and 1055 articles were selected for final inclusion and extraction.Data extractionSeven reviewers recorded type of cancer, disease stage, age of participants, phase of treatment, time since diagnosis, application to physical function, study design, impairments related to physical function, and measurement instruments used.Data synthesisApproximately one-third of the articles included patients with various cancer diagnoses (30.3%), whereas the rest focused on a single cancer, most commonly breast (24.8%). Most articles (77%) measured physical function following the completion of active cancer treatment with 64% representing the assessment domain. The most commonly used measures of physical function were the Medical Outcomes Study 36-Item Health Survey Questionnaire (29%) and the European Organization for Research and Treatment of cancer Quality of Life Questionnaire-Cancer 30 (21.5%).LimitationsStudies not written in English, study protocols, conference abstracts, and unpublished data were excluded.ConclusionsThis review elucidated significant inconsistencies in the literature regarding language used to define physical function, measurement tools used to characterize function, and the use of those tools across the cancer treatment and survivorship trajectory. The findings suggested that physical function in cancer research is predominantly measured using general health-related quality-of-life tools rather than more precise functional assessment tools. Interdisciplinary and clinician-researcher collaborative efforts should be directed toward a unified definition and assessment of physical function.
| S-EPMC8204886 | biostudies-literature
Project description:BACKGROUND:In the data pipeline from the data collection process to the planned statistical analyses, initial data analysis (IDA) typically takes place between the end of the data collection and do not touch the research questions. A systematic process for IDA and clear reporting of the findings would help to understand the potential shortcomings of a dataset, such as missing values, or subgroups with small sample sizes, or shortcomings in the collection process, and to evaluate the impact of these shortcomings on the research results. A clear reporting of findings is also relevant when making datasets available to other researchers. Initial data analyses can provide valuable insights into the suitability of a data set for a future research study. Our aim was to describe the practice of reporting of initial data analyses in observational studies in five highly ranked medical journals with focus on data cleaning, screening, and reporting of findings which led to a potential change in the analysis plan. METHODS:This review was carried out using systematic search strategies with eligibility criteria for articles to be reviewed. A total of 25 papers about observational studies were selected from five medical journals published in 2018. Each paper was reviewed by two reviewers and IDA statements were further discussed by all authors. The consensus was reported. RESULTS:IDA statements were reported in the methods, results, discussion, and supplement of papers. Ten out of 25 papers (40%) included a statement about data cleaning. Data screening statements were included in all articles, and 18 (72%) indicated the methods used to describe them. Item missingness was reported in 11 papers (44%), unit missingness in 15 papers (60%). Eleven papers (44%) mentioned some changes in the analysis plan. Reported changes referred to missing data treatment, unexpected values, population heterogeneity and aspects related to variable distributions or data properties. CONCLUSION:Reporting of initial data analyses were sparse, and statements on IDA were located throughout the research articles. There is a lack of systematic reporting of IDA. We conclude the article with recommendations on how to overcome shortcomings in the practice of IDA reporting in observational studies.
| S-EPMC7071755 | biostudies-literature
Project description:ObjectivesTransparent reporting of clinical trials is essential to assess the risk of bias and translate research findings into clinical practice. While existing studies have shown that deficiencies are common, detailed empirical and field-specific data are scarce. Therefore, this study aimed to examine current clinical trial reporting and transparent research practices in sports medicine and orthopaedics.SettingExploratory meta-research study on reporting quality and transparent research practices in orthopaedics and sports medicine clinical trials.ParticipantsThe sample included clinical trials published in the top 25% of sports medicine and orthopaedics journals over 9 months.Primary and secondary outcome measuresTwo independent reviewers assessed pre-registration, open data and criteria related to scientific rigour, like randomisation, blinding, and sample size calculations, as well as the study sample, and data analysis.ResultsThe sample included 163 clinical trials from 27 journals. While the majority of trials mentioned rigour criteria, essential details were often missing. Sixty per cent (95% confidence interval (CI) 53% to 68%) of trials reported sample size calculations, but only 32% (95% CI 25% to 39%) justified the expected effect size. Few trials indicated the blinding status of all main stakeholders (4%; 95% CI 1% to 7%). Only 18% (95% CI 12% to 24%) included information on randomisation type, method and concealed allocation. Most trials reported participants' sex/gender (95%; 95% CI 92% to 98%) and information on inclusion and exclusion criteria (78%; 95% CI 72% to 84%). Only 20% (95% CI 14% to 26%) of trials were pre-registered. No trials deposited data in open repositories.ConclusionsThese results will aid the sports medicine and orthopaedics community in developing tailored interventions to improve reporting. While authors typically mention blinding, randomisation and other factors, essential details are often missing. Greater acceptance of open science practices, like pre-registration and open data, is needed. As these practices have been widely encouraged, we discuss systemic interventions that may improve clinical trial reporting.
| S-EPMC9364413 | biostudies-literature
Project description:Importance:Representative racial/ethnic participation in research, especially in clinical trials that establish standards of care, is necessary to minimize disparities in outcomes and to uphold societal equity in health care. Objective:To evaluate the frequency of race reporting and proportional race representation in trials supporting US Food and Drug Administration (FDA) oncology drug approvals. Design, Setting, and Participants:Database study of all reported trials supporting FDA oncology drug approvals granted between July 2008 and June 2018. Primary reports of trials were obtained from PubMed and ClinicalTrials.gov. Food and Drug Administration approvals were identified using the FDA archives. The US population-based cancer estimates by race were calculated using National Cancer Institute-Surveillance, Epidemiology, and End Results and US Census databases. Main Outcomes and Measures:Primary outcomes were the proportion of trials reporting race and the proportion of patients by race participating in trials. Secondary outcomes included race subgroup analyses reporting and gaps between race proportion in trials and the US population. Descriptive statistics, Fisher exact, and ?2 tests were used to analyze the data. Proportions and odds ratios (OR) with 95% CIs were reported. Results:Among 230 trials with a total of 112?293 participants, 145 (63.0%) reported on at least 1 race, 18 (7.8%) documented the 4 major races in the United States (white, Asian, black, and Hispanic), and 58 (25.2%) reported race subgroup analyses. Reporting on white, Asian, black, and Hispanic races was included in 144 (62.6%), 110 (47.8%), 88 (38.2%), and 23 (10.0%) trials, respectively. Between July 2008 and June 2013 vs July 2013 and June 2018, the number of trials reporting race (45 [56.6%] vs 100 [67.1%]; OR, 1.63; 95% CI, 0.93-2.87; P?=?.09) and race subgroup analysis (13 [16.1%] vs 45 [30.2%]; OR, 2.26, 95% CI, 1.16-4.67; P?=?.03) changed minimally and varied across races. Whites, Asians, blacks, and Hispanics represented 76.3%, 18.3%, 3.1% and 6.1% of trial participants, respectively, and the proportion for each race enrolled over time changed nominally (blacks, 3.6% vs 2.9% and Hispanics, 5.3% vs 6.7%) from July 2008 to June 2013 vs July 2013 to June 2018. Compared with their proportion of US cancer incidence, blacks (22% of expected) and Hispanics (44% of expected) were underrepresented compared with whites (98% of expected) and Asians (438% of expected). Conclusions and Relevance:Race and race subgroup analysis reporting occurs infrequently, and black and Hispanic races are consistently underrepresented compared with their burden of cancer incidence in landmark trials that led to FDA oncology drug approvals. Enhanced minority engagement is needed in trials to ensure the validity of results and reliable benefits to all.
| S-EPMC6696743 | biostudies-literature
Project description:OBJECTIVE:To determine (i) the difference in the frequency of serious adverse events (SAEs) reported in trial registrations and their respective primary publications and (ii) the effect of adding SAE data from registries to a network meta-analysis (NMA) in changing the surface under the cumulative ranking (SUCRA) curve values of interventions. DESIGN:Secondary analysis of primary publications from two NMAs. ELIGIBILITY CRITERIA FOR SELECTING STUDIES:We included randomised trials published in English after 2005 that were included in two NMAs of pharmacological interventions for Alzheimer's disease and chronic obstructive pulmonary disease. DATA EXTRACTION:Two reviewers independently searched multiple international trial registries for registration status and abstracted data from the included study publications and ClinicalTrials.gov. RESULTS:Of the 203 randomised trials included, 140 (69.0%) were registered with a trial registry and 72 (35.5%) posted results in the registry. The proportion of registered trials increased over time (38.5% in 2005 vs 78.6% in 2014). Of the publications with results posted in a trial registry, 14 (19.4%) had inconsistent reporting of overall SAEs; 7 (10.4%) studies did not report SAEs in the publication but did in the registry. In the 134 randomised trials with a prespecified primary outcome in the registry, 19 studies (9.4%) had a change in the primary outcome in the publication. Adding SAEs reported in registries to the NMAs did not affect the ranking of interventions. CONCLUSION:We identified inconsistent reporting of SAEs in randomised trials that were included in two NMAs. Findings highlight the importance of including trial registries in the grey literature search and verifying safety data before incorporating it into NMAs. STUDY REGISTRATION:osf.io/mk6dr.
| S-EPMC6731894 | biostudies-other
Project description:The number of biomedical image analysis challenges organized per year is steadily increasing. These international competitions have the purpose of benchmarking algorithms on common data sets, typically to identify the best method for a given problem. Recent research, however, revealed that common practice related to challenge reporting does not allow for adequate interpretation and reproducibility of results. To address the discrepancy between the impact of challenges and the quality (control), the Biomedical Image Analysis ChallengeS (BIAS) initiative developed a set of recommendations for the reporting of challenges. The BIAS statement aims to improve the transparency of the reporting of a biomedical image analysis challenge regardless of field of application, image modality or task category assessed. This article describes how the BIAS statement was developed and presents a checklist which authors of biomedical image analysis challenges are encouraged to include in their submission when giving a paper on a challenge into review. The purpose of the checklist is to standardize and facilitate the review process and raise interpretability and reproducibility of challenge results by making relevant information explicit.
| S-EPMC7441980 | biostudies-literature
Project description:IntroductionWe aimed to assess the reproducibility of empirical research by determining the availability of components required for replication of a study, including materials, raw data, analysis scripts, protocols, and preregistration.MethodsWe used the National Library of Medicine catalog to identify MEDLINE-indexed emergency medicine (EM) journals. Thirty journals met the inclusion criteria. From January 1, 2014-December 31, 2018, 300 publications were randomly sampled using a PubMed search. Additionally, we included four high-impact general medicine journals, which added 106 publications. Two investigators were blinded for independent extraction. Extracted data included statements regarding the availability of materials, data, analysis scripts, protocols, and registration.ResultsAfter the search, we found 25,473 articles, from which we randomly selected 300. Of the 300, only 287 articles met the inclusion criteria. Additionally, we added 106 publications from high-impact journals of which 77 met the inclusion criteria. Together, 364 publications were included, of which 212 articles contained empirical data to analyze. Of the eligible empirical articles, 2.49%, (95% confidence interval [CI], 0.33% to 4.64%] provided a material statement, 9.91% (95% CI, 5.88% to 13.93%) provided a data statement, 0 provided access to analysis scripts, 25.94% (95% CI, 20.04% to 31.84%) linked the protocol, and 39.15% (95% CI, 32.58% to 45.72%) were preregistered.ConclusionStudies in EM lack indicators required for reproducibility. The majority of studies fail to report factors needed to reproduce research to ensure credibility. Thus, an intervention is required and can be achieved through the collaboration of researchers, peer reviewers, funding agencies, and journals.
| S-EPMC8328179 | biostudies-literature
Project description:Gender inequality has exploded as a recent issue within mainstream media across US and UK cultural commentary. High-profile scandals of sexual harassment and gender pay differences have focused attention on the on-going disparity between sexes and political status. This paper presents a novel experiment in the application of so-called "big data" to analyse gender inequality. Using Artificial Intelligence (AI) techniques in the form of Natural Language Processing, a web crawler is used to audit the whole.uk online domain, and to measure the United Kingdom's (UK's) online economic presence for gender representation in terms of: prominence, job roles, and leadership within and across economic sectors. The procedure scans over 200 million web pages, and harvests 157,032 organisations and over 2.3 million people. The results reveal material bias (60%+) towards the representation of men over the majority of economic sectors, and across representation of power and status within job roles and professional titles. The experiment highlights not only new levels of gender bias but also the use of the Internet as a valuable source of plentiful data for social and economic analysis.
| S-EPMC6288449 | biostudies-literature
Project description:ClinicalTrials.gov requires reporting of result summaries for many drug and device trials.To evaluate the consistency of reporting of trials that are registered in the ClinicalTrials.gov results database and published in the literature.ClinicalTrials.gov results database and matched publications identified through ClinicalTrials.gov and a manual search of 2 electronic databases.10% random sample of phase 3 or 4 trials with results in the ClinicalTrials.gov results database, completed before 1 January 2009, with 2 or more groups.One reviewer extracted data about trial design and results from the results database and matching publications. A subsample was independently verified.Of 110 trials with results, most were industry-sponsored, parallel-design drug studies. The most common inconsistency was the number of secondary outcome measures reported (80%). Sixteen trials (15%) reported the primary outcome description inconsistently, and 22 (20%) reported the primary outcome value inconsistently. Thirty-eight trials inconsistently reported the number of individuals with a serious adverse event (SAE); of these, 33 (87%) reported more SAEs in ClinicalTrials.gov. Among the 84 trials that reported SAEs in ClinicalTrials.gov, 11 publications did not mention SAEs, 5 reported them as zero or not occurring, and 21 reported a different number of SAEs. Among 29 trials that reported deaths in ClinicalTrials.gov, 28% differed from the matched publication.Small sample that included earliest results posted to the database.Reporting discrepancies between the ClinicalTrials.gov results database and matching publications are common. Which source contains the more accurate account of results is unclear, although ClinicalTrials.gov may provide a more comprehensive description of adverse events than the publication.Agency for Healthcare Research and Quality.
| S-EPMC4617780 | biostudies-literature
Project description:Audit and feedback (A&F) is a common strategy for helping health providers to implement evidence into practice. Despite being extensively studied, health care A&F interventions remain variably effective, with overall effect sizes that have not improved since 2003. Contributing to this stagnation is the fact that most health care A&F interventions have largely been designed without being informed by theoretical understanding from the behavioral and social sciences. To determine if the trend can be improved, the objective of this study was to develop a list of testable, theory-informed hypotheses about how to design more effective A&F interventions.Using purposive sampling, semi-structured 60-90-min telephone interviews were conducted with experts in theories related to A&F from a range of fields (e.g., cognitive, health and organizational psychology, medical decision-making, economics). Guided by detailed descriptions of A&F interventions from the health care literature, interviewees described how they would approach the problem of designing improved A&F interventions. Specific, theory-informed hypotheses about the conditions for effective design and delivery of A&F interventions were elicited from the interviews. The resulting hypotheses were assigned by three coders working independently into themes, and categories of themes, in an iterative process.We conducted 28 interviews and identified 313 theory-informed hypotheses, which were placed into 30 themes. The 30 themes included hypotheses related to the following five categories: A&F recipient (seven themes), content of the A&F (ten themes), process of delivery of the A&F (six themes), behavior that was the focus of the A&F (three themes), and other (four themes).We have identified a set of testable, theory-informed hypotheses from a broad range of behavioral and social science that suggest conditions for more effective A&F interventions. This work demonstrates the breadth of perspectives about A&F from non-healthcare-specific disciplines in a way that yields testable hypotheses for healthcare A&F interventions. These results will serve as the foundation for further work seeking to set research priorities among the A&F research community.
| S-EPMC5622490 | biostudies-other