Dataset Information

A Systematic Review of the Reporting Quality of Observational Studies That Use Mediation Analyses.

ABSTRACT: Mediation analysis is a common statistical method used to investigate mechanisms of health exposure and interventions. The reporting quality of mediation studies used in randomised controlled trials has been considered heterogeneous and incomplete. The reporting quality of mediation analysis in observational studies is unknown. We conducted a systematic review to describe the reporting standards of recently published observational studies that used mediation analysis to understand the mechanism of health exposures. We searched for studies published between June 2017 and June 2019 indexed in EMBASE, MEDLINE and PsycINFO. Two reviewers screened articles and selected a random sample of 50 eligible studies for inclusion. We included studies across 13 healthcare fields and ten different health conditions. Most studies (74%) collected data on healthy individuals to assess their risk of developing a health disorder. Psychosocial and behavioural factors (self-control, self-esteem, alcohol consumption, pain) were the most prevalent exposures (n = 30, 60%), outcomes (n = 23, 46%) and mediators (n = 29, 58%). Most studies used a cross-sectional design (64%, n = 32), and a few studies reported sample size calculations (4%, n = 8). In 20% (n = 10) of the studies, adjustment for confounders was reported. Only 10% (n = 5) of studies reported the assumptions underlying the mediation analysis, and 14% (n = 7) of studies conducted some sensitivity analysis to assess the degree which unmeasured confounders would affect the estimate of the mediation effect. Mediation analysis is a common method used to investigate mechanisms in prevention research. The reporting of mediation analysis in observational studies is incomplete and may impact reproducibility, evidence synthesis and implementation.

SUBMITTER: Rizzo RRN

PROVIDER: S-EPMC9343342 | biostudies-literature | 2022 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Systematic Review of the Reporting Quality of Observational Studies That Use Mediation Analyses.

Rizzo Rodrigo R N RRN Cashin Aidan G AG Bagg Matthew K MK Gustin Sylvia M SM Lee Hopin H McAuley James H JH

Prevention science : the official journal of the Society for Prevention Research 20220215 6

Mediation analysis is a common statistical method used to investigate mechanisms of health exposure and interventions. The reporting quality of mediation studies used in randomised controlled trials has been considered heterogeneous and incomplete. The reporting quality of mediation analysis in observational studies is unknown. We conducted a systematic review to describe the reporting standards of recently published observational studies that used mediation analysis to understand the mechanism ...[more]

PMID: 35167030

Similar Datasets

Project description:ImportanceMediation analyses of randomized trials and observational studies can generate evidence about the mechanisms by which interventions and exposures may influence health outcomes. Publications of mediation analyses are increasing, but the quality of their reporting is suboptimal.ObjectiveTo develop international, consensus-based guidance for the reporting of mediation analyses of randomized trials and observational studies (A Guideline for Reporting Mediation Analyses; AGReMA).Design, setting, and participantsThe AGReMA statement was developed using the Enhancing Quality and Transparency of Health Research (EQUATOR) methodological framework for developing reporting guidelines. The guideline development process included (1) an overview of systematic reviews to assess the need for a reporting guideline; (2) review of systematic reviews of relevant evidence on reporting mediation analyses; (3) conducting a Delphi survey with panel members that included methodologists, statisticians, clinical trialists, epidemiologists, psychologists, applied clinical researchers, clinicians, implementation scientists, evidence synthesis experts, representatives from the EQUATOR Network, and journal editors (n = 19; June-November 2019); (4) having a consensus meeting (n = 15; April 28-29, 2020); and (5) conducting a 4-week external review and pilot test that included methodologists and potential users of AGReMA (n = 21; November 2020).ResultsA previously reported overview of 54 systematic reviews of mediation studies demonstrated the need for a reporting guideline. Thirty-three potential reporting items were identified from 3 systematic reviews of mediation studies. Over 3 rounds, the Delphi panelists ranked the importance of these items, provided 60 qualitative comments for item refinement and prioritization, and suggested new items for consideration. All items were reviewed during a 2-day consensus meeting and participants agreed on a 25-item AGReMA statement for studies in which mediation analyses are the primary focus and a 9-item short-form AGReMA statement for studies in which mediation analyses are a secondary focus. These checklists were externally reviewed and pilot tested by 21 expert methodologists and potential users, which led to minor adjustments and consolidation of the checklists.Conclusions and relevanceThe AGReMA statement provides recommendations for reporting primary and secondary mediation analyses of randomized trials and observational studies. Improved reporting of studies that use mediation analyses could facilitate peer review and help produce publications that are complete, accurate, transparent, and reproducible.

Project description:ImportanceObservational (nonexperimental) studies that aim to emulate a randomized trial (ie, the target trial) are increasingly informing medical and policy decision-making, but it is unclear how these studies are reported in the literature. Consistent reporting is essential for quality appraisal, evidence synthesis, and translation of evidence to policy and practice.ObjectiveTo assess the reporting of observational studies that explicitly aimed to emulate a target trial.Evidence reviewWe searched Medline, Embase, PsycINFO, and Web of Science for observational studies published between March 2012 and October 2022 that explicitly aimed to emulate a target trial of a health or medical intervention. Two reviewers double-screened and -extracted data on study characteristics, key predefined components of the target trial protocol and its emulation (eligibility criteria, treatment strategies, treatment assignment, outcome[s], follow-up, causal contrast[s], and analysis plan), and other items related to the target trial emulation.FindingsA total of 200 studies that explicitly aimed to emulate a target trial were included. These studies included 26 subfields of medicine, and 168 (84%) were published from January 2020 to October 2022. The aim to emulate a target trial was explicit in 70 study titles (35%). Forty-three studies (22%) reported use of a published reporting guideline (eg, Strengthening the Reporting of Observational Studies in Epidemiology). Eighty-five studies (43%) did not describe all key items of how the target trial was emulated and 113 (57%) did not describe the protocol of the target trial and its emulation.Conclusion and relevanceIn this systematic review of 200 studies that explicitly aimed to emulate a target trial, reporting of how the target trial was emulated was inconsistent. A reporting guideline for studies explicitly aiming to emulate a target trial may improve the reporting of the target trial protocols and other aspects of these emulation attempts.

Project description:IntroductionComplete reporting assists readers in confirming the methodological rigor and validity of findings and allows replication. The reporting quality of observational functional magnetic resonance imaging (fMRI) studies involving clinical participants is unclear.ObjectivesWe sought to determine the quality of reporting in observational fMRI studies involving clinical participants.MethodsWe searched OVID MEDLINE for fMRI studies in six leading journals between January 2010 and December 2011.Three independent reviewers abstracted data from articles using an 83-item checklist adapted from the guidelines proposed by Poldrack et al. (Neuroimage 2008; 40: 409-14). We calculated the percentage of articles reporting each item of the checklist and the percentage of reported items per article.ResultsA random sample of 100 eligible articles was included in the study. Thirty-one items were reported by fewer than 50% of the articles and 13 items were reported by fewer than 20% of the articles. The median percentage of reported items per article was 51% (ranging from 30% to 78%). Although most articles reported statistical methods for within-subject modeling (92%) and for between-subject group modeling (97%), none of the articles reported observed effect sizes for any negative finding (0%). Few articles reported justifications for fixed-effect inferences used for group modeling (3%) and temporal autocorrelations used to account for within-subject variances and correlations (18%). Other under-reported areas included whether and how the task design was optimized for efficiency (22%) and distributions of inter-trial intervals (23%).ConclusionsThis study indicates that substantial improvement in the reporting of observational clinical fMRI studies is required. Poldrack et al.'s guidelines provide a means of improving overall reporting quality. Nonetheless, these guidelines are lengthy and may be at odds with strict word limits for publication; creation of a shortened-version of Poldrack's checklist that contains the most relevant items may be useful in this regard.

Project description:BackgroundAutomated language analysis of radiology reports using natural language processing (NLP) can provide valuable information on patients' health and disease. With its rapid development, NLP studies should have transparent methodology to allow comparison of approaches and reproducibility. This systematic review aims to summarise the characteristics and reporting quality of studies applying NLP to radiology reports.MethodsWe searched Google Scholar for studies published in English that applied NLP to radiology reports of any imaging modality between January 2015 and October 2019. At least two reviewers independently performed screening and completed data extraction. We specified 15 criteria relating to data source, datasets, ground truth, outcomes, and reproducibility for quality assessment. The primary NLP performance measures were precision, recall and F1 score.ResultsOf the 4,836 records retrieved, we included 164 studies that used NLP on radiology reports. The commonest clinical applications of NLP were disease information or classification (28%) and diagnostic surveillance (27.4%). Most studies used English radiology reports (86%). Reports from mixed imaging modalities were used in 28% of the studies. Oncology (24%) was the most frequent disease area. Most studies had dataset size > 200 (85.4%) but the proportion of studies that described their annotated, training, validation, and test set were 67.1%, 63.4%, 45.7%, and 67.7% respectively. About half of the studies reported precision (48.8%) and recall (53.7%). Few studies reported external validation performed (10.8%), data availability (8.5%) and code availability (9.1%). There was no pattern of performance associated with the overall reporting quality.ConclusionsThere is a range of potential clinical applications for NLP of radiology reports in health services and research. However, we found suboptimal reporting quality that precludes comparison, reproducibility, and replication. Our results support the need for development of reporting standards specific to clinical NLP studies.

Project description:BackgroundDiabetes mellitus (DM) is a major health concern among children with the widespread adoption of advanced technologies. However, concerns are growing about the transparency, replicability, biasedness, and overall validity of artificial intelligence studies in medicine.ObjectiveWe aimed to systematically review the reporting quality of machine learning (ML) studies of pediatric DM using the Minimum Information About Clinical Artificial Intelligence Modelling (MI-CLAIM) checklist, a general reporting guideline for medical artificial intelligence studies.MethodsWe searched the PubMed and Web of Science databases from 2016 to 2020. Studies were included if the use of ML was reported in children with DM aged 2 to 18 years, including studies on complications, screening studies, and in silico samples. In studies following the ML workflow of training, validation, and testing of results, reporting quality was assessed via MI-CLAIM by consensus judgments of independent reviewer pairs. Positive answers to the 17 binary items regarding sufficient reporting were qualitatively summarized and counted as a proxy measure of reporting quality. The synthesis of results included testing the association of reporting quality with publication and data type, participants (human or in silico), research goals, level of code sharing, and the scientific field of publication (medical or engineering), as well as with expert judgments of clinical impact and reproducibility.ResultsAfter screening 1043 records, 28 studies were included. The sample size of the training cohort ranged from 5 to 561. Six studies featured only in silico patients. The reporting quality was low, with great variation among the 21 studies assessed using MI-CLAIM. The number of items with sufficient reporting ranged from 4 to 12 (mean 7.43, SD 2.62). The items on research questions and data characterization were reported adequately most often, whereas items on patient characteristics and model examination were reported adequately least often. The representativeness of the training and test cohorts to real-world settings and the adequacy of model performance evaluation were the most difficult to judge. Reporting quality improved over time (r=0.50; P=.02); it was higher than average in prognostic biomarker and risk factor studies (P=.04) and lower in noninvasive hypoglycemia detection studies (P=.006), higher in studies published in medical versus engineering journals (P=.004), and higher in studies sharing any code of the ML pipeline versus not sharing (P=.003). The association between expert judgments and MI-CLAIM ratings was not significant.ConclusionsThe reporting quality of ML studies in the pediatric population with DM was generally low. Important details for clinicians, such as patient characteristics; comparison with the state-of-the-art solution; and model examination for valid, unbiased, and robust results, were often the weak points of reporting. To assess their clinical utility, the reporting standards of ML studies must evolve, and algorithms for this challenging population must become more transparent and replicable.

Project description:BackgroundThe QUOROM and PRISMA statements were published in 1999 and 2009, respectively, to improve the consistency of reporting systematic reviews (SRs)/meta-analyses (MAs) of clinical trials. However, not all SRs/MAs adhere completely to these important standards. In particular, it is not clear how well SRs/MAs of acupuncture studies adhere to reporting standards and which reporting criteria are generally ignored in these analyses.ObjectivesTo evaluate reporting quality in SRs/MAs of acupuncture studies.MethodsWe performed a literature search for studies published prior to 2014 using the following public archives: PubMed, EMBASE, Web of Science, the Cochrane Database of Systematic Reviews (CDSR), the Chinese Biomedical Literature Database (CBM), the Traditional Chinese Medicine (TCM) database, the Chinese Journal Full-text Database (CJFD), the Chinese Scientific Journal Full-text Database (CSJD), and the Wanfang database. Data were extracted into pre-prepared Excel data-extraction forms. Reporting quality was assessed based on the PRISMA checklist (27 items).ResultsOf 476 appropriate SRs/MAs identified in our search, 203, 227, and 46 were published in Chinese journals, international journals, and the Cochrane Database, respectively. In 476 SRs/MAs, only 3 reported the information completely. By contrast, approximately 4.93% (1/203), 8.81% (2/227) and 0.00% (0/46) SRs/Mas reported less than 10 items in Chinese journals, international journals and CDSR, respectively. In general, the least frequently reported items (reported≤50%) in SRs/MAs were "protocol and registration", "risk of bias across studies", and "additional analyses" in both methods and results sections.ConclusionsSRs/MAs of acupuncture studies have not comprehensively reported information recommended in the PRISMA statement. Our study underscores that, in addition to focusing on careful study design and performance, attention should be paid to comprehensive reporting standards in SRs/MAs on acupuncture studies.

Project description:To determine the association of socioeconomic disadvantage with the prevalence of childhood disabling chronic conditions in high-income countries.Systematic review and meta-analyses.6 electronic databases, relevant websites, reference lists and experts in the field.160 observational studies conducted in high-income countries with data on socioeconomic status and disabling chronic conditions in childhood, published between 1 January 1991 and 31 December 2013.Abstracts were reviewed, full papers obtained, and papers identified for inclusion by 2 independent reviewers. Inclusion decisions were checked by a third reviewer. Where reported, ORs were extracted for low versus high socioeconomic status. For studies reporting raw data but not ORs, ORs were calculated. Narrative analysis was undertaken for studies without data suitable for meta-analysis.126 studies had data suitable for meta-analysis. ORs for risk estimates were: all-cause disabling chronic conditions 1.72 (95% CI 1.48 to 2.01); psychological disorders 1.88 (95% CI 1.68 to 2.10); intellectual disability 2.41 (95% CI 2.03 to 2.86); activity-limiting asthma 2.20 (95% CI 1.87 to 2.85); cerebral palsy 1.42 (95% CI 1.26 to 1.61); congenital abnormalities 1.41 (95% CI 1.24 to 1.61); epilepsy 1.38 (95% CI 1.20 to 1.59); sensory impairment 1.70 (95% CI 1.39 to 2.07). Heterogeneity was high across most estimates (I(2)>75%). Of the 34 studies without data suitable for meta-analysis, 26 reported results consistent with increased risk associated with low socioeconomic status.The findings indicate that, in high-income countries, childhood disabling chronic conditions are associated with social disadvantage. Although evidence of an association is consistent across different countries, the review provides limited evidence to explain the association; future research, using longitudinal data, will be required to distinguish low socioeconomic status as the cause or consequence of childhood disabling chronic conditions and the aetiological pathways and mechanisms.

Project description:AimsWe conducted a systematic review assessing the reporting quality of studies validating models based on machine learning (ML) for clinical diagnosis, with a specific focus on the reporting of information concerning the participants on which the diagnostic task was evaluated on.MethodMedline Core Clinical Journals were searched for studies published between July 2015 and July 2018. Two reviewers independently screened the retrieved articles, a third reviewer resolved any discrepancies. An extraction list was developed from the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis guideline. Two reviewers independently extracted the data from the eligible articles. Third and fourth reviewers checked, verified the extracted data as well as resolved any discrepancies between the reviewers.ResultsThe search results yielded 161 papers, of which 28 conformed to the eligibility criteria. Detail of data source was reported in 24 of the 28 papers. For all of the papers, the set of patients on which the ML-based diagnostic system was evaluated was partitioned from a larger dataset, and the method for deriving such set was always reported. Information on the diagnostic/non-diagnostic classification was reported well (23/28). The least reported items were the use of reporting guideline (0/28), distribution of disease severity (8/28 patient flow diagram (10/28) and distribution of alternative diagnosis (10/28). A large proportion of studies (23/28) had a delay between the conduct of the reference standard and ML tests, while one study did not and four studies were unclear. For 15 studies, it was unclear whether the evaluation group corresponded to the setting in which the ML test will be applied to.ConclusionAll studies in this review failed to use reporting guidelines, and a large proportion of them lacked adequate detail on participants, making it difficult to replicate, assess and interpret study findings.Prospero registration numberCRD42018099167.

Dataset Information

A Systematic Review of the Reporting Quality of Observational Studies That Use Mediation Analyses.

Publications

A Systematic Review of the Reporting Quality of Observational Studies That Use Mediation Analyses.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets