The use and quality of reporting of propensity score methods in multiple sclerosis literature: A review.
Ontology highlight
ABSTRACT: Background
Propensity score (PS) analyses are increasingly used in multiple sclerosis (MS) research, largely owing to the greater availability of large observational cohorts and registry databases.Objective
To evaluate the use and quality of reporting of PS methods in the recent MS literature.Methods
We searched the PubMed database for articles published between January 2013 and July 2019. We restricted the search to comparative effectiveness studies of two disease-modifying therapies.Results
Thirty-nine studies were included in the review, with most studies (62%) published within the past 3 years. All studies reported the list of covariates used for the PS model, but only 21% of studies mentioned how those covariates were selected. Most studies used PS matching (72%), followed by PS adjustment (18%), weighting (15%), and stratification (3%), with some overlap. Most studies using matching or weighting reported checking post-PS covariate imbalance (91%), although about 45% of these studies relied on p values from various statistical tests. Only 25% of studies using matching reported calculating robust standard errors for the PS analyses.Conclusions
The quality of reporting of PS methods in the MS literature is sub-optimal in general, and in some cases, inappropriate methods are used.
SUBMITTER: Karim ME
PROVIDER: S-EPMC9260477 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundWith many disease-modifying therapies currently approved for the management of multiple sclerosis, there is a growing need to evaluate the comparative effectiveness and safety of those therapies from real-world data sources. Propensity score methods have recently gained popularity in multiple sclerosis research to generate real-world evidence. Recent evidence suggests, however, that the conduct and reporting of propensity score analyses are often suboptimal in multiple sclerosis studies.ObjectivesTo provide practical guidance to clinicians and researchers on the use of propensity score methods within the context of multiple sclerosis research.MethodsWe summarize recommendations on the use of propensity score matching and weighting based on the current methodological literature, and provide examples of good practice.ResultsStep-by-step recommendations are presented, starting with covariate selection and propensity score estimation, followed by guidance on the assessment of covariate balance and implementation of propensity score matching and weighting. Finally, we focus on treatment effect estimation and sensitivity analyses.ConclusionThis comprehensive set of recommendations highlights key elements that require careful attention when using propensity score methods.
| S-EPMC9260471 | biostudies-literature
Project description:BackgroundPropensity score techniques can reduce confounding and bias in observational studies. Such analyses are able to measure and balance pre-determined covariates between treated and untreated groups, leading to results that can approximate those generated by randomized prospective studies when such trials are not feasible. The most commonly used propensity score -based analytic technique is propensity score matching (PSM). Although PSM popularity has continued to increase in medical literature, improper methodology or methodological reporting may lead to biased interpretation of treatment effects or limited scientific reproducibility and generalizability. In this study, we aim to characterize and assess the quality of PSM methodology reporting in high-impact otolaryngologic literature.MethodsPubMed and Embase based systematic review of the top 20 journals in otolaryngology, as measured by impact factor from the Journal Citations Reports from 2012 to 2018, for articles using PSM analysis throughout their publication history. Eligible articles were reviewed and assessed for quality and reporting of PSM methodology.ResultsOur search yielded 101 studies, of which 92 were eligible for final analysis and review. The proportion of studies utilizing PSM increased significantly over time (p < 0.001). Nearly all studies (96.7%, n = 89) specified the covariates used to calculate propensity scores. Covariate balance was illustrated in 67.4% (n = 62) of studies, most frequently through p-values. A minority (17.4%, n = 16) of studies were found to be fully reproducible according to previously established criteria.ConclusionsWhile PSM analysis is becoming increasingly prevalent in otolaryngologic literature, the quality of PSM methodology reporting can be improved. We provide potential recommendations for authors regarding optimal reporting for analyses using PSM.
| S-EPMC7774981 | biostudies-literature
Project description:Background: Propensity score (PS) analysis is increasingly being used in observational studies, especially in some cancer studies where random assignment is not feasible. This systematic review evaluates the use and reporting quality of PS analysis in oncology studies.Methods: We searched PubMed to identify the use of PS methods in cancer studies (CS) and cancer surgical studies (CSS) in major medical, cancer, and surgical journals over time and critically evaluated 33 CS published in top medical and cancer journals in 2014 and 2015 and 306 CSS published up to November 26, 2015, without earlier date limits. The quality of reporting in PS analysis was evaluated. It was also compared over time and among journals with differing impact factors. All statistical tests were two-sided.ResultsMore than 50% of the publications with PS analysis from the past decade occurred within the past two years. Of the studies critically evaluated, a considerable proportion did not clearly provide the variables used to estimate PS (CS 12.1%, CSS 8.8%), incorrectly included non baseline variables (CS 3.4%, CSS 9.3%), neglected the comparison of baseline characteristics (CS 21.9%, CSS 15.6%), or did not report the matching algorithm utilized (CS 19.0%, CSS 36.1%). In CSS, the reporting of the matching algorithm improved in 2014 and 2015 ( P ?=?.04), and the reporting of variables used to estimate PS was better in top surgery journals ( P ?=?.008). However, there were no statistically significant differences for the inclusion of non baseline variables and reporting of comparability of baseline characteristics.ConclusionsThe use of PS in cancer studies has dramatically increased recently, but there is substantial room for improvement in the quality of reporting even in top journals. Herein we have proposed reporting guidelines for PS analyses that are broadly applicable to different areas of medical research that will allow better evaluation and comparison across studies applying this approach.
| S-EPMC6059208 | biostudies-literature
Project description:IntroductionSeveral published systematic reviews have examined the potential associations between e-cigarette use and cigarette smoking, but their methodological and/or reporting quality have not yet been assessed. This systematic quality review followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and AMSTAR (A MeaSurement Tool to Assess systematic Reviews) 2 to evaluate the quality of systematic reviews investigating potential associations between e-cigarette use and cigarette smoking.Materials and methodsPubMed/MEDLINE, Embase, and PsycINFO were searched from 01 January 2007 to 24 June 2020. Methodological quality was assessed using AMSTAR 2, and reporting quality was assessed using PRISMA guidelines.ResultsOf 331 potentially relevant systematic reviews, 20 met predefined inclusion criteria. Most reviews (n = 15; 75%) reported on e-cigarette use and cigarette smoking cessation, while three reported on e-cigarette use and cigarette smoking initiation (15%); and two reported on cigarette smoking initiation and cessation (10%). According to AMSTAR 2 guidelines, 18 of the 20 reviews (90%) were "critically low" in overall confidence of the results, while two were ranked "low." Additionally, reporting quality varied across the reviews, with only 60% reporting at least half of the PRISMA items.DiscussionMethodological limitations were identified across reviews examining potential associations between e-cigarette use and cigarette smoking behaviors, indicating that findings from these reviews should be interpreted with caution.ConclusionsFuture systematic reviews in this field should strive to adhere to AMSTAR 2 and PRISMA guidelines, to provide high quality syntheses of the available data with transparent and complete reporting.
| S-EPMC8627036 | biostudies-literature
Project description:BackgroundObservational analysis methods can be refined by benchmarking against randomized trials. We reviewed studies systematically comparing observational analyses using propensity score methods against randomized trials to explore whether intervention or outcome characteristics predict agreement between designs.MethodsWe searched PubMed (from January 1, 2000, to April 30, 2017), the AHRQ Scientific Resource Center Methods Library, reference lists, and bibliographies to identify systematic reviews that compared estimates from observational analyses using propensity scores against randomized trials across three or more clinical topics; reported extractable relative risk (RR) data; and were published in English. One reviewer extracted data from all eligible systematic reviews; a second reviewer verified the extracted data.ResultsSix systematic reviews matching published observational studies to randomized trials, published between 2012 and 2016, met our inclusion criteria. The reviews reported on 127 comparisons overall, in cardiology (29 comparisons), surgery (49), critical care medicine and sepsis (46), nephrology (2), and oncology (1). Disagreements were large (relative RR < 0.7 or > 1.43) in 68 (54%) and statistically significant in 12 (9%) of the comparisons. The degree of agreement varied among reviews but was not strongly associated with intervention or outcome characteristics.DiscussionDisagreements between observational studies using propensity score methods and randomized trials can occur for many reasons and the available data cannot be used to discern the reasons behind specific disagreements. Better benchmarking of observational analyses using propensity scores (and other causal inference methods) is possible using observational studies that explicitly attempt to emulate target trials.
| S-EPMC7210373 | biostudies-literature
Project description:The maintenance of the excitability of neurons and circuits is a fundamental process for healthy brain functions. One of the main homeostatic mechanisms responsible for such regulation is synaptic scaling. While this type of plasticity is well-characterized through a robust body of literature, there are no systematic evaluations of the methodological and reporting features from these studies. Our review yielded 168 articles directly investigating synaptic scaling mechanisms, which display relatively high impact, with a median impact factor of 7.76 for the publishing journals. Our methodological analysis identified that 86% of the articles made use of inhibitory interventions to induce synaptic scaling, while only 41% of those studies contain excitatory manipulations. To verify the effects of synaptic scaling, the most assessed outcome was miniature excitatory postsynaptic current (mEPSC) recordings, performed in 71% of the articles. We could also observe that the field is mostly focused on mechanistic studies of the synaptic scaling pathways (70%), rather than the interaction with other types of plasticity, such as Hebbian processes (4%). We found that more than half of the articles failed to describe simple features, such as regulatory compliance statements, ethics committee approval, or statements of conflict of interests. In light of these results, we discuss the strengths and pitfalls existing in synaptic scaling literature.
| S-EPMC7309364 | biostudies-literature
Project description:BackgroundMixed methods designs are increasingly used in health care research to enrich findings. However, little is known about the frequency of use of this methodology in chiropractic research, or the quality of reporting among chiropractic studies using mixed methods.ObjectiveTo quantify the use and quality of mixed methods in chiropractic research, and explore the association of study characteristics (e.g., authorship, expertise, journal impact factor, country and year of publication) with reporting quality.MethodsWe will conduct a systematic search of MEDLINE, EMBASE, CINAHL, and the Index to Chiropractic Literature to identify all chiropractic mixed methods studies published from inception of each database to December 31, 2020. Articles reporting the use of both qualitative and quantitative methods, or mixed qualitative methods, will be included. Pairs of reviewers will perform article screening, data extraction, risk of bias with the Mixed Methods Appraisal Tool (MMAT), and appraisal of reporting quality using the Good Reporting of A Mixed Methods Study (GRAMMS) guideline. We will explore the correlation between GRAMMS and MMAT scores, and construct generalized estimating equations to explore factors associated with reporting quality.DiscussionThis will be the first methodological review to examine the reporting quality of published mixed methods studies involving chiropractic research. The results of our review will inform opportunities to improve reporting in chiropractic mixed methods studies. Our results will be disseminated in a peer-reviewed publication and presented publicly at conferences and as part of a doctoral thesis.
| S-EPMC8442283 | biostudies-literature
Project description:Quantifying cell subpopulations in biological fluids aids in diagnosis and understanding of the mechanisms of injury. Although much has been learned from cerebrospinal fluid (CSF) flow cytometry in neuroimmunological disorders, such as multiple sclerosis (MS), previous studies did not contain enough healthy donors (HD) to derive age- and gender-related normative data and sufficient heterogeneity of other inflammatory neurological disease (OIND) controls to identify MS specific changes. The goals of this blinded training and validation study of MS patients and embedded controls, representing 1,240 prospectively acquired paired CSF/blood samples from 588 subjects was (1) to define physiological age-/gender-related changes in CSF cells, (2) to define/validate cellular abnormalities in blood and CSF of untreated MS through disease duration (DD) and determine which are MS-specific, and (3) to compare effect(s) of low-efficacy (i.e., interferon-beta [IFN-beta] and glatiramer acetate [GA]) and high-efficacy drugs (i.e., natalizumab, daclizumab, and ocrelizumab) on MS-related cellular abnormalities using propensity score matching. Physiological gender differences are less pronounced in the CSF compared to blood, and age-related changes suggest decreased immunosurveillance of CNS by activated HLA-DR+T cells associated with natural aging. Results from patient samples support the concept of MS being immunologically single disease evolving in time. Initially, peripherally activated innate and adaptive immune cells migrate into CSF to form MS lesions. With progression, T cells (CD8+ > CD4+), NK cells, and myeloid dendritic cells are depleted from blood as they continue to accumulate, together with B cells, in the CSF and migrate to CNS tissue, forming compartmentalized inflammation. All MS drugs inhibit non-physiological accumulation of immune cells in the CSF. Although low-efficacy drugs tend to normalize it, high-efficacy drugs overshoot some aspects of CSF physiology, suggesting impairment of CNS immunosurveillance. Comparable inhibition of MS-related CSF abnormalities advocates changes within CNS parenchyma responsible for differences in drug efficacy on MS disability progression. Video summarizing all results may become useful educational tool.
| S-EPMC7732581 | biostudies-literature
Project description:Non-potassium-sparing diuretics are commonly used in heart failure (HF). They activate the neurohormonal system, and are potentially harmful. Yet, the long-term effects of chronic diuretic use in HF are largely unknown. We retrospectively analysed the Digitalis Investigation Group (DIG) data to determine the effects of diuretics on HF outcomes.Propensity scores for diuretic use were calculated for each of the 7788 DIG participants using a non-parsimonious multivariable logistic regression model, and were used to match 1391 (81%) no-diuretic patients with 1391 diuretic patients. Effects of diuretics on mortality and hospitalization at 40 months of median follow-up were assessed using matched Cox regression models. All-cause mortality was 21% for no-diuretic patients and 29% for diuretic patients [hazard ratio (HR) 1.31; 95% confidence interval (CI) 1.11-1.55; P = 0.002]. HF hospitalizations occurred in 18% of no-diuretic patients and 23% of diuretic patients (HR 1.37; 95% CI 1.13-1.65; P = 0.001).Chronic diuretic use was associated with increased long-term mortality and hospitalizations in a wide spectrum of ambulatory chronic systolic and diastolic HF patients. The findings of the current study challenge the wisdom of routine chronic use of diuretics in HF patients who are asymptomatic or minimally symptomatic without fluid retention, and are on complete neurohormonal blockade. These findings, based on a non-randomized design, need to be further studied in randomized trials.
| S-EPMC2443408 | biostudies-other
Project description:BackgroundMultiple sclerosis (MS) is a chronic, demyelinating disease of the central nervous system that results in progressive and irreversible disability. Fatigue is one of the most common MS-related symptoms and is characterized by a persistent lack of energy that impairs daily functioning. The burden of MS-related fatigue is complex and multidimensional, and to our knowledge, no systematic literature review has been conducted on this subject. The purpose of this study was to conduct a systematic literature review on the epidemiology and burden of fatigue in people with multiple sclerosis (pwMS).MethodsSystematic searches were conducted in MEDLINE, Embase, and Evidence-Based Medicine Reviews to identify relevant studies of fatigue in pwMS. English-language records published from 2010 to January 2020 that met predefined eligibility criteria were included. We initially selected studies that reported quality of life (QoL) and economic outcomes according to categories of fatigue (e.g., fatigued vs non-fatigued). Studies assessing associations between economic outcomes and fatigue as a continuous measure were later included to supplement the available data.ResultsThe search identified 8147 unique records, 54 of which met the inclusion criteria. Of these, 39 reported epidemiological outcomes, 11 reported QoL, and 9 reported economic outcomes. The supplementary screen for economic studies with fatigue as a continuous measure included an additional 20 records. Fatigue prevalence in pwMS ranged from 36.5 to 78.0%. MS-related fatigue was consistently associated with significantly lower QoL. Results on the economic impact of fatigue were heterogeneous, but most studies reported a significant association between presence or severity of fatigue and employment status, capacity to work, and sick leave. There was a gap in evidence regarding the direct costs of MS-related fatigue and the burden experienced by caregivers of pwMS.ConclusionFatigue is a prevalent symptom in pwMS and is associated with considerable QoL and economic burden. There are gaps in the evidence related to the direct costs of MS-related fatigue and the burden of fatigue on caregivers. Addressing fatigue over the clinical course of the disease may improve health and economic outcomes for patients with MS.
| S-EPMC8638268 | biostudies-literature