Development of a Methodological Quality Criteria List for Observational Studies: The Observational Study Quality Evaluation.
Ontology highlight
ABSTRACT:
SUBMITTER: Drukker M
PROVIDER: S-EPMC8317224 | biostudies-literature | 2021
REPOSITORIES: biostudies-literature
Publications
Development of a Methodological Quality Criteria List for Observational Studies: The Observational Study Quality Evaluation.
Drukker Marjan M Weltens Irene I van Hooijdonk Carmen F M CFM Vandenberk Emma E Bak Maarten M
Frontiers in research metrics and analytics 20210714
<b>Background:</b> Existing study quality and risk of bias lists for observational studies have important disadvantages. For this reason, a comprehensive widely applicable quality assessment tool for observational studies was developed. <b>Methods:</b> Criteria from three quality lists were merged into a new quality assessment tool: the observational study quality evaluation (OSQE). OSQE consists of a cohort, case-control, and cross-sectional version. <b>Results:</b> The OSQE cohort, the OSQE ca ...[more]
Similar Datasets
Project description:Background and objectiveObservational studies may provide valuable evidence on real-world causal effects of drug effectiveness in patients with coronavirus disease 2019 (COVID-19). As patients are usually observed from hospital admission to discharge and drug initiation starts during hospitalization, advanced statistical methods are needed to account for time-dependent drug exposure, confounding and competing events. Our objective is to evaluate the observational studies on the three common methodological pitfalls in time-to-event analyses: immortal time bias, confounding bias and competing risk bias.MethodsWe performed a systematic literature search on 23 October 2020, in the PubMed database to identify observational cohort studies that evaluated drug effectiveness in hospitalized patients with COVID-19. We included articles published in four journals: British Medical Journal, New England Journal of Medicine, Journal of the American Medical Association and The Lancet as well as their sub-journals.ResultsOverall, out of 255 articles screened, 11 observational cohort studies on treatment effectiveness with drug exposure-outcome associations were evaluated. All studies were susceptible to one or more types of bias in the primary study analysis. Eight studies had a time-dependent treatment. However, the hazard ratios were not adjusted for immortal time in the primary analysis. Even though confounders presented at baseline have been addressed in nine studies, time-varying confounding caused by time-varying treatment exposure and clinical variables was less recognized. Only one out of 11 studies addressed competing event bias by extending follow-up beyond patient discharge.ConclusionsIn the observational cohort studies on drug effectiveness for treatment of COVID-19 published in four high-impact journals, the methodological biases were concerningly common. Appropriate statistical tools are essential to avoid misleading conclusions and to obtain a better understanding of potential treatment effects.
| S-EPMC8015394 | biostudies-literature
Project description:Introduction and objectiveSystematic review tools that provide guidance on evaluating epidemiology studies are receiving increasing attention and support because their application facilitates improved quality of the review, consistency across reviewers, and transparency for readers. The U.S. Environmental Protection Agency's Integrated Risk Information System (IRIS) Program has developed an approach for systematic review of evidence of health effects from chemical exposures that includes structured approaches for literature search and screening, study evaluation, data extraction, and evidence synthesis and integration. This approach recognizes the need for developing outcome-specific criteria for study evaluation. Because studies are assessed at the outcome level, a study could be considered high quality for one investigated outcome, and low quality for another, due to differences in the outcome measures, analytic strategies, how relevant a certain bias is to the outcome, and how the exposure measure relates to the outcome. The objective of this paper is to illustrate the need for outcome-specific criteria in study evaluation or risk of bias evaluation, describe the process we used to develop the criteria, and summarize the resulting criteria.MethodsWe used a process of expert consultation to develop several sets of outcome-specific criteria to guide study reviewers, improve consistency, and ensure consideration of critical issues specific to the outcomes. The criteria were developed using the following domains: outcome assessment, exposure measurement (specifically timing of exposure in relation to outcome; other exposure measurement issues would be addressed in exposure-specific criteria), participant selection, confounding, analysis, and sensitivity (the study's ability to detect a true effect or hazard).ResultsWe discuss the application of this process to pregnancy-related outcomes (preterm birth, spontaneous abortion), other reproductive-related outcomes (male reproductive hormones, sperm parameters, time to pregnancy, pubertal development), chronic disease (diabetes, insulin resistance), and acute or episodic conditions (asthma, allergies), and provide examples of the criteria developed. For each outcome the most influential methodological considerations are highlighted including biological sample collection and quality control, sensitivity and specificity of ascertainment tools, optimal timing for recruitment into the study (e.g., preconception, specific trimesters), the etiologically relevant window for exposure assessments, and important potential confounders.ConclusionsOutcome-specific criteria are an important part of a systematic review and will facilitate study evaluations by epidemiologists with experience in evaluating studies using systematic review methods who may not have extensive discipline-specific experience in the outcomes being reviewed.
| S-EPMC8522891 | biostudies-literature
Project description:BackgroundConfounder adjustment is critical for accurate causal inference in observational studies. However, the appropriateness of methods for confounder adjustment in studies investigating multiple risk factors, where the factors are not simply mutually confounded, is often overlooked. This study aims to summarise the methods for confounder adjustment and the related issues in studies investigating multiple risk factors.MethodsA methodological study was performed. We searched PubMed from January 2018 to March 2023 to identify cohort and case-control studies investigating multiple risk factors for three chronic diseases (cardiovascular disease, diabetes and dementia). Study selection and data extraction were conducted independently by two reviewers. The study objectives were grouped into two categories: widely exploring potential risk factors and examining specific risk factors. The methods for confounder adjustment were classified based on a summarisation of the included studies, identifying six categories: (1) each risk factor was adjusted for potential confounders separately (the recommended method); (2) all risk factors were mutually adjusted (i.e. including all factors in a multivariable model); (3) all risk factors were adjusted for the same confounders separately; (4) all risk factors were adjusted for the same confounders with some factors being mutually adjusted; (5) all risk factors were adjusted for the same confounders with mutual adjustment among them being unclear; and (6) unable to judge. All data were descriptively analysed.ResultsA total of 162 studies were included, with 88 (54.3%) exploring potential risk factors and 74 (45.7%) examining specific risk factors. The current status of confounder adjustment was unsatisfactory: only ten studies (6.2%) used the recommended method, all of which aimed at examining several specific risk factors; in contrast, mutual adjustment was adopted in over 70% of the studies. The remaining studies either adjusted for the same confounders across all risk factors, or unable to judge.ConclusionsThere is substantial variation in the methods for confounder adjustment among studies investigating multiple risk factors. Mutual adjustment was the most commonly adopted method, which might lead to overadjustment bias and misleading effect estimates. Future research should avoid indiscriminately including all risk factors in a multivariable model to prevent inappropriate adjustment.
| S-EPMC11881322 | biostudies-literature
Project description:BackgroundThe overall confidence in the results of systematic reviews including animal models can be heterogeneous. We assessed the methodological quality of systematic reviews including animal models in dentistry as well as the overall confidence in the results of those systematic reviews.Material & methodsPubMed, Web of Science and Scopus were searched for systematic reviews including animal studies in dentistry published later than January 2010 until 18th of July 2022. Overall confidence in the results was assessed using a modified version of the A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2) checklist. Checklist items were rated as yes, partial yes, no and not applicable. Linear regression analysis was used to investigate associations between systematic review characteristics and the overall adherence to the AMSTAR-2 checklist. The overall confidence in the results was calculated based on the number of critical and non-critical weaknesses presented in the AMSTAR-2 items and rated as high, moderate, low and critical low.ResultsOf initially 951 retrieved systematic reviews, 190 were included in the study. The overall confidence in the results was low in 43 (22.6%) and critically low in 133 (70.0%) systematic reviews. While some AMSTAR-2 items were regularly reported (e.g. conflict of interest, selection in duplicate), others were not (e.g.Fundingn = 1; 0.5%). Multivariable linear regression analysis showed that the adherence scores of AMSTAR-2 was significantly associated with publication year, journal impact factor (IF), topic, and the use of tools to assess risk of bias (RoB) of the systematic reviews.ConclusionAlthough the methodological quality of dental systematic reviews of animal models improved over the years, it is still suboptimal. The overall confidence in the results was mostly low or critically low. Systematic reviews, which were published later, published in a journal with a higher IF, focused on non-surgery topics, and used at least one tool to assess RoB correlated with greater adherence to the AMSTAR-2 guidelines.
| S-EPMC10720166 | biostudies-literature
Project description:BackgroundMachine learning radiomics models are increasingly being used to predict gastric cancer prognoses. However, the methodological quality of these models has not been evaluated. Therefore, this study aimed to evaluate the methodological quality of radiomics studies in predicting the prognosis of gastric cancer, summarize their methodological characteristics and performance.MethodsThe PubMed and Embase databases were searched for radiomics studies used to predict the prognosis of gastric cancer published in last 5 years. The characteristics of the studies and the performance of the models were extracted from the eligible full texts. The methodological quality, reporting completeness and risk of bias of the included studies were evaluated using the RQS, TRIPOD and PROBAST. The discrimination ability scores of the models were also compared.ResultsOut of 283 identified records, 22 studies met the inclusion criteria. The study endpoints included survival time, treatment response, and recurrence, with reported discriminations ranging between 0.610 and 0.878 in the validation dataset. The mean overall RQS value was 15.32 ± 3.20 (range: 9 to 21). The mean adhered items of the 35 item of TRIPOD checklist was 20.45 ± 1.83. The PROBAST showed all included studies were at high risk of bias.ConclusionThe current methodological quality of gastric cancer radiomics studies is insufficient. Large and reasonable sample, prospective, multicenter and rigorously designed studies are required to improve the quality of radiomics models for gastric cancer prediction.Study registrationThis protocol was prospectively registered in the Open Science Framework Registry (https://osf.io/ja52b).
| S-EPMC10507631 | biostudies-literature
Project description:IntroductionMethodological studies (ie, studies that evaluate the design, conduct, analysis or reporting of other studies in health research) address various facets of health research including, for instance, data collection techniques, differences in approaches to analyses, reporting quality, adherence to guidelines or publication bias. As a result, methodological studies can help to identify knowledge gaps in the methodology of health research and strategies for improvement in research practices. Differences in methodological study names and a lack of reporting guidance contribute to lack of comparability across studies and difficulties in identifying relevant previous methodological studies. This paper outlines the methods we will use to develop an evidence-based tool-the MethodologIcal STudy reportIng Checklist-to harmonise naming conventions and improve the reporting of methodological studies.Methods and analysisWe will search for methodological studies in the Cumulative Index to Nursing and Allied Health Literature, Cochrane Library, Embase, MEDLINE, Web of Science, check reference lists and contact experts in the field. We will extract and summarise data on the study names, design and reporting features of the included methodological studies. Consensus on study terms and recommended reporting items will be achieved via video conference meetings with a panel of experts including researchers who have published methodological studies.Ethics and disseminationThe consensus study has been exempt from ethics review by the Hamilton Integrated Research Ethics Board. The results of the review and the reporting guideline will be disseminated in stakeholder meetings, conferences, peer-reviewed publications, in requests to journal editors (to endorse or make the guideline a requirement for authors), and on the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network and reporting guideline websites.RegistrationWe have registered the development of the reporting guideline with the EQUATOR Network and publicly posted this project on the Open Science Framework (www.osf.io/9hgbq).
| S-EPMC7747548 | biostudies-literature
Project description:ObjectivesTo assess the methodological quality of prognostic model development studies pertaining to post resection prognosis of pancreatic ductal adenocarcinoma (PDAC).Design/settingA narrative systematic review of international peer reviewed journals DATA SOURCE: Searches were conducted of: MEDLINE, Embase, PubMed, Cochrane database and Google Scholar for predictive modelling studies applied to the outcome of prognosis for patients with PDAC post resection. Predictive modelling studies in this context included prediction model development studies with and without external validation and external validation studies with model updating. Data was extracted following the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) checklist.Primary and secondary outcome measuresPrimary outcomes were all components of the CHARMS checklist. Secondary outcomes included frequency of variables included across predictive models.Results263 studies underwent full text review. 15 studies met the inclusion criteria. 3 studies underwent external validation. Multivariable Cox proportional hazard regression was the most commonly employed modelling method (n=13). 10 studies were based on single centre databases. Five used prospective databases, seven used retrospective databases and three used cancer data registry. The mean number of candidate predictors was 19.47 (range 7 to 50). The most commonly included variables were tumour grade (n=9), age (n=8), tumour stage (n=7) and tumour size (n=5). Mean sample size was 1367 (range 50 to 6400). 5 studies reached statistical power. None of the studies reported blinding of outcome measurement for predictor values. The most common form of presentation was nomograms (n=5) and prognostic scores (n=5) followed by prognostic calculators (n=3) and prognostic index (n=2).ConclusionsAreas for improvement in future predictive model development have been highlighted relating to: general aspects of model development and reporting, applicability of models and sources of bias.Trial registration numberCRD42018105942.
| S-EPMC6707674 | biostudies-literature
Project description:Multivariate matching has two goals: (i) to construct treated and control groups that have similar distributions of observed covariates, and (ii) to produce matched pairs or sets that are homogeneous in a few key covariates. When there are only a few binary covariates, both goals may be achieved by matching exactly for these few covariates. Commonly, however, there are many covariates, so goals (i) and (ii) come apart, and must be achieved by different means. As is also true in a randomized experiment, similar distributions can be achieved for a high-dimensional covariate, but close pairs can be achieved for only a few covariates. We introduce a new polynomial-time method for achieving both goals that substantially generalizes several existing methods; in particular, it can minimize the earthmover distance between two marginal distributions. The method involves minimum cost flow optimization in a network built around a tripartite graph, unlike the usual network built around a bipartite graph. In the tripartite graph, treated subjects appear twice, on the far left and the far right, with controls sandwiched between them, and efforts to balance covariates are represented on the right, while efforts to find close individual pairs are represented on the left. In this way, the two efforts may be pursued simultaneously without conflict. The method is applied to our on-going study in the Medicare population of the relationship between superior nursing and sepsis mortality. The match2C package in R implements the method.
| S-EPMC10281706 | biostudies-literature
Project description:This systematic review assessed the methodological quality of behavioural weight loss intervention studies conducted among adults and associations between quality and statistically significant weight loss outcome, strength of intervention effectiveness and sample size. Searches for trials published between January, 2009 and December, 2014 were conducted using PUBMED, MEDLINE and PSYCINFO and identified ninety studies. Methodological quality indicators included study design, anthropometric measurement approach, sample size calculations, intent-to-treat (ITT) analysis, loss to follow-up rate, missing data strategy, sampling strategy, report of treatment receipt and report of intervention fidelity (mean = 6.3). Indicators most commonly utilized included randomized design (100%), objectively measured anthropometrics (96.7%), ITT analysis (86.7%) and reporting treatment adherence (76.7%). Most studies (62.2%) had a follow-up rate > 75% and reported a loss to follow-up analytic strategy or minimal missing data (69.9%). Describing intervention fidelity (34.4%) and sampling from a known population (41.1%) were least common. Methodological quality was not associated with reporting a statistically significant result, effect size or sample size. This review found the published literature of behavioural weight loss trials to be of high quality for specific indicators, including study design and measurement. Identified for improvement include utilization of more rigorous statistical approaches to loss to follow up and better fidelity reporting.
| S-EPMC5996374 | biostudies-literature
Project description:BackgroundMost retractions of obstetrics and gynecology manuscripts are because of scientific misconduct. It would be preferable to prevent randomized controlled trials with scientific misconduct from ever appearing in the peer-reviewed scientific literature, rather than to have to retract them later.ObjectiveThis study aimed to evaluate the policies of obstetrics and gynecology and top medical journals in their author guidelines and electronic submission systems regarding prospective randomized controlled trial registration, ethics committee approval, research protocols, Consolidated Standards of Reporting Trial guidelines, and data sharing and to detect the most common quality criteria requested for randomized controlled trials in these journals.Study designAuthor guidelines were identified via online Google searches from the websites of selected peer-reviewed medical journals. Journals in obstetrics and gynecology were selected from the list of journals with impact factors based on the Journal Citation Report released by Clarivate Analytics on June 29, 2020, focusing on those publishing original clinical research in obstetrics, in particular randomized controlled trials. In addition, 4 of the top impact factor peer-reviewed general medical journals publishing randomized controlled trials were included. The requirements for selected quality criteria for randomized controlled trials analyzed in the author guidelines for each journal were details of 5 general issues: prospective randomized controlled trial registration (4 subcategories), ethics committee approval (4 subcategories), research protocol (3 subcategories), Consolidated Standards of Reporting Trials guidelines (3 subcategories), and data sharing (3 subcategories). To evaluate the requirements within the electronic submission system, a mock submission of a randomized controlled trial was also done for each journal, and the same criteria were assessed on the online software for submission. The primary outcome was the overall percentage for each of the quality criteria that were listed as required within the author guidelines or required in the submission system among all journals. Planned subgroup analyses were top general medicine vs obstetrics and gynecology journals and top 4 obstetrics and gynecology vs other obstetrics and gynecology journals.ResultsMost studied peer-reviewed journals listed in their author guidelines 7 specific criteria for submission of randomized controlled trials: prospective registration and registration number, statement of ethical approval with name of approving committee and statement of informed consent, statement of adherence to Consolidated Standards of Reporting Trials guidelines, and data sharing statement. For most journals, the submission software did not require these or any other criteria for submission. There were minimal differences in criteria listed for top medical journals vs other obstetrics and gynecology journals and among top vs other obstetrics and gynecology journals.ConclusionProspective registration and registration number, statement of ethical approval with name of approving committee and statement of informed consent, statement of adherence to Consolidated Standards of Reporting Trials guidelines, and data sharing statement are the randomized controlled trial quality criteria requested by leading medical and obstetrics and gynecology journals. These obstetrics and gynecology journals agree to make, as much as possible, these criteria uniform and mandatory in author guidelines and also through improved submission software.
| S-EPMC8324065 | biostudies-literature