Measurement properties of quality assessment tools for studies of diagnostic accuracy.
Ontology highlight
ABSTRACT:
SUBMITTER: Kaizik MA
PROVIDER: S-EPMC7082671 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundPoor reporting quality in diagnostic accuracy studies hampers an adequate judgment of the validity of the study. The Standards for Reporting of Diagnostic Accuracy Studies (STARD) statement was published to improve the reporting quality of diagnostic accuracy studies. This study aimed to evaluate the adherence of diagnostic accuracy studies published in Annals of Laboratory Medicine (ALM) to STARD 2015 and to identify directions for improvement in the reporting quality of these studies.MethodsTwo independent authors assessed articles published in ALM between 2012-2018 for compliance with 30 STARD 2015 checklist items to identify all eligible diagnostic accuracy studies published during this period. We included 66 diagnostic accuracy studies. A total of the fulfilled STARD items were calculated, and adherence was analyzed on an individual-item basis.ResultsThe overall mean±SD number of STARD items reported for the included studies was 11.2±2.7. Only five (7.6%) studies adhered to more than 50% of the 30 items. No study satisfied more than 80% of the items. Large variability in adherence to reporting standards was detected across items, ranging from 0% to 100%.ConclusionsAdherence to STARD 2015 is suboptimal among diagnostic accuracy studies published in ALM. Our study emphasizes the necessity of adherence to STARD to improve the reporting quality of future diagnostic accuracy studies to be published in ALM.
| S-EPMC6933069 | biostudies-literature
Project description:INTRODUCTION:Recanalisation therapy in acute ischaemic stroke is highly time-sensitive, and requires early identification of eligible patients to ensure better outcomes. Thus, a number of clinical assessment tools have been developed and this review examines their diagnostic capabilities. METHODS:Diagnostic performance of currently available clinical tools for identification of acute ischaemic and haemorrhagic strokes and stroke mimicking conditions was reviewed. A systematic search of the literature published in 2015-2018 was conducted using PubMed, EMBASE, Scopus and The Cochrane Library. Prehospital and in-hospital studies with a minimum sample size of 300 patients reporting diagnostic accuracy were selected. RESULTS:Twenty-five articles were included. Cortical signs (gaze deviation, aphasia and neglect) were shown to be significant indicators of large vessel occlusion (LVO). Sensitivity values for selecting subjects with LVO ranged from 23 to 99% whereas specificity was 24 to 97%. Clinical tools, such as FAST-ED, NIHSS, and RACE incorporating cortical signs as well as motor dysfunction demonstrated the best diagnostic accuracy. Tools for identification of stroke mimics showed sensitivity varying from 44 to 91%, and specificity of 27 to 98% with the best diagnostic performance demonstrated by FABS (90% sensitivity, 91% specificity). Hypertension and younger age predicted intracerebral haemorrhage whereas history of atrial fibrillation and diabetes were associated with ischaemia. There was a variation in approach used to establish the definitive diagnosis. Blinding of the index test assessment was not specified in about 50% of included studies. CONCLUSIONS:A wide range of clinical assessment tools for selecting subjects with acute stroke has been developed in recent years. Assessment of both cortical and motor function using RACE, FAST-ED and NIHSS showed the best diagnostic accuracy values for selecting subjects with LVO. There were limited data on clinical tools that can be used to differentiate between acute ischaemia and haemorrhage. Diagnostic accuracy appeared to be modest for distinguishing between acute stroke and stroke mimics with optimal diagnostic performance demonstrated by the FABS tool. Further prehospital research is required to improve the diagnostic utility of clinical assessments with possible application of a two-step clinical assessment or involvement of simple brain imaging, such as transcranial ultrasonography.
| S-EPMC6727516 | biostudies-literature
Project description:ObjectiveMachine learning (ML) diagnostic tools have significant potential to improve health care. However, methodological pitfalls may affect diagnostic test accuracy studies used to appraise such tools. We aimed to evaluate the prevalence and reporting of design characteristics within the literature. Further, we sought to empirically assess whether design features may be associated with different estimates of diagnostic accuracy.Materials and methodsWe systematically retrieved 2 × 2 tables (n = 281) describing the performance of ML diagnostic tools, derived from 114 publications in 38 meta-analyses, from PubMed. Data extracted included test performance, sample sizes, and design features. A mixed-effects metaregression was run to quantify the association between design features and diagnostic accuracy.ResultsParticipant ethnicity and blinding in test interpretation was unreported in 90% and 60% of studies, respectively. Reporting was occasionally lacking for rudimentary characteristics such as study design (28% unreported). Internal validation without appropriate safeguards was used in 44% of studies. Several design features were associated with larger estimates of accuracy, including having unreported (relative diagnostic odds ratio [RDOR], 2.11; 95% confidence interval [CI], 1.43-3.1) or case-control study designs (RDOR, 1.27; 95% CI, 0.97-1.66), and recruiting participants for the index test (RDOR, 1.67; 95% CI, 1.08-2.59).DiscussionSignificant underreporting of experimental details was present. Study design features may affect estimates of diagnostic performance in the ML diagnostic test accuracy literature.ConclusionsThe present study identifies pitfalls that threaten the validity, generalizability, and clinical value of ML diagnostic tools and provides recommendations for improvement.
| S-EPMC7647361 | biostudies-literature
Project description:BackgroundIn the era of evidence based medicine, with systematic reviews as its cornerstone, adequate quality assessment tools should be available. There is currently a lack of a systematically developed and evaluated tool for the assessment of diagnostic accuracy studies. The aim of this project was to combine empirical evidence and expert opinion in a formal consensus method to develop a tool to be used in systematic reviews to assess the quality of primary studies of diagnostic accuracy.MethodsWe conducted a Delphi procedure to develop the quality assessment tool by refining an initial list of items. Members of the Delphi panel were experts in the area of diagnostic research. The results of three previously conducted reviews of the diagnostic literature were used to generate a list of potential items for inclusion in the tool and to provide an evidence base upon which to develop the tool.ResultsA total of nine experts in the field of diagnostics took part in the Delphi procedure. The Delphi procedure consisted of four rounds, after which agreement was reached on the items to be included in the tool which we have called QUADAS. The initial list of 28 items was reduced to fourteen items in the final tool. Items included covered patient spectrum, reference standard, disease progression bias, verification bias, review bias, clinical review bias, incorporation bias, test execution, study withdrawals, and indeterminate results. The QUADAS tool is presented together with guidelines for scoring each of the items included in the tool.ConclusionsThis project has produced an evidence based quality assessment tool to be used in systematic reviews of diagnostic accuracy studies. Further work to determine the usability and validity of the tool continues.
| S-EPMC305345 | biostudies-literature
Project description:BackgroundFrailty is associated with multiple adverse outcomes in stage-5 chronic kidney disease (CKD-5) and upwards of one third of people receiving haemodialysis (HD) are frail. While many frailty screening methods are available in both uremic and non-uremic populations, their implementation in clinical settings is often challenged by time and resource constraints. In this study, we explored the diagnostic accuracy of time-efficient screening tools in people receiving HD.MethodsA convenience sample of 76 people receiving HD [mean age = 61.1 years (SD = 14), 53.9% male] from three Renal Units were recruited for this cross-sectional study. Frailty was diagnosed by means of the Fried phenotype. Physical performance-based screening tools encompassed handgrip strength, 15-ft gait speed, timed up and go (TUG), and five-repetition sit to stand (STS-5) tests. In addition, participants completed the SF-36 Health Survey, the short-form international physical activity questionnaire and the Tinetti falls efficacy scale (FES) as further frailty-related measures. Outcome measures included the area under the curve (AUC), sensitivity, specificity, positive (PPV) and negative predictive values (NPV). The diagnostic performance of screening tools in assessing fall-risk was also investigated.ResultsOverall, 36.8% of participants were classified as frail. All the examined instruments could significantly discriminate frailty status in the study population. Gait speed [AUC = 0.89 (95%CI: 0.81-0.98), sensitivity = 75%, specificity = 93%] and TUG [AUC = 0.90 (95%CI: 0.80-0.99), sensitivity = 89%, specificity = 85%] exhibited the highest diagnostic accuracy. There was a significant difference in gait speed AUC (20%, p = 0.013) between participants aged 65 years or older (n = 36) and those under 65 years of age (n = 40), with better discriminating performance in the younger sub-group. The Tinetti FES was the only instrument showing good diagnostic accuracy (AUCs≥0.80) for both frailty (sensitivity = 82%, specificity = 79%) and fall-risk (sensitivity = 82%, specificity = 71%) screening.ConclusionsThis cross-sectional study revealed that time- and cost-efficient walking performance measures can accurately be used for frailty-screening purposes in people receiving HD. While self-selected gait speed had an excellent performance in people under 65 years of age, TUG may be a more suitable screening method for elderly patients (≥65 years). The Tinetti FES may be a clinically useful test when physical testing is not achievable.
| S-EPMC8252257 | biostudies-literature
Project description:Pharmacoepidemiological studies are an important hypothesis-testing tool in the evaluation of postmarketing drug safety. Despite the potential to produce robust value-added data, interpretation of findings can be hindered due to well-recognised methodological limitations of these studies. Therefore, assessment of their quality is essential to evaluating their credibility. The objective of this review was to evaluate the suitability and relevance of available tools for the assessment of pharmacoepidemiological safety studies. We created an a priori assessment framework consisting of reporting elements (REs) and quality assessment attributes (QAAs). A comprehensive literature search identified distinct assessment tools and the prespecified elements and attributes were evaluated. The primary outcome measure was the percentage representation of each domain, RE and QAA for the quality assessment tools. A total of 61 tools were reviewed. Most tools were not designed to evaluate pharmacoepidemiological safety studies. More than 50% of the reviewed tools considered REs under the research aims, analytical approach, outcome definition and ascertainment, study population and exposure definition and ascertainment domains. REs under the discussion and interpretation, results and study team domains were considered in less than 40% of the tools. Except for the data source domain, quality attributes were considered in less than 50% of the tools. Many tools failed to include critical assessment elements relevant to observational pharmacoepidemiological safety studies and did not distinguish between REs and QAAs. Further, there is a lack of considerations on the relative weights of different domains and elements. The development of a quality assessment tool would facilitate consistent, objective and evidence-based assessments of pharmacoepidemiological safety studies.
| S-EPMC3467649 | biostudies-literature
Project description:IntroductionUsing specific tools to assess the measurement properties of health status instruments is recommended both to standardise the review process and to improve the methodological quality of systematic reviews. However, depending on the measurement standards on which these tools are developed, the approach to appraise the measurement properties of instruments may vary. For this reason, the present meta-review aims to: (1) identify systematic reviews assessing the measurement properties of instruments evaluating health-related quality of life (HRQoL); (2) identify the tools applied to assess the measurement properties of HRQoL instruments; (3) describe the characteristics of the tools applied to assess the measurement properties of HRQoL instruments; (4) identify the measurement standards on which these tools were developed or conform to and (5) compare the similarities and differences among the identified measurement standards.Methods and analysisA systematic review will be conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols Guidelines. Electronic search will be carried out on bibliographic databases, including PubMed, Cumulative Index to Nursing and Allied Health Literature, Psychological Information, SCOPUS, Web of Science, COSMIN database and ProQuest Dissertations & Theses Global, being limited by time (2008-2018) and language (English). Descriptive analyses of different aspects of tools applied to evaluate the measurement properties of HRQoL instruments will be presented; the different measurement standards will be described and some recommendations about the methodological and research applications will be made.Ethics and disseminationEthical approval is not necessary for systematic review protocols. The results will be disseminated by its publication in a peer-reviewed journal and presented at a relevant conference.Prospero registration numberCRD42017065232.
| S-EPMC6059301 | biostudies-literature
Project description:ObjectivesThis study aimed to evaluate the measurement properties and methodological quality of assessment tools for Kinesophobia among patients with cardiovascular disease and provide a reference for healthcare professionals in selecting high-quality assessment tools.MethodsA systematic search was performed on specific databases: Embase, the Cochrane Library, PubMed, Web of Science, China National Knowledge Infrastructure, Wanfang database, China Biological Medicine disc, CINAHL, and China Science and Technology Journal Database, from inception to April 1, 2023. The researchers retrieved studies on the measurement attributes of the exercise fear scale in patients with cardiovascular diseases. They also traced back the references of the included studies to supplement relevant literature. According to the inclusion and exclusion criteria, screening and data extraction were independently undertaken by two reviewers. Two researchers individually used the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) Risk of Bias Checklist to assess the methodological quality of the scale, applied the COSMIN criteria to evaluate the measurement properties of the scale, and used a modified Grading, Recommendations, Assessment, Development, and Evaluation system to assess the certainty of evidence.ResultsSeventeen studies were identified that reported the psychometric properties of six patient reported outcome measurement tools (included different languages version) The methodological quality of content validity was adequate in only two studies, the remaining patient-reported outcome measures demonstrated doubtful content validity. Limited information on cross-cultural validity/measurement invariance, measurement error, and responsiveness was retrieved. The Swedish version and the Chinese version of the Tampa Scale for Kinesiophobia Heart were graded "A." The remaining instruments were graded "B."ConclusionsThe methodological and measurement attributes of the Swedish and Chinese versions of the Tampa Scale for Kinesiophobia Heart are relatively high quality and can be tentatively recommended. The measurement properties of the remaining scales remain to be verified.
| S-EPMC10859577 | biostudies-literature
Project description:BackgroundThe use of malnutrition outcome measures (OM) by registered dietitians (RD) with inpatients in hospitals has increased promoting the achievement of nutritional care goals and supporting decision-making for the allocation of nutritional care resources in hospitals. There are 3 commonly used OMs: Subjective Global Assessment (SGA), Patient Generated-Subjective Global Assessment (PG-SGA) and Mini Nutritional Assessment (MNA). The purpose of this current study was to systematically review the evidence of the clinical measurement properties of malnutrition assessment tools for use with patients admitted in hospitals.MethodsMEDLINE, Cinahl, EMBASE, and PubMed were searched for articles published between 2000 and 2019. Research articles were selected if they established reliability, validity, and responsiveness to change properties of the SGA, PG-SGA and MNA tools, were written in English, and used any of these OMs as an outcome measure. Abstracts were not considered. The risk of bias within studies was assessed using the Quality Appraisal for Clinical Measurement Study (QA-CMS).ResultsFive hundred five studies were identified, of which 34 articles were included in the final review: SGA (n = 8), PG-SGA (n = 13), and MNA (n = 13). Of the 34 studies, 8 had a quality score greater than 75%; 23 had a quality score of 40-75% and 3 studies had a quality score of less than 40%. PG-SGA was found to have excellentdiagnostic accuracy (ROC: 0.92-0.975; Sensitivity: 88.6-98%; Specificity: 82-100%), sufficient internal consistency (Cronbach's alpha: 0.722-0.73), and strong test-retest reliability (r = 0.866). There was insufficient evidence to suggest adequate diagnostic accuracy and good inter-rater reliability for SGA. Only one study examined the minimum detectable change of MNA (MDC = 2.1).ConclusionsThe evidence of validity for the existing malnutrition assessment tools supports the use of these tools, but more studies with sound methodological quality are needed to assess the responsiveness of these OMs to detect the change in nutritional status.
| S-EPMC7507822 | biostudies-literature
Project description:Artificial intelligence (AI) centred diagnostic systems are increasingly recognised as robust solutions in healthcare delivery pathways. In turn, there has been a concurrent rise in secondary research studies regarding these technologies in order to influence key clinical and policymaking decisions. It is therefore essential that these studies accurately appraise methodological quality and risk of bias within shortlisted trials and reports. In order to assess whether this critical step is performed, we undertook a meta-research study evaluating adherence to the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool within AI diagnostic accuracy systematic reviews. A literature search was conducted on all studies published from 2000 to December 2020. Of 50 included reviews, 36 performed the quality assessment, of which 27 utilised the QUADAS-2 tool. Bias was reported across all four domains of QUADAS-2. Two hundred forty-three of 423 studies (57.5%) across all systematic reviews utilising QUADAS-2 reported a high or unclear risk of bias in the patient selection domain, 110 (26%) reported a high or unclear risk of bias in the index test domain, 121 (28.6%) in the reference standard domain and 157 (37.1%) in the flow and timing domain. This study demonstrates the incomplete uptake of quality assessment tools in reviews of AI-based diagnostic accuracy studies and highlights inconsistent reporting across all domains of quality assessment. Poor standards of reporting act as barriers to clinical implementation. The creation of an AI-specific extension for quality assessment tools of diagnostic accuracy AI studies may facilitate the safe translation of AI tools into clinical practice.
| S-EPMC8795185 | biostudies-literature