Dataset Information

Updating standards for reporting diagnostic accuracy: the development of STARD 2015.

ABSTRACT: Background:Although the number of reporting guidelines has grown rapidly, few have gone through an updating process. The STARD statement (Standards for Reporting Diagnostic Accuracy), published in 2003 to help improve the transparency and completeness of reporting of diagnostic accuracy studies, was recently updated in a systematic way. Here, we describe the steps taken and a justification for the changes made. Results:A 4-member Project Team coordinated the updating process; a 14-member Steering Committee was regularly solicited by the Project Team when making critical decisions. First, a review of the literature was performed to identify topics and items potentially relevant to the STARD updating process. After this, the 85 members of the STARD Group were invited to participate in two online surveys to identify items that needed to be modified, removed from, or added to the STARD checklist. Based on the results of the literature review process, 33 items were presented to the STARD Group in the online survey: 25 original items and 8 new items; 73 STARD Group members (86 %) completed the first survey, and 79 STARD Group members (93 %) completed the second survey.Then, an in-person consensus meeting was organized among the members of the Project Team and Steering Committee to develop a consensual draft version of STARD 2015. This version was piloted in three rounds among a total of 32 expert and non-expert users. Piloting mostly led to rewording of items. After this, the update was finalized. The updated STARD 2015 list now consists of 30 items. Compared to the previous version of STARD, three original items were each converted into two new items, four original items were incorporated into other items, and seven new items were added. Conclusions:After a systematic updating process, STARD 2015 provides an updated list of 30 essential items for reporting diagnostic accuracy studies.

SUBMITTER: Korevaar DA

PROVIDER: S-EPMC5803584 | biostudies-literature | 2016

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Updating standards for reporting diagnostic accuracy: the development of STARD 2015.

Korevaar Daniël A DA Cohen Jérémie F JF Reitsma Johannes B JB Bruns David E DE Gatsonis Constantine A CA Glasziou Paul P PP Irwig Les L Moher David D de Vet Henrica C W HCW Altman Douglas G DG Hooft Lotty L Bossuyt Patrick M M PMM

Research integrity and peer review 20160607

<h4>Background</h4>Although the number of reporting guidelines has grown rapidly, few have gone through an updating process. The STARD statement (Standards for Reporting Diagnostic Accuracy), published in 2003 to help improve the transparency and completeness of reporting of diagnostic accuracy studies, was recently updated in a systematic way. Here, we describe the steps taken and a justification for the changes made.<h4>Results</h4>A 4-member Project Team coordinated the updating process; a 14 ...[more]

PMID: 29451535

Similar Datasets

Project description:Research has shown a modest adherence of diagnostic test accuracy (DTA) studies in glaucoma to the Standards for Reporting of Diagnostic Accuracy Studies (STARD). We have applied the updated 30-item STARD 2015 checklist to a set of studies included in a Cochrane DTA systematic review of imaging tools for diagnosing manifest glaucoma.Three pairs of reviewers, including one senior reviewer who assessed all studies, independently checked the adherence of each study to STARD 2015. Adherence was analyzed on an individual-item basis. Logistic regression was used to evaluate the effect of publication year and impact factor on adherence.We included 106 DTA studies, published between 2003-2014 in journals with a median impact factor of 2.6. Overall adherence was 54.1% for 3,286 individual rating across 31 items, with a mean of 16.8 (SD: 3.1; range 8-23) items per study. Large variability in adherence to reporting standards was detected across individual STARD 2015 items, ranging from 0 to 100%. Nine items (1: identification as diagnostic accuracy study in title/abstract; 6: eligibility criteria; 10: index test (a) and reference standard (b) definition; 12: cut-off definitions for index test (a) and reference standard (b); 14: estimation of diagnostic accuracy measures; 21a: severity spectrum of diseased; 23: cross-tabulation of the index and reference standard results) were adequately reported in more than 90% of the studies. Conversely, 10 items (3: scientific and clinical background of the index test; 11: rationale for the reference standard; 13b: blinding of index test results; 17: analyses of variability; 18; sample size calculation; 19: study flow diagram; 20: baseline characteristics of participants; 28: registration number and registry; 29: availability of study protocol; 30: sources of funding) were adequately reported in less than 30% of the studies. Only four items showed a statistically significant improvement over time: missing data (16), baseline characteristics of participants (20), estimates of diagnostic accuracy (24) and sources of funding (30).Adherence to STARD 2015 among DTA studies in glaucoma research is incomplete, and only modestly increasing over time.

Project description:Importance:Incomplete reporting of diagnostic accuracy research impairs assessment of risk of bias and limits generalizability. Point-of-care ultrasound has become an important diagnostic tool for acute care physicians, but studies assessing its use are of varying methodological quality. Objective:To assess adherence to the Standards for Reporting of Diagnostic Accuracy (STARD) 2015 guidelines in the literature on acute care point-of-care ultrasound. Evidence Review:MEDLINE was searched to identify diagnostic accuracy studies assessing point-of-care ultrasound published in critical care, emergency medicine, or anesthesia journals from 2016 to 2019. Studies were evaluated for adherence to the STARD 2015 guidelines, with the following variables analyzed: journal, country, STARD citation, STARD-adopting journal, impact factor, patient population, use of supplemental material, and body region. Data analysis was performed in November 2019. Findings:Seventy-four studies were included in this systematic review for assessment. Overall adherence to STARD was moderate, with 66% (mean [SD], 19.7 [2.9] of 30 items) of STARD items reported. Items pertaining to imaging specifications, patient population, and readers of the index test were frequently reported (>66% of studies). Items pertaining to blinding of readers to clinical data and to the index or reference standard, analysis of heterogeneity, indeterminate and missing data, and time intervals between index and reference test were either moderately (33%-66%) or infrequently (<33%) reported. Studies in STARD-adopting journals (mean [SD], 20.5 [2.9] items in adopting journals vs 18.6 [2.3] items in nonadopting journals; P?=?.002) and studies citing STARD (mean [SD], 21.3 [0.9] items in citing studies vs 19.5 [2.9] items in nonciting studies; P?=?.01) reported more items. Variation by country and journal of publication were identified. No differences in STARD adherence were identified by body region imaged (mean [SD], abdominal, 20.0 [2.5] items; head and neck, 17.8 [1.6] items; musculoskeletal, 19.2 [3.1] items; thoracic, 20.2 [2.8] items; and other or procedural, 19.8 [2.7] items; P?=?.29), study design (mean [SD], prospective, 19.7 [2.9] items; retrospective, 19.7 [1.8] items; P?>?.99), patient population (mean [SD], pediatric, 20.0 [3.1] items; adult, 20.2 [2.7] items; mixed, 17.9 [1.9] items; P?=?.09), use of supplementary materials (mean [SD], yes, 19.2 [3.0] items; no, 19.7 [2.8] items; P?=?.91), or journal impact factor (mean [SD], higher impact factor, 20.3 [3.1] items; lower impact factor, 19.1 [2.4] items; P?=?.08). Conclusions and Relevance:Overall, the literature on acute care point-of-care ultrasound showed moderate adherence to the STARD 2015 guidelines, with more complete reporting found in studies citing STARD and those published in STARD-adopting journals. This study has established a current baseline for reporting; however, future studies are required to understand barriers to complete reporting and to develop strategies to mitigate them.

Project description:IntroductionStandards for Reporting of Diagnostic Accuracy Study (STARD) was developed to improve the completeness and transparency of reporting in studies investigating diagnostic test accuracy. However, its current form, STARD 2015 does not address the issues and challenges raised by artificial intelligence (AI)-centred interventions. As such, we propose an AI-specific version of the STARD checklist (STARD-AI), which focuses on the reporting of AI diagnostic test accuracy studies. This paper describes the methods that will be used to develop STARD-AI.Methods and analysisThe development of the STARD-AI checklist can be distilled into six stages. (1) A project organisation phase has been undertaken, during which a Project Team and a Steering Committee were established; (2) An item generation process has been completed following a literature review, a patient and public involvement and engagement exercise and an online scoping survey of international experts; (3) A three-round modified Delphi consensus methodology is underway, which will culminate in a teleconference consensus meeting of experts; (4) Thereafter, the Project Team will draft the initial STARD-AI checklist and the accompanying documents; (5) A piloting phase among expert users will be undertaken to identify items which are either unclear or missing. This process, consisting of surveys and semistructured interviews, will contribute towards the explanation and elaboration document and (6) On finalisation of the manuscripts, the group's efforts turn towards an organised dissemination and implementation strategy to maximise end-user adoption.Ethics and disseminationEthical approval has been granted by the Joint Research Compliance Office at Imperial College London (reference number: 19IC5679). A dissemination strategy will be aimed towards five groups of stakeholders: (1) academia, (2) policy, (3) guidelines and regulation, (4) industry and (5) public and non-specific stakeholders. We anticipate that dissemination will take place in Q3 of 2021.

Project description:BackgroundIn January 2003, STAndards for the Reporting of Diagnostic accuracy studies (STARD) were published in a number of journals, to improve the quality of reporting in diagnostic accuracy studies. We designed a study to investigate the inter-assessment reproducibility, and intra- and inter-observer reproducibility of the items in the STARD statement.MethodsThirty-two diagnostic accuracy studies published in 2000 in medical journals with an impact factor of at least 4 were included. Two reviewers independently evaluated the quality of reporting of these studies using the 25 items of the STARD statement. A consensus evaluation was obtained by discussing and resolving disagreements between reviewers. Almost two years later, the same studies were evaluated by the same reviewers. For each item, percentages agreement and Cohen's kappa between first and second consensus assessments (inter-assessment) were calculated. Intraclass Correlation coefficients (ICC) were calculated to evaluate its reliability.ResultsThe overall inter-assessment agreement for all items of the STARD statement was 85% (Cohen's kappa 0.70) and varied from 63% to 100% for individual items. The largest differences between the two assessments were found for the reporting of the rationale of the reference standard (kappa 0.37), number of included participants that underwent tests (kappa 0.28), distribution of the severity of the disease (kappa 0.23), a cross tabulation of the results of the index test by the results of the reference standard (kappa 0.33) and how indeterminate results, missing data and outliers were handled (kappa 0.25). Within and between reviewers, also large differences were observed for these items. The inter-assessment reliability of the STARD checklist was satisfactory (ICC = 0.79 [95% CI: 0.62 to 0.89]).ConclusionAlthough the overall reproducibility of the quality of reporting on diagnostic accuracy studies using the STARD statement was found to be good, substantial disagreements were found for specific items. These disagreements were not so much caused by differences in interpretation of the items by the reviewers but rather by difficulties in assessing the reporting of these items due to lack of clarity within the articles. Including a flow diagram in all reports on diagnostic accuracy studies would be very helpful in reducing confusion between readers and among reviewers.

Project description:BackgroundSurgery for gliomas is often confounded by difficulties in distinguishing tumor from surrounding normal brain. For better discrimination, intraoperative optical imaging methods using fluorescent dyes are currently being explored. Understandably, such methods require the demonstration of a high degree of diagnostic accuracy and clinical benefit. Currently, clinical utility is determined by tissue biopsies which are correlated to optical signals, and quantified using measures such as sensitivity, specificity, positive predictive values, and negative predictive values. In addition, surgical outcomes, such as extent of resection rates and/or survival (progression-free survival (PFS) and overall survival (OS)) have been measured. These assessments, however, potentially involve multiple biases and confounders, which have to be minimized to ensure reproducibility, generalizability and comparability of test results. Test should aim at having a high internal and external validity. The objective of this article is to analyze how diagnostic accuracy and outcomes are utilized in available studies describing intraoperative imaging and furthermore, to derive recommendations for reliable and reproducible evaluations.MethodsA review of the literature was performed for assessing the use of measures of diagnostic accuracy and outcomes of intraoperative optical imaging methods. From these data, we derive recommendations for designing and reporting future studies.ResultsAvailable literature indicates that potential confounders and biases for reporting the diagnostic accuracy and usefulness of intraoperative optical imaging methods are seldom accounted for. Furthermore, methods for bias reduction are rarely used nor reported.ConclusionsDetailed, transparent, and uniform reporting on diagnostic accuracy of intraoperative imaging methods is necessary. In the absence of such reporting, studies will not be comparable or reproducible. Future studies should consider some of the recommendations given here.

Dataset Information

Updating standards for reporting diagnostic accuracy: the development of STARD 2015.

Publications

Updating standards for reporting diagnostic accuracy: the development of STARD 2015.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets