Dataset Information

Reporting of prognostic studies of tumour markers: a review of published articles in relation to REMARK guidelines.

ABSTRACT:

Background

Poor reporting compromises the reliability and clinical value of prognostic tumour marker studies. We review articles to assess the reporting of patients and events using REMARK guidelines, at the time of guideline publication.

Methods

We sampled 50 prognostic tumour marker studies from higher impact cancer journals between 2006 and 2007. The inclusion criteria were cancer; focus on single biological tumour marker; survival analysis; multivariable analysis; and not gene array or proteomic data. Articles were assessed for the REMARK profile and other REMARK guideline items. We propose a reporting aid, the REMARK profile, motivated by the CONSORT flowchart.

Results

In 50 studies assessed for the REMARK profile, the number of eligible patients (56% of articles), excluded patients (54%) and patients in analyses (98%) was reported. Only 50% of articles reported the number of outcome events. In multivariable analyses, 54% and 30% of articles reported patient and event numbers for all variables. Of the studies, 66% used archival samples, indicating a potentially biased patient selection. Only 36% of studies reported clearly defined outcomes.

Conclusions

Good reporting is critical for the interpretability and clinical applicability of prognostic studies. Current reporting of key information, such as the number of outcome events in all patients and subgroups, is poor. Use of the REMARK profile would greatly improve reporting and enhance prognostic research.

SUBMITTER: Mallett S

PROVIDER: S-EPMC2795163 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThe reporting of machine learning (ML) prognostic and diagnostic modeling studies is often inadequate, making it difficult to understand and replicate such studies. To address this issue, multiple consensus and expert reporting guidelines for ML studies have been published. However, these guidelines cover different parts of the analytics lifecycle, and individually, none of them provide a complete set of reporting requirements.ObjectiveWe aimed to consolidate the ML reporting guidelines and checklists in the literature to provide reporting items for prognostic and diagnostic ML in in-silico and shadow mode studies.MethodsWe conducted a literature search that identified 192 unique peer-reviewed English articles that provide guidance and checklists for reporting ML studies. The articles were screened by their title and abstract against a set of 9 inclusion and exclusion criteria. Articles that were filtered through had their quality evaluated by 2 raters using a 9-point checklist constructed from guideline development good practices. The average κ was 0.71 across all quality criteria. The resulting 17 high-quality source papers were defined as having a quality score equal to or higher than the median. The reporting items in these 17 articles were consolidated and screened against a set of 6 inclusion and exclusion criteria. The resulting reporting items were sent to an external group of 11 ML experts for review and updated accordingly. The updated checklist was used to assess the reporting in 6 recent modeling papers in JMIR AI. Feedback from the external review and initial validation efforts was used to improve the reporting items.ResultsIn total, 37 reporting items were identified and grouped into 5 categories based on the stage of the ML project: defining the study details, defining and collecting the data, modeling methodology, model evaluation, and explainability. None of the 17 source articles covered all the reporting items. The study details and data description reporting items were the most common in the source literature, with explainability and methodology guidance (ie, data preparation and model training) having the least coverage. For instance, a median of 75% of the data description reporting items appeared in each of the 17 high-quality source guidelines, but only a median of 33% of the data explainability reporting items appeared. The highest-quality source articles tended to have more items on reporting study details. Other categories of reporting items were not related to the source article quality. We converted the reporting items into a checklist to support more complete reporting.ConclusionsOur findings supported the need for a set of consolidated reporting items, given that existing high-quality guidelines and checklists do not individually provide complete coverage. The consolidated set of reporting items is expected to improve the quality and reproducibility of ML modeling studies.