Dataset Information

Recommendations for improving the quality of reporting clinical electrochemotherapy studies based on qualitative systematic review.

ABSTRACT: BACKGROUND:Electrochemotherapy is becoming a well-established treatment for malignancies of skin and non-skin origin and its use is widening across Europe. The technique was developed and optimized from solid experimental and clinical evidence. A consensus document is now warranted to formalize reporting results, which should strengthen evidence-based practice recommendations. This consensus should be derived from high quality clinical data collection, clinical expertise and summarizing patient feedback. The first step, which is addressed in this paper, aims to critically analyze the quality of published studies and to provide the recommendations for reporting clinical trials on electrochemotherapy. METHODS:The quality of reporting in published studies on electrochemotherapy was analyzed in order to produce procedure specific reporting recommendations. A comprehensive literature search of studies published from 2006 to 2015 was performed followed by qualitative analysis of manuscripts assessing for 47 quality criteria grouped into four major clusters: (1) trial design, (2) description of patient population, (3) description of treatment delivery and patient outcome, (4) analysis of results and their interpretation. The summary measure during literature assessment was the proportion of studies fulfilling each manuscript quality criteria. RESULTS:A total of 56 studies were screened, from the period 2006 to 2015, of which 33 were included in the qualitative analysis, with a total of 1215 patients. Overall, the quality of reporting was highly variable. Twenty-four reports (73%) were single-center, non-comparative studies, and only 15 (45%) were prospective in nature (only 2 of them were entered into a clinical trials registry). Electrochemotherapy technique was consistently reported, with most studies (31/33) adhering closely to published standard operating procedures. The quality of reporting the patient population was variable among the analyzed studies, with only between 45% and 100% achieving dedicated quality criteria. Reporting of treatment delivery and patient outcome was also highly variable with studies only fulfilling between 3% and 100%. Finally, reporting study results critically varied, fulfilling from 27% to 100% of the quality criteria. Based on the critical issues emerging from this analysis, recommendations and minimal requirements for reporting clinical data on electrochemotherapy were prepared and summarized into a checklist. CONCLUSIONS:There is an increasing body of published clinical data on electrochemotherapy, but more high quality clinical data are needed. Published papers often lack accurate description of study population, treatment delivery as well as patient outcome. Our recommendations, provided in the form of a summary checklist, are intended to ameliorate data reporting in future studies on electrochemotherapy and help researchers to provide a solid evidence basis for clinical practice.

SUBMITTER: Campana LG

PROVIDER: S-EPMC4825335 | biostudies-literature | 2016 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Recommendations for improving the quality of reporting clinical electrochemotherapy studies based on qualitative systematic review.

Campana Luca G LG Clover A James P AJ Valpione Sara S Quaglino Pietro P Gehl Julie J Kunte Christian C Snoj Marko M Cemazar Maja M Rossi Carlo R CR Miklavcic Damijan D Sersa Gregor G

Radiology and oncology 20160216 1

<h4>Background</h4>Electrochemotherapy is becoming a well-established treatment for malignancies of skin and non-skin origin and its use is widening across Europe. The technique was developed and optimized from solid experimental and clinical evidence. A consensus document is now warranted to formalize reporting results, which should strengthen evidence-based practice recommendations. This consensus should be derived from high quality clinical data collection, clinical expertise and summarizing ...[more]

PMID: 27069444

Similar Datasets

Project description:BackgroundThere has been a rapid increase in the number of Artificial Intelligence (AI) studies of cardiac MRI (CMR) segmentation aiming to automate image analysis. However, advancement and clinical translation in this field depend on researchers presenting their work in a transparent and reproducible manner. This systematic review aimed to evaluate the quality of reporting in AI studies involving CMR segmentation.MethodsMEDLINE and EMBASE were searched for AI CMR segmentation studies in April 2022. Any fully automated AI method for segmentation of cardiac chambers, myocardium or scar on CMR was considered for inclusion. For each study, compliance with the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) was assessed. The CLAIM criteria were grouped into study, dataset, model and performance description domains.Results209 studies published between 2012 and 2022 were included in the analysis. Studies were mainly published in technical journals (58%), with the majority (57%) published since 2019. Studies were from 37 different countries, with most from China (26%), the United States (18%) and the United Kingdom (11%). Short axis CMR images were most frequently used (70%), with the left ventricle the most commonly segmented cardiac structure (49%). Median compliance of studies with CLAIM was 67% (IQR 59-73%). Median compliance was highest for the model description domain (100%, IQR 80-100%) and lower for the study (71%, IQR 63-86%), dataset (63%, IQR 50-67%) and performance (60%, IQR 50-70%) description domains.ConclusionThis systematic review highlights important gaps in the literature of CMR studies using AI. We identified key items missing-most strikingly poor description of patients included in the training and validation of AI models and inadequate model failure analysis-that limit the transparency, reproducibility and hence validity of published AI studies. This review may support closer adherence to established frameworks for reporting standards and presents recommendations for improving the quality of reporting in this field.Systematic review registration[www.crd.york.ac.uk/prospero/], identifier [CRD42022279214].

Project description:BackgroundAutomated language analysis of radiology reports using natural language processing (NLP) can provide valuable information on patients' health and disease. With its rapid development, NLP studies should have transparent methodology to allow comparison of approaches and reproducibility. This systematic review aims to summarise the characteristics and reporting quality of studies applying NLP to radiology reports.MethodsWe searched Google Scholar for studies published in English that applied NLP to radiology reports of any imaging modality between January 2015 and October 2019. At least two reviewers independently performed screening and completed data extraction. We specified 15 criteria relating to data source, datasets, ground truth, outcomes, and reproducibility for quality assessment. The primary NLP performance measures were precision, recall and F1 score.ResultsOf the 4,836 records retrieved, we included 164 studies that used NLP on radiology reports. The commonest clinical applications of NLP were disease information or classification (28%) and diagnostic surveillance (27.4%). Most studies used English radiology reports (86%). Reports from mixed imaging modalities were used in 28% of the studies. Oncology (24%) was the most frequent disease area. Most studies had dataset size > 200 (85.4%) but the proportion of studies that described their annotated, training, validation, and test set were 67.1%, 63.4%, 45.7%, and 67.7% respectively. About half of the studies reported precision (48.8%) and recall (53.7%). Few studies reported external validation performed (10.8%), data availability (8.5%) and code availability (9.1%). There was no pattern of performance associated with the overall reporting quality.ConclusionsThere is a range of potential clinical applications for NLP of radiology reports in health services and research. However, we found suboptimal reporting quality that precludes comparison, reproducibility, and replication. Our results support the need for development of reporting standards specific to clinical NLP studies.

Project description:BackgroundDiabetes mellitus (DM) is a major health concern among children with the widespread adoption of advanced technologies. However, concerns are growing about the transparency, replicability, biasedness, and overall validity of artificial intelligence studies in medicine.ObjectiveWe aimed to systematically review the reporting quality of machine learning (ML) studies of pediatric DM using the Minimum Information About Clinical Artificial Intelligence Modelling (MI-CLAIM) checklist, a general reporting guideline for medical artificial intelligence studies.MethodsWe searched the PubMed and Web of Science databases from 2016 to 2020. Studies were included if the use of ML was reported in children with DM aged 2 to 18 years, including studies on complications, screening studies, and in silico samples. In studies following the ML workflow of training, validation, and testing of results, reporting quality was assessed via MI-CLAIM by consensus judgments of independent reviewer pairs. Positive answers to the 17 binary items regarding sufficient reporting were qualitatively summarized and counted as a proxy measure of reporting quality. The synthesis of results included testing the association of reporting quality with publication and data type, participants (human or in silico), research goals, level of code sharing, and the scientific field of publication (medical or engineering), as well as with expert judgments of clinical impact and reproducibility.ResultsAfter screening 1043 records, 28 studies were included. The sample size of the training cohort ranged from 5 to 561. Six studies featured only in silico patients. The reporting quality was low, with great variation among the 21 studies assessed using MI-CLAIM. The number of items with sufficient reporting ranged from 4 to 12 (mean 7.43, SD 2.62). The items on research questions and data characterization were reported adequately most often, whereas items on patient characteristics and model examination were reported adequately least often. The representativeness of the training and test cohorts to real-world settings and the adequacy of model performance evaluation were the most difficult to judge. Reporting quality improved over time (r=0.50; P=.02); it was higher than average in prognostic biomarker and risk factor studies (P=.04) and lower in noninvasive hypoglycemia detection studies (P=.006), higher in studies published in medical versus engineering journals (P=.004), and higher in studies sharing any code of the ML pipeline versus not sharing (P=.003). The association between expert judgments and MI-CLAIM ratings was not significant.ConclusionsThe reporting quality of ML studies in the pediatric population with DM was generally low. Important details for clinicians, such as patient characteristics; comparison with the state-of-the-art solution; and model examination for valid, unbiased, and robust results, were often the weak points of reporting. To assess their clinical utility, the reporting standards of ML studies must evolve, and algorithms for this challenging population must become more transparent and replicable.

Project description:AIMS:We conducted a systematic review assessing the reporting quality of studies validating models based on machine learning (ML) for clinical diagnosis, with a specific focus on the reporting of information concerning the participants on which the diagnostic task was evaluated on. METHOD:Medline Core Clinical Journals were searched for studies published between July 2015 and July 2018. Two reviewers independently screened the retrieved articles, a third reviewer resolved any discrepancies. An extraction list was developed from the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis guideline. Two reviewers independently extracted the data from the eligible articles. Third and fourth reviewers checked, verified the extracted data as well as resolved any discrepancies between the reviewers. RESULTS:The search results yielded 161 papers, of which 28 conformed to the eligibility criteria. Detail of data source was reported in 24 of the 28 papers. For all of the papers, the set of patients on which the ML-based diagnostic system was evaluated was partitioned from a larger dataset, and the method for deriving such set was always reported. Information on the diagnostic/non-diagnostic classification was reported well (23/28). The least reported items were the use of reporting guideline (0/28), distribution of disease severity (8/28 patient flow diagram (10/28) and distribution of alternative diagnosis (10/28). A large proportion of studies (23/28) had a delay between the conduct of the reference standard and ML tests, while one study did not and four studies were unclear. For 15 studies, it was unclear whether the evaluation group corresponded to the setting in which the ML test will be applied to. CONCLUSION:All studies in this review failed to use reporting guidelines, and a large proportion of them lacked adequate detail on participants, making it difficult to replicate, assess and interpret study findings. PROSPERO REGISTRATION NUMBER:CRD42018099167.

Project description:IntroductionComplete reporting assists readers in confirming the methodological rigor and validity of findings and allows replication. The reporting quality of observational functional magnetic resonance imaging (fMRI) studies involving clinical participants is unclear.ObjectivesWe sought to determine the quality of reporting in observational fMRI studies involving clinical participants.MethodsWe searched OVID MEDLINE for fMRI studies in six leading journals between January 2010 and December 2011.Three independent reviewers abstracted data from articles using an 83-item checklist adapted from the guidelines proposed by Poldrack et al. (Neuroimage 2008; 40: 409-14). We calculated the percentage of articles reporting each item of the checklist and the percentage of reported items per article.ResultsA random sample of 100 eligible articles was included in the study. Thirty-one items were reported by fewer than 50% of the articles and 13 items were reported by fewer than 20% of the articles. The median percentage of reported items per article was 51% (ranging from 30% to 78%). Although most articles reported statistical methods for within-subject modeling (92%) and for between-subject group modeling (97%), none of the articles reported observed effect sizes for any negative finding (0%). Few articles reported justifications for fixed-effect inferences used for group modeling (3%) and temporal autocorrelations used to account for within-subject variances and correlations (18%). Other under-reported areas included whether and how the task design was optimized for efficiency (22%) and distributions of inter-trial intervals (23%).ConclusionsThis study indicates that substantial improvement in the reporting of observational clinical fMRI studies is required. Poldrack et al.'s guidelines provide a means of improving overall reporting quality. Nonetheless, these guidelines are lengthy and may be at odds with strict word limits for publication; creation of a shortened-version of Poldrack's checklist that contains the most relevant items may be useful in this regard.

Project description:BackgroundDespite more than 60 years of clinical trials, tuberculosis (TB) still causes a high global burden of mortality and morbidity. Treatment currently requires multiple drugs in combination, taken over a prolonged period. New drugs are needed to shorten treatment duration, prevent resistance and reduce adverse events. However, to improve on current methodology in drug development, a more complete understanding of the existing clinical evidence base is required.MethodsA systematic review was undertaken to summarise outcomes reported in phase III trials of patients with newly diagnosed pulmonary TB. A systematic search of databases (PubMed, MEDLINE, EMBASE, CENTRAL and LILACs) was conducted on 30 November 2017 to retrieve relevant peer-reviewed articles. Reference lists of included studies were also searched. This systematic review considered all reported outcomes.ResultsOf 248 included studies, 229 considered "on-treatment" outcomes whilst 148 reported "off-treatment" outcomes. There was wide variation and ambiguity in the definition of reported outcomes, including their relationship to treatment and in the time points evaluated. Additional challenges were observed regarding the analysis approach taken (per protocol versus intention to treat) and the varying durations of "intensive" and "continuation" phases of treatment. Bacteriological outcomes were most frequently reported but radiological and clinical data were often included as an implicit or explicit component of the overall definition of outcome.ConclusionsTerminology used to define long-term outcomes in phase III trials is inconsistent, reflecting evolving differences in protocols and practices. For successful future cumulative meta-analysis, the findings of this review suggest that greater availability of individual patient data and the development of a core outcome set would be desirable. In the meantime, we propose a simple and logical approach which should facilitate combination of key evidence and inform improvements in the methodology of TB drug development and clinical trials.

Dataset Information

Recommendations for improving the quality of reporting clinical electrochemotherapy studies based on qualitative systematic review.

Publications

Recommendations for improving the quality of reporting clinical electrochemotherapy studies based on qualitative systematic review.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets