The Pursuit of Generalizability to Enable Clinical Translation of Radiomics.
Ontology highlight
ABSTRACT:
SUBMITTER: Tiwari P
PROVIDER: S-EPMC7845780 | biostudies-literature | 2021 Jan
REPOSITORIES: biostudies-literature
Publications
The Pursuit of Generalizability to Enable Clinical Translation of Radiomics.
Tiwari Pallavi P Verma Ruchika R
Radiology. Artificial intelligence 20201216 1
Similar Datasets
Project description:ObjectiveTo evaluate the quality of radiomics studies on pituitary adenoma according to the radiomics quality score (RQS) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD).MethodsPubMed MEDLINE and EMBASE were searched to identify radiomics studies on pituitary adenomas. From 138 articles, 20 relevant original research articles were included. Studies were scored based on RQS and TRIPOD guidelines.ResultsMost included studies did not perform pre-processing; isovoxel resampling, signal intensity normalization, and N4 bias field correction were performed in only five (25%), eight (40%), and four (20%) studies, respectively. Only two (10%) studies performed external validation. The mean RQS and basic adherence rate were 2.8 (7.6%) and 26.6%, respectively. There was a low adherence rate for conducting comparison to "gold-standard" (20%), multiple segmentation (25%), and stating potential clinical utility (25%). No study stated the biological correlation, conducted a test-retest or phantom study, was a prospective study, conducted cost-effectiveness analysis, or provided open-source code and data, which resulted in low-level evidence. The overall adherence rate for TRIPOD was 54.6%, and it was low for reporting the title (5%), abstract (0%), explaining the sample size (10%), and suggesting a full prediction model (5%).ConclusionThe radiomics reporting quality for pituitary adenoma is insufficient. Pre-processing is required for feature reproducibility and external validation is necessary. Feature reproducibility, clinical utility demonstration, higher evidence levels, and open science are required. Titles, abstracts, and full prediction model suggestions should be improved for transparent reporting.Advances in knowledgeDespite the rapidly increasing number of radiomics researches on pituitary adenoma, the quality of science in these researches is unknown. Our study indicates that the overall quality needs to be significantly improved in radiomics studies on pituitary adenoma, and since the concept of RQS and IBSI is still unfamiliar to clinicians and radiologist researchers, our study may help to reach higher technical and clinical impact in the future study.
| S-EPMC9793472 | biostudies-literature
Project description:ObjectiveOur study aimed to evaluate the quality of radiomics studies on brain metastases based on the radiomics quality score (RQS), Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) checklist, and the Image Biomarker Standardization Initiative (IBSI) guidelines.Materials and methodsPubMed MEDLINE, and EMBASE were searched for articles on radiomics for evaluating brain metastases, published until February 2021. Of the 572 articles, 29 relevant original research articles were included and evaluated according to the RQS, TRIPOD checklist, and IBSI guidelines.ResultsExternal validation was performed in only three studies (10.3%). The median RQS was 3.0 (range, -6 to 12), with a low basic adherence rate of 50.0%. The adherence rate was low in comparison to the "gold standard" (10.3%), stating the potential clinical utility (10.3%), performing the cut-off analysis (3.4%), reporting calibration statistics (6.9%), and providing open science and data (3.4%). None of the studies involved test-retest or phantom studies, prospective studies, or cost-effectiveness analyses. The overall rate of adherence to the TRIPOD checklist was 60.3% and low for reporting title (3.4%), blind assessment of outcome (0%), description of the handling of missing data (0%), and presentation of the full prediction model (0%). The majority of studies lacked pre-processing steps, with bias-field correction, isovoxel resampling, skull stripping, and gray-level discretization performed in only six (20.7%), nine (31.0%), four (3.8%), and four (13.8%) studies, respectively.ConclusionThe overall scientific and reporting quality of radiomics studies on brain metastases published during the study period was insufficient. Radiomics studies should adhere to the RQS, TRIPOD, and IBSI guidelines to facilitate the translation of radiomics into the clinical field.
| S-EPMC8743155 | biostudies-literature
Project description:Although there have been a number of clinical trials evaluating treatments for adolescents with major depressive disorder (MDD), the generalizability of those trials to samples of depressed adolescents who present for routine clinical care is unknown. Examining the generalizability of clinical trials of pharmacological and psychotherapy interventions for adolescent depression can help administrators and frontline practitioners determine the relevance of these studies for their patients and may also guide eligibility criteria for future clinical trials in this clinical population.Data on nationally representative adolescents were derived from the National Comorbidity Survey: Adolescent Supplement. To assess the generalizability of adolescent clinical trials for MDD, we applied a standard set of eligibility criteria representative of clinical trials to all adolescents in the National Comorbidity Survey: Adolescent Supplement with a Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition diagnosis of MDD (N = 592).From the overall MDD sample, 61.9% would have been excluded from a typical pharmacological trial, whereas 42.2% would have been excluded from a psychotherapy trial. Among those who sought treatment (n = 412), the corresponding exclusion rates were 72.7% for a pharmacological trial and 52.2% for a psychotherapy trial. The criterion leading to the largest number of exclusions was "significant risk of suicide" in both pharmacological and psychotherapy trials.Pharmacological and, to a lesser extent, psychotherapy clinical trials likely exclude most adolescents with MDD. Careful consideration should be given to balancing eligibility criteria and internal validity with applicability in routine clinical care while ensuring patient safety.
| S-EPMC5703774 | biostudies-other
Project description:ObjectiveTo conduct an overview of meta-analyses of radiomics studies assessing their study quality and evidence level.MethodsA systematical search was updated via peer-reviewed electronic databases, preprint servers, and systematic review protocol registers until 15 November 2022. Systematic reviews with meta-analysis of primary radiomics studies were included. Their reporting transparency, methodological quality, and risk of bias were assessed by PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) 2020 checklist, AMSTAR-2 (A MeaSurement Tool to Assess systematic Reviews, version 2) tool, and ROBIS (Risk Of Bias In Systematic reviews) tool, respectively. The evidence level supporting the radiomics for clinical use was rated.ResultsWe identified 44 systematic reviews with meta-analyses on radiomics research. The mean ± standard deviation of PRISMA adherence rate was 65 ± 9%. The AMSTAR-2 tool rated 5 and 39 systematic reviews as low and critically low confidence, respectively. The ROBIS assessment resulted low, unclear and high risk in 5, 11, and 28 systematic reviews, respectively. We reperformed 53 meta-analyses in 38 included systematic reviews. There were 3, 7, and 43 meta-analyses rated as convincing, highly suggestive, and weak levels of evidence, respectively. The convincing level of evidence was rated in (1) T2-FLAIR radiomics for IDH-mutant vs IDH-wide type differentiation in low-grade glioma, (2) CT radiomics for COVID-19 vs other viral pneumonia differentiation, and (3) MRI radiomics for high-grade glioma vs brain metastasis differentiation.ConclusionsThe systematic reviews on radiomics were with suboptimal quality. A limited number of radiomics approaches were supported by convincing level of evidence.Clinical relevance statementThe evidence supporting the clinical application of radiomics are insufficient, calling for researches translating radiomics from an academic tool to a practicable adjunct towards clinical deployment.
| S-EPMC10279606 | biostudies-literature
Project description:The heterogeneity of MRI is one of the major reasons for decreased performance of a radiomics model on external validation, limiting the model's generalizability and clinical application. We aimed to establish a generalizable radiomics model to predict meningioma grade on external validation through leveraging Cycle-Consistent Adversarial Networks (CycleGAN). In this retrospective study, 257 patients with meningioma were included in the institutional training set. Radiomic features (n = 214) were extracted from T2-weighted (T2) and contrast-enhanced T1 (T1C) images. After radiomics feature selection, extreme gradient boosting classifiers were developed. The models were validated in the external validation set consisting of 61 patients with meningiomas. To reduce the gap in generalization associated with the inter-institutional heterogeneity of MRI, the smaller image set style of the external validation was translated into the larger image set style of the institutional training set using CycleGAN. On external validation before CycleGAN application, the performance of the combined T2 and T1C models showed an area under the curve (AUC), accuracy, and F1 score of 0.77 (95% confidence interval 0.63-0.91), 70.7%, and 0.54, respectively. After applying CycleGAN, the performance of the combined T2 and T1C models increased, with an AUC, accuracy, and F1 score of 0.83 (95% confidence interval 0.70-0.97), 73.2%, and 0.59, respectively. Quantitative metrics (by Fréchet Inception Distance) showed that CycleGAN can decrease inter-institutional image heterogeneity while preserving predictive information. In conclusion, leveraging CycleGAN may be helpful to increase the generalizability of a radiomics model in differentiating meningioma grade on external validation.
| S-EPMC9055063 | biostudies-literature
Project description:Hypertension, exercise, and pregnancy are common triggers of cardiac remodeling, which occurs primarily through the hypertrophy of individual cardiomyocytes. During hypertrophy, stress-induced signal transduction increases cardiomyocyte transcription and translation, which promotes the addition of new contractile units through poorly understood mechanisms. The cardiomyocyte microtubule network is also implicated in hypertrophy, but via an unknown role. Here, we show that microtubules are indispensable for cardiac growth via spatiotemporal control of the translational machinery. We find that the microtubule motor Kinesin-1 distributes mRNAs and ribosomes along microtubule tracks to discrete domains within the cardiomyocyte. Upon hypertrophic stimulation, microtubules redistribute mRNAs and new protein synthesis to sites of growth at the cell periphery. If the microtubule network is disrupted, mRNAs and ribosomes collapse around the nucleus, which results in mislocalized protein synthesis, the rapid degradation of new proteins, and a failure of growth, despite normally increased translation rates. Together, these data indicate that mRNAs and ribosomes are actively transported to specific sites to facilitate local translation and assembly of contractile units, and suggest that properly localized translation - and not simply translation rate - is a critical determinant of cardiac hypertrophy. In this work, we find that microtubule based-transport is essential to couple augmented transcription and translation to productive cardiomyocyte growth during cardiac stress.
| S-EPMC7952726 | biostudies-literature
Project description:ObjectivesThis study examines factors affecting reliability, or consistency of assessment scores, from an objective structured clinical examination (OSCE) in neurology through generalizability theory (G theory).MethodsData include assessments from a multistation OSCE taken by 194 medical students at the completion of a neurology clerkship. Facets evaluated in this study include cases, domains, and items. Domains refer to areas of skill (or constructs) that the OSCE measures. G theory is used to estimate variance components associated with each facet, derive reliability, and project the number of cases required to obtain a reliable (consistent, precise) score.ResultsReliability using G theory is moderate (Φ coefficient = 0.61, G coefficient = 0.64). Performance is similar across cases but differs by the particular domain, such that the majority of variance is attributed to the domain. Projections in reliability estimates reveal that students need to participate in 3 OSCE cases in order to increase reliability beyond the 0.70 threshold.ConclusionsThis novel use of G theory in evaluating an OSCE in neurology provides meaningful measurement characteristics of the assessment. Differing from prior work in other medical specialties, the cases students were randomly assigned did not influence their OSCE score; rather, scores varied in expected fashion by domain assessed.
| S-EPMC4642144 | biostudies-literature
Project description:Artificial intelligence (AI) and machine learning (ML) are becoming critical in developing and deploying personalized medicine and targeted clinical trials. Recent advances in ML have enabled the integration of wider ranges of data including both medical records and imaging (radiomics). However, the development of prognostic models is complex as no modeling strategy is universally superior to others and validation of developed models requires large and diverse datasets to demonstrate that prognostic models developed (regardless of method) from one dataset are applicable to other datasets both internally and externally. Using a retrospective dataset of 2,552 patients from a single institution and a strict evaluation framework that included external validation on three external patient cohorts (873 patients), we crowdsourced the development of ML models to predict overall survival in head and neck cancer (HNC) using electronic medical records (EMR) and pretreatment radiological images. To assess the relative contributions of radiomics in predicting HNC prognosis, we compared 12 different models using imaging and/or EMR data. The model with the highest accuracy used multitask learning on clinical data and tumor volume, achieving high prognostic accuracy for 2-year and lifetime survival prediction, outperforming models relying on clinical data only, engineered radiomics, or complex deep neural network architecture. However, when we attempted to extend the best performing models from this large training dataset to other institutions, we observed significant reductions in the performance of the model in those datasets, highlighting the importance of detailed population-based reporting for AI/ML model utility and stronger validation frameworks. We have developed highly prognostic models for overall survival in HNC using EMRs and pretreatment radiological images based on a large, retrospective dataset of 2,552 patients from our institution.Diverse ML approaches were used by independent investigators. The model with the highest accuracy used multitask learning on clinical data and tumor volume.External validation of the top three performing models on three datasets (873 patients) with significant differences in the distributions of clinical and demographic variables demonstrated significant decreases in model performance.SignificanceML combined with simple prognostic factors outperformed multiple advanced CT radiomics and deep learning methods. ML models provided diverse solutions for prognosis of patients with HNC but their prognostic value is affected by differences in patient populations and require extensive validation.
| S-EPMC10309070 | biostudies-literature
Project description:TAM-163, an agonist monoclonal antibody targeting tyrosine receptor kinase-B (TrkB), is currently being investigated as a potential body weight modulatory agent in humans. To support the selection of the dose range for the first-in-human (FIH) trial of TAM-163, we conducted a mechanistic analysis of the pharmacokinetic (PK) and pharmacodynamic (PD) data (e.g., body weight gain) obtained in lean cynomolgus and obese rhesus monkeys following single doses ranging from 0.3 to 60 mg/kg. A target-mediated drug disposition (TMDD) model was used to describe the observed nonlinear PK and Emax approach was used to describe the observed dose-dependent PD effect. The TMDD model development was supported by the experimental determination of the binding affinity constant (9.4 nM) and internalization rate of the drug-target complex (2.08 h(-1)). These mechanistic analyses enabled linking of exposure, target (TrkB) coverage, and pharmacological activity (e.g., PD) in monkeys, and indicated that ≥ 38% target coverage (time-average) was required to achieve significant body weight gain in monkeys. Based on the scaling of the TMDD model from monkeys to humans and assuming similar relationship between the target coverage and pharmacological activity between monkey and humans, subcutaneous (SC) doses of 1 and 15 mg/kg in humans were projected to be the minimally and the fully pharmacologically active doses, respectively. Based on the minimal anticipated biological effect level (MABEL) approach for starting dose selection, the dose of 0.05 mg/kg (3 mg for a 60 kg human) SC was recommended as the starting dose for FIH trials, because at this dose level<10% target coverage was projected at Cmax (and all other time points). This study illustrates a rational mechanistic approach for the selection of FIH dose range for a therapeutic protein with a complex model of action.
| S-EPMC4169031 | biostudies-literature
Project description:Lyme disease, caused by the spirochete Borrelia burgdorferi, is the most common vector-borne illness in the United States. Many aspects of the disease are still topics of controversy within the scientific and medical communities. One particular point of debate is the etiology behind antibiotic treatment failure of a significant portion (10-30%) of Lyme disease patients. The condition in which patients with Lyme disease continue to experience a variety of symptoms months to years after the recommended antibiotic treatment is most recently referred to in the literature as post treatment Lyme disease syndrome (PTLDS) or just simply post treatment Lyme disease (PTLD). The most commonly proposed mechanisms behind treatment failure include host autoimmune responses, long-term sequelae from the initial Borrelia infection, and persistence of the spirochete. The aims of this review will focus on the in vitro, in vivo, and clinical evidence that either validates or challenges these mechanisms, particularly with regard to the role of the immune response in disease and resolution of the infection. Next generation treatments and research into identifying biomarkers to predict treatment responses and outcomes for Lyme disease patients are also discussed. It is essential that definitions and guidelines for Lyme disease evolve with the research to translate diagnostic and therapeutic advances to patient care.
| S-EPMC10244525 | biostudies-literature