Dataset Information

Multi-institutional generalizability of a plan complexity machine learning model for predicting pre-treatment quality assurance results in radiotherapy.

ABSTRACT:

Background and purpose

Treatment plans in radiotherapy are subject to measurement-based pre-treatment verifications. In this study, plan complexity metrics (PCMs) were calculated per beam and used as input features to develop a predictive model. The aim of this study was to determine the robustness against differences in machine type and institutional-specific quality assurance (QA).

Material and methods

A number of 567 beams were collected, where 477 passed and 90 failed the pre-treatment QA. Treatment plans of different anatomical regions were included. One type of linear accelerator was represented. For all beams, 16 PCMs were calculated. A random forest classifier was trained to distinct between acceptable and non-acceptable beams. The model was validated on other datasets to investigate its robustness. Firstly, plans for another machine type from the same institution were evaluated. Secondly, an inter-institutional validation was conducted on three datasets from different centres with their associated QA.

Results

Intra-institutionally, the PCMs beam modulation, mean MLC gap, Q1 gap, and Modulation Complexity Score were the most informative to detect failing beams. Eighty-tree percent of the failed beams (15/18) were detected correctly. The model could not detect over-modulated beams of another machine type. Inter-institutionally, the model performance reached higher accuracy for centres with comparable equipment both for treatment and QA as the local institute.

Conclusions

The study demonstrates that the robustness decreases when major differences appear in the QA platform or in planning strategies, but that it is feasible to extrapolate institutional-specific trained models between centres with similar clinical practice. Predictive models should be developed for each machine type.

SUBMITTER: Claessens M

PROVIDER: S-EPMC10776441 | biostudies-literature | 2024 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Multi-institutional generalizability of a plan complexity machine learning model for predicting pre-treatment quality assurance results in radiotherapy.

Claessens Michaël M De Kerf Geert G Vanreusel Verdi V Mollaert Isabelle I Hernandez Victor V Saez Jordi J Jornet Núria N Verellen Dirk D

Physics and imaging in radiation oncology 20231219

<h4>Background and purpose</h4>Treatment plans in radiotherapy are subject to measurement-based pre-treatment verifications. In this study, plan complexity metrics (PCMs) were calculated per beam and used as input features to develop a predictive model. The aim of this study was to determine the robustness against differences in machine type and institutional-specific quality assurance (QA).<h4>Material and methods</h4>A number of 567 beams were collected, where 477 passed and 90 failed the pre- ...[more]

PMID: 38204910

Similar Datasets

Project description:PurposeArtificial intelligence applications in radiation oncology have been the focus of study in the last decade. The introduction of automated and intelligent solutions for routine clinical tasks, such as treatment planning and quality assurance, has the potential to increase safety and efficiency of radiotherapy. In this work, we present a multi-institutional study across three different institutions internationally on a Bayesian network (BN)-based initial plan review assistive tool that alerts radiotherapy professionals for potential erroneous or suboptimal treatment plans.MethodsClinical data were collected from the oncology information systems in three institutes in Europe (Maastro clinic - 8753 patients treated between 2012 and 2020) and the United States of America (University of Vermont Medical Center [UVMMC] - 2733 patients, University of Washington [UW] - 6180 patients, treated between 2018 and 2021). We trained the BN model to detect potential errors in radiotherapy treatment plans using different combinations of institutional data and performed single-site and cross-site validation with simulated plans with embedded errors. The simulated errors consisted of three different categories: i) patient setup, ii) treatment planning and iii) prescription. We also compared the strategy of using only diagnostic parameters or all variables as evidence for the BN. We evaluated the model performance utilizing the area under the receiver-operating characteristic curve (AUC).ResultsThe best network performance was observed when the BN model is trained and validated using the dataset in the same center. In particular, the testing and validation using UVMMC data has achieved an AUC of 0.92 with all parameters used as evidence. In cross-validation studies, we observed that the BN model performed better when it was trained and validated in institutes with similar technology and treatment protocols (for instance, when testing on UVMMC data, the model trained on UW data achieved an AUC of 0.84, compared with an AUC of 0.64 for the model trained on Maastro data). Also, combining training data from larger clinics (UW and Maastro clinic) and using it on smaller clinics (UVMMC) leads to satisfactory performance with an AUC of 0.85. Lastly, we found that in general the BN model performed better when all variables are considered as evidence.ConclusionWe have developed and validated a Bayesian network model to assist initial treatment plan review using multi-institutional data with different technology and clinical practices. The model has shown good performance even when trained on data from clinics with divergent profiles, suggesting that the model is able to adapt to different data distributions.

Project description:BackgroundAntimicrobial resistance (AMR) remains a significant global health threat particularly impacting low- and middle-income countries (LMICs). These regions often grapple with limited healthcare resources and access to advanced diagnostic tools. Consequently, there is a pressing need for innovative approaches that can enhance AMR surveillance and management. Machine learning (ML) though underutilized in these settings, presents a promising avenue. This study leverages ML models trained on whole-genome sequencing data from England, where such data is more readily available, to predict AMR in E. coli, targeting key antibiotics such as ciprofloxacin, ampicillin, and cefotaxime. A crucial part of our work involved the validation of these models using an independent dataset from Africa, specifically from Uganda, Nigeria, and Tanzania, to ascertain their applicability and effectiveness in LMICs.ResultsModel performance varied across antibiotics. The Support Vector Machine excelled in predicting ciprofloxacin resistance (87% accuracy, F1 Score: 0.57), Light Gradient Boosting Machine for cefotaxime (92% accuracy, F1 Score: 0.42), and Gradient Boosting for ampicillin (58% accuracy, F1 Score: 0.66). In validation with data from Africa, Logistic Regression showed high accuracy for ampicillin (94%, F1 Score: 0.97), while Random Forest and Light Gradient Boosting Machine were effective for ciprofloxacin (50% accuracy, F1 Score: 0.56) and cefotaxime (45% accuracy, F1 Score:0.54), respectively. Key mutations associated with AMR were identified for these antibiotics.ConclusionAs the threat of AMR continues to rise, the successful application of these models, particularly on genomic datasets from LMICs, signals a promising avenue for improving AMR prediction to support large AMR surveillance programs. This work thus not only expands our current understanding of the genetic underpinnings of AMR but also provides a robust methodological framework that can guide future research and applications in the fight against AMR.

Project description:PurposeThe PERYTON trial is a multicenter randomized controlled trial that will investigate whether the treatment outcome of salvage external beam radiation therapy (sEBRT) will be improved with hypofractionated radiation therapy. A pretrial quality assurance (QA) program was undertaken to ensure protocol compliance within the PERYTON trial and to assess variation in sEBRT treatment protocols between the participating centers.Methods and materialsCompletion of the QA program was mandatory for each participating center (N = 8) to start patient inclusion. The pretrial QA program included (1) a questionnaire on the center-specific sEBRT protocol, (2) a delineation exercise of the clinical target volume (CTV) and organs at risk, and (3) a treatment planning exercise. All contours were analyzed using the pairwise dice similarity coefficient (DSC) and the 50th and 95th percentile Hausdorff distance (HD50 and HD95, respectively). The submitted treatment plans were reviewed for protocol compliance.ResultsThe results of the questionnaire showed that high-quality, state-of-the-art radiation therapy techniques were used in the participating centers and identified variations of the sEBRT protocols used concerning the position verification and preparation techniques. The submitted CTVs showed significant variation, with a range in volume of 29 cm3 to 167 cm3, a mean pairwise DSC of 0.52, and a mean HD50 and HD95 of 2.3 mm and 24.4 mm, respectively. Only in 1 center the treatment plan required adaptation before meeting all constraints of the PERYTON protocol.ConclusionsThe pretrial QA of the PERYTON trial demonstrated that high-quality, but variable, radiation techniques were used in the 8 participating centers. The treatment planning exercise confirmed that the dose constraints of the PERYTON protocol were feasible for all participating centers. The observed variation in CTV delineation led to agreement on a new (image-based) delineation guideline to be used by all participating centers within the PERYTON trial.

Dataset Information

Multi-institutional generalizability of a plan complexity machine learning model for predicting pre-treatment quality assurance results in radiotherapy.

Background and purpose

Material and methods

Results

Conclusions

Publications

Multi-institutional generalizability of a plan complexity machine learning model for predicting pre-treatment quality assurance results in radiotherapy.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets