Dataset Information

Evaluating Modeling and Validation Strategies for Tooth Loss.

ABSTRACT: Prediction models learn patterns from available data (training) and are then validated on new data (testing). Prediction modeling is increasingly common in dental research. We aimed to evaluate how different model development and validation steps affect the predictive performance of tooth loss prediction models of patients with periodontitis. Two independent cohorts (627 patients, 11,651 teeth) were followed over a mean ± SD 18.2 ± 5.6 y (Kiel cohort) and 6.6 ± 2.9 y (Greifswald cohort). Tooth loss and 10 patient- and tooth-level predictors were recorded. The impact of different model development and validation steps was evaluated: 1) model complexity (logistic regression, recursive partitioning, random forest, extreme gradient boosting), 2) sample size (full data set or 10%, 25%, or 75% of cases dropped at random), 3) prediction periods (maximum 10, 15, or 20 y or uncensored), and 4) validation schemes (internal or external by centers/time). Tooth loss was generally a rare event (880 teeth were lost). All models showed limited sensitivity but high specificity. Patients' age and tooth loss at baseline as well as probing pocket depths showed high variable importance. More complex models (random forest, extreme gradient boosting) had no consistent advantages over simpler ones (logistic regression, recursive partitioning). Internal validation (in sample) overestimated the predictive power (area under the curve up to 0.90), while external validation (out of sample) found lower areas under the curve (range 0.62 to 0.82). Reducing the sample size decreased the predictive power, particularly for more complex models. Censoring the prediction period had only limited impact. When the model was trained in one period and tested in another, model outcomes were similar to the base case, indicating temporal validation as a valid option. No model showed higher accuracy than the no-information rate. In conclusion, none of the developed models would be useful in a clinical setting, despite high accuracy. During modeling, rigorous development and external validation should be applied and reported accordingly.

SUBMITTER: Krois J

PROVIDER: S-EPMC6710618 | biostudies-literature | 2019 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluating Modeling and Validation Strategies for Tooth Loss.

Krois J J Graetz C C Holtfreter B B Brinkmann P P Kocher T T Schwendicke F F

Journal of dental research 20190730 10

Prediction models learn patterns from available data (training) and are then validated on new data (testing). Prediction modeling is increasingly common in dental research. We aimed to evaluate how different model development and validation steps affect the predictive performance of tooth loss prediction models of patients with periodontitis. Two independent cohorts (627 patients, 11,651 teeth) were followed over a mean ± SD 18.2 ± 5.6 y (Kiel cohort) and 6.6 ± 2.9 y (Greifswald cohort). Tooth l ...[more]

PMID: 31361174

Similar Datasets

Project description:ObjectivesTo illustrate the epidemiologic and cost-effectiveness impact of shifting the focus from population-based screening toward a targeted management approach for genital chlamydia infection.DesignModeling study, implementing an individual-based, stochastic, dynamic network model.SettingHong Kong.PopulationA hypothetical sample network of 10,000 people with a partnership distribution based on Hong Kong's sexually active population of reproductive age (age 18-49 years).InterventionsIn this study, we present several scenarios with different implementations of universal vs. targeted screening (based on partner numbers). We also explored the impact of (1) screening only, (2) screening plus expedited partner therapy, and (3) screening plus partner testing.Primary outcome measuresChange of chlamydia prevalence before and after implementing the different strategies. The cost-effectiveness analysis reports total direct cost from a health provider perspective, the QALYs gained, and incremental cost-effectiveness ratios (ICER).ResultsIn comparing the effects of universal screening only and targeted screening of the high-risk population, the mean prevalence during the 10th year of intervention was 2.75 ± 0.30% and 2.35 ± 0.21%, respectively (compared with 3.24 ± 0.30% and 3.35 ± 0.21% before the interventions, respectively). The addition of contact tracing to the latter targeted screening scenario reduces the mean prevalence during the 10th year of intervention to 1.48 ± 0.13% (compared with 3.31 ± 0.33% at baseline) in the best-case of testing before treatment and maximal contact-tracing effectiveness (40%). Overall, the most effective scenarios were those for which interventions focused on the high-risk population defined by the number of partners, with contact tracing included. The ICER for targeted screening with contact tracing at 20% and 40% efficiency was $4,634 and $7,219 per QALY gained, respectively (10-year time horizon). Expedited partner therapy did not significantly impact overall chlamydia prevalence and caused overtreatment.ConclusionsOur study suggests that targeted screening with strengthened contact tracing efforts is the most cost-effective strategy to reduce the prevalence of chlamydia in Hong Kong.

Dataset Information

Evaluating Modeling and Validation Strategies for Tooth Loss.

Publications

Evaluating Modeling and Validation Strategies for Tooth Loss.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets