Dataset Information

Unremarked or Unperformed? Systematic Review on Reporting of Validation Efforts of Health Economic Decision Models in Seasonal Influenza and Early Breast Cancer.

ABSTRACT: Transparent reporting of validation efforts of health economic models give stakeholders better insight into the credibility of model outcomes. In this study we reviewed recently published studies on seasonal influenza and early breast cancer in order to gain insight into the reporting of model validation efforts in the overall health economic literature.A literature search was performed in Pubmed and Embase to retrieve health economic modelling studies published between 2008 and 2014. Reporting on model validation was evaluated by checking for the word validation, and by using AdViSHE (Assessment of the Validation Status of Health Economic decision models), a tool containing a structured list of relevant items for validation. Additionally, we contacted corresponding authors to ask whether more validation efforts were performed other than those reported in the manuscripts.A total of 53 studies on seasonal influenza and 41 studies on early breast cancer were included in our review. The word validation was used in 16 studies (30 %) on seasonal influenza and 23 studies (56 %) on early breast cancer; however, in a minority of studies, this referred to a model validation technique. Fifty-seven percent of seasonal influenza studies and 71 % of early breast cancer studies reported one or more validation techniques. Cross-validation of study outcomes was found most often. A limited number of studies reported on model validation efforts, although good examples were identified. Author comments indicated that more validation techniques were performed than those reported in the manuscripts.Although validation is deemed important by many researchers, this is not reflected in the reporting habits of health economic modelling studies. Systematic reporting of validation efforts would be desirable to further enhance decision makers' confidence in health economic models and their outcomes.

SUBMITTER: de Boer PT

PROVIDER: S-EPMC4980411 | biostudies-other | 2016 Sep

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Unremarked or Unperformed? Systematic Review on Reporting of Validation Efforts of Health Economic Decision Models in Seasonal Influenza and Early Breast Cancer.

de Boer Pieter T PT Frederix Geert W J GW Feenstra Talitha L TL Vemer Pepijn P

PharmacoEconomics 20160901 9

<h4>Background</h4>Transparent reporting of validation efforts of health economic models give stakeholders better insight into the credibility of model outcomes. In this study we reviewed recently published studies on seasonal influenza and early breast cancer in order to gain insight into the reporting of model validation efforts in the overall health economic literature.<h4>Methods</h4>A literature search was performed in Pubmed and Embase to retrieve health economic modelling studies publishe ...[more]

PMID: 27129572

Similar Datasets

Project description:BackgroundBefore considering whether to use a multivariable (diagnostic or prognostic) prediction model, it is essential that its performance be evaluated in data that were not used to develop the model (referred to as external validation). We critically appraised the methodological conduct and reporting of external validation studies of multivariable prediction models.MethodsWe conducted a systematic review of articles describing some form of external validation of one or more multivariable prediction models indexed in PubMed core clinical journals published in 2010. Study data were extracted in duplicate on design, sample size, handling of missing data, reference to the original study developing the prediction models and predictive performance measures.Results11,826 articles were identified and 78 were included for full review, which described the evaluation of 120 prediction models. in participant data that were not used to develop the model. Thirty-three articles described both the development of a prediction model and an evaluation of its performance on a separate dataset, and 45 articles described only the evaluation of an existing published prediction model on another dataset. Fifty-seven percent of the prediction models were presented and evaluated as simplified scoring systems. Sixteen percent of articles failed to report the number of outcome events in the validation datasets. Fifty-four percent of studies made no explicit mention of missing data. Sixty-seven percent did not report evaluating model calibration whilst most studies evaluated model discrimination. It was often unclear whether the reported performance measures were for the full regression model or for the simplified models.ConclusionsThe vast majority of studies describing some form of external validation of a multivariable prediction model were poorly reported with key details frequently not presented. The validation studies were characterised by poor design, inappropriate handling and acknowledgement of missing data and one of the most key performance measures of prediction models i.e. calibration often omitted from the publication. It may therefore not be surprising that an overwhelming majority of developed prediction models are not used in practice, when there is a dearth of well-conducted and clearly reported (external validation) studies describing their performance on independent participant data.

Project description:BackgroundA number of cost-effectiveness analysis of influenza vaccination have been conducted to estimate value of influenza vaccines in elderly and health workers (HWs). This study aims to summarize cost-effectiveness evidence by pooling the incremental net monetary benefit (INMB) of influenza vaccination.MethodsA systematic review was performed in electronic databases from their inceptions to February 2022. Cost-effectiveness studies reporting quality-adjusted life year (QALY), or life year (LY) of influenza vaccination were included. Stratified meta-analyses by population, perspective, country income-level, and herd-effect were performed to pool INMB across studies. The protocol was registered at PROSPERO (CRD42021246746).FindingsA total of 21 studies were included. Eighteen studies were conducted in elderly, two studies were conducted in HWs, and one study was conducted in both elderly and HWs. According to pre-specified analyses, studies for elderly in high-income economies (countries) (HIEs) and upper-middle income economies (UMIEs) without herd effect could be pooled. For HIEs under a societal perspective, the perspective which identify all relevant costs occurred in the society including direct medical cost, direct non-medical cost and indirect cost, pooled INMB was $217·38 (206·23, 228·53, I2 =28.2%), while that for healthcare provider/payer perspective was $0·20 (-11,908·67, 11,909·07, I2 = 0.0%). For societal perspective in UMIEs, pooled INMB was $28·39 (-190·65, 133·87, I2 = 92.8%). The findings were robust across a series of sensitivity analyses for HIEs. Studies in HWs indicated that influenza vaccination was cost-effective compared to no vaccination or current practice.InterpretationInfluenza vaccination might be cost-effective for HWs and elderly in HIEs under a societal perspective with relatively small variations among included studies, while there remains limited evidence for healthcare provider/payer perspective or other level of incomes. Further evidence is warranted.FundingThis study was funded by a grant of Immunization, Vaccine and Biologicals department of the World Health Organization. The authors would like to acknowledge the contributions of the US CDC which provided financial support to the development and publication of this report. Grant number US CDC, WHO IVR (U50CK000431).

Dataset Information

Unremarked or Unperformed? Systematic Review on Reporting of Validation Efforts of Health Economic Decision Models in Seasonal Influenza and Early Breast Cancer.

Publications

Unremarked or Unperformed? Systematic Review on Reporting of Validation Efforts of Health Economic Decision Models in Seasonal Influenza and Early Breast Cancer.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets