Dataset Information

AdViSHE: A Validation-Assessment Tool of Health-Economic Models for Decision Makers and Model Users.

ABSTRACT:

Background

A trade-off exists between building confidence in health-economic (HE) decision models and the use of scarce resources. We aimed to create a practical tool providing model users with a structured view into the validation status of HE decision models, to address this trade-off.

Methods

A Delphi panel was organized, and was completed by a workshop during an international conference. The proposed tool was constructed iteratively based on comments from, and the discussion amongst, panellists. During the Delphi process, comments were solicited on the importance and feasibility of possible validation techniques for modellers, their relevance for decision makers, and the overall structure and formulation in the tool.

Results

The panel consisted of 47 experts in HE modelling and HE decision making from various professional and international backgrounds. In addition, 50 discussants actively engaged in the discussion at the conference workshop and returned 19 questionnaires with additional comments. The final version consists of 13 items covering all relevant aspects of HE decision models: the conceptual model, the input data, the implemented software program, and the model outcomes.

Conclusions

Assessment of the Validation Status of Health-Economic decision models (AdViSHE) is a validation-assessment tool in which model developers report in a systematic way both on validation efforts performed and on their outcomes. Subsequently, model users can establish whether confidence in the model is justified or whether additional validation efforts should be undertaken. In this way, AdViSHE enhances transparency of the validation status of HE models and supports efficient model validation.

SUBMITTER: Vemer P

PROVIDER: S-EPMC4796331 | biostudies-literature | 2016 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

AdViSHE: A Validation-Assessment Tool of Health-Economic Models for Decision Makers and Model Users.

Vemer P P Corro Ramos I I van Voorn G A K GA Al M J MJ Feenstra T L TL

PharmacoEconomics 20160401 4

<h4>Background</h4>A trade-off exists between building confidence in health-economic (HE) decision models and the use of scarce resources. We aimed to create a practical tool providing model users with a structured view into the validation status of HE decision models, to address this trade-off.<h4>Methods</h4>A Delphi panel was organized, and was completed by a workshop during an international conference. The proposed tool was constructed iteratively based on comments from, and the discussion a ...[more]

PMID: 26660529

Similar Datasets

Project description:BackgroundPromising health interventions tested in pilot studies will only achieve population-wide impact if they are implemented at scale across communities and health systems. Scaling up effective health interventions is vital as not doing so denies the community the most effective services and programmes. However, there remains a paucity of practical tools to assess the suitability of health interventions for scale-up. The Intervention Scalability Assessment Tool (ISAT) was developed to support policy-makers and practitioners to make systematic assessments of the suitability of health interventions for scale-up.MethodsThe ISAT was developed over three stages; the first stage involved a literature review to identify similar tools and frameworks that could be used to guide scalability assessments, and expert input to develop draft ISAT content. In the second stage, the draft ISAT tool was tested with end users. The third stage involved revising and re-testing the ISAT with end users to further refine the language and structure of the final ISAT.ResultsA variety of information and sources of evidence should be used to complete the ISAT. The ISAT consists of three parts. Part A: 'setting the scene' requires consideration of the context in which the intervention is being considered for scale-up and consists of five domains, as follows: (1) the problem; (2) the intervention; (3) strategic/political context; (4) evidence of effectiveness; and (5) intervention costs and benefits. Part B asks users to assess the potential implementation and scale-up requirements within five domains, namely (1) fidelity and adaptation; (2) reach and acceptability; (3) delivery setting and workforce; (4) implementation infrastructure; and (5) sustainability. Part C generates a graphical representation of the strengths and weaknesses of the readiness of the proposed intervention for scale-up. Users are also prompted for a recommendation as to whether the intervention (1) is recommended for scale-up, (2) is promising but needs further information before scaling up, or (3) does not yet merit scale-up.ConclusionThe ISAT fills an important gap in applied scalability assessment and can become a critical decision support tool for policy-makers and practitioners when selecting health interventions for scale-up. Although the ISAT is designed to be a health policy and practitioner tool, it can also be used by researchers in the design of research to fill important evidence gaps.

Project description:Background. Metamodels can simplify complex health policy models and yield instantaneous results to inform policy decisions. We investigated the predictive validity of linear regression metamodels used to support a real-time decision-making tool that compares infant HIV testing/screening strategies. Methods. We developed linear regression metamodels of the Cost-Effectiveness of Preventing AIDS Complications Pediatric (CEPAC-P) microsimulation model used to predict life expectancy and lifetime HIV-related costs/person of two infant HIV testing/screening programs in South Africa. Metamodel performance was assessed with cross-validation and Bland-Altman plots, showing between-method differences in predicted outcomes against their means. Predictive validity was determined by the percentage of simulations in which the metamodels accurately predicted the strategy with the greatest net health benefit (NHB) as projected by the CEPAC-P model. We introduced a zone of indifference and investigated the width needed to produce between-method agreement in 95% of the simulations. We also calculated NHB losses from "wrong" decisions by the metamodel. Results. In cross-validation, linear regression metamodels accurately approximated CEPAC-P-projected outcomes. For life expectancy, Bland-Altman plots showed good agreement between CEPAC-P and the metamodel (within 1.1 life-months difference). For costs, 95% of between-method differences were within $65/person. The metamodels predicted the same optimal strategy as the CEPAC-P model in 87.7% of simulations, increasing to 95% with a zone of indifference of 0.24 life-months ( ? 7 days). The losses in health benefits due to "wrong" choices by the metamodel were modest (range: 0.0002-1.1 life-months). Conclusions. For this policy question, linear regression metamodels offered sufficient predictive validity for the optimal testing strategy as compared with the CEPAC-P model. Metamodels can simulate different scenarios in real time, based on sets of input parameters that can be depicted in a widely accessible decision-support tool.

Project description:BackgroundSurrogate decision makers of stroke patients are often unprepared to make critical decisions on life-sustaining treatments. We describe the development process and key features for the Understanding Stroke web-based decision support tool.MethodsWe used multiple strategies to develop a patient-centered, tailored decision aid. We began by forming a Patient and Family Advisory Council to provide continuous input to our multidisciplinary team on the development of the tool. Additionally, focus groups consisting of nurses, therapists, social workers, physicians, stroke survivors, and family members reviewed key elements of the tool, including prognostic information, graphical displays, and values clarification exercise. To design the values clarification exercise, we asked focus groups to provide feedback on a list of important activities of daily living. An ordinal prognostic model was developed for ischemic stroke and intracerebral hemorrhage using data taken from the Virtual International Stroke Trials Archive Plus, and incorporated into the tool.ResultsFocus group participants recommended making numeric prognostic information optional due to possible emotional distress. Pie charts were generally favored by participants for graphical presentation of prognostic information, though a horizontal stacked bar chart was also added due to its prevalence in stroke literature. Plain language descriptions of the modified Rankin Scale were created to accompany the prognostic information. A values clarification exercise was developed consisting of a list of 13 situations that may make an individual consider comfort measures only. The final version of the web based tool (which can be viewed on tablets) included the following sections: general introduction to stroke, outcomes (prognostic information and recovery), in-hospital and life-sustaining treatments, decision making and values clarification, post-hospital care, tips for talking to the health care team, and a summary report. Preliminary usability testing received generally favorable feedback.ConclusionWe developed Understanding Stroke, a tailored decision support tool for surrogate decision makers of stroke patients. The tool was well received and will be formally pilot tested in a group of stroke surrogate decision makers.Trial registrationClinicalTrials.gov ( NCT03427645 ).

Project description:BackgroundContinued improvements in occupational health can only be ensured if decisions regarding the implementation and continuation of occupational health and safety interventions (OHS interventions) are based on the best available evidence. To ensure that this is the case, scientific evidence should meet the needs of decision-makers. As a first step in bridging the gap between the economic evaluation literature and daily practice in occupational health, this study aimed to provide insight into the occupational health decision-making process and information needs of decision-makers.MethodsAn exploratory qualitative study was conducted with a purposeful sample of occupational health decision-makers in the Ontario healthcare sector. Eighteen in-depth interviews were conducted to explore the process by which occupational health decisions are made and the importance given to the financial implications of OHS interventions. Twenty-five structured telephone interviews were conducted to explore the sources of information used during the decision-making process, and decision-makers' knowledge on economic evaluation methods. In-depth interview data were analyzed according to the constant comparative method. For the structured telephone interviews, summary statistics were prepared.ResultsThe occupational health decision-making process generally consists of three stages: initiation stage, establishing the need for an intervention; pre-implementation stage, developing an intervention and its business case in order to receive senior management approval; and implementation and evaluation stage, implementing and evaluating an intervention. During this process, information on the financial implications of OHS interventions was found to be of great importance, especially the employer's costs and benefits. However, scientific evidence was rarely consulted, sound ex-post program evaluations were hardly ever performed, and there seemed to be a need to advance the economic evaluation skill set of decision-makers.ConclusionsFinancial information is particularly important at the front end of implementation decisions, and can be a key deciding factor of whether to go forward with a new OHS intervention. In addition, it appears that current practice in occupational health in the healthcare sector is not solidly grounded in evidence-based decision-making and strategies should be developed to improve this.

Project description:BackgroundSystems science methods such as dynamic simulation modelling are well suited to address questions about public health policy as they consider the complexity, context and dynamic nature of system-wide behaviours. Advances in technology have led to increased accessibility and interest in systems methods to address complex health policy issues. However, the involvement of policy decision makers in health-related simulation model development has been lacking. Where end-users have been included, there has been limited examination of their experience of the participatory modelling process and their views about the utility of the findings. This paper reports the experience of end-user decision makers, including senior public health policy makers and health service providers, who participated in three participatory simulation modelling for health policy case studies (alcohol related harm, childhood obesity prevention, diabetes in pregnancy), and their perceptions of the value and efficacy of this method in an applied health sector context.MethodsSemi-structured interviews were conducted with end-user participants from three participatory simulation modelling case studies in Australian real-world policy settings. Interviewees were employees of government agencies with jurisdiction over policy and program decisions and were purposively selected to include perspectives at different stages of model development.ResultsThe 'co-production' aspect of the participatory approach was highly valued. It was reported as an essential component of building understanding of the modelling process, and thus trust in the model and its outputs as a decision-support tool. The unique benefits of simulation modelling included its capacity to explore interactions of risk factors and combined interventions, and the impact of scaling up interventions. Participants also valued simulating new interventions prior to implementation in the real world, and the comprehensive mapping of evidence and its gaps to prioritise future research. The participatory aspect of simulation modelling was time and resource intensive and therefore most suited to high priority complex topics with contested options for intervening.ConclusionThese findings highlight the value of a participatory approach to dynamic simulation modelling to support its utility in applied health policy settings.

Dataset Information

AdViSHE: A Validation-Assessment Tool of Health-Economic Models for Decision Makers and Model Users.

Background

Methods

Results

Conclusions

Publications

AdViSHE: A Validation-Assessment Tool of Health-Economic Models for Decision Makers and Model Users.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets