Dataset Information

Criteria and Scoring Functions Used in Multi-criteria Decision Analysis and Value Frameworks for the Assessment of Rare Disease Therapies: A Systematic Literature Review.

ABSTRACT:

Background

Traditionally, the economic value of health technologies is assessed with cost-effectiveness (CE) and budget impact (BI) analyses. However, the evaluation of rare disease therapies often considers novel value criteria. Multi-criteria decision analysis (MCDA) is a promising tool in the assessment of value criteria that typically cannot be captured with traditional approaches.

Objectives

The objective of this research was to investigate the criteria and scoring functions applied in value frameworks and MCDA tools relevant to the evaluation of rare disease therapies. The aim was to gain a better understanding of the domains and measurement of commonly referenced novel value criteria.

Methods

A systematic literature review was performed covering the period from 2013 to 2019. MCDA or value framework articles and structured review papers on orphan-drug-specific MCDA articles were reviewed. Information sources included MEDLINE, Embase, Scopus, and 26 other gray literature sources. A descriptive review of identified criteria and scoring functions was performed, with special focus on "novel" value criteria that are traditionally not considered in CE or BI analyses.

Results

In total, 15 relevant value frameworks and MCDA tools were identified. These studies included a large number (n = 56) of individual value criteria. The most commonly included novel criteria were unmet medical need, severity of disease, and reduction in uncertainty. The identified scoring functions (measurement methods) for novel criteria were highly heterogeneous and tailored. Standardized scoring functions were not observed. Additionally, the studies did not provide their rationale for choosing a specific scoring function for a criterion.

Conclusions

MCDA is a promising tool to include novel value criteria into the health technology assessment of therapies for rare diseases. To support the development of a transparent and justified evaluation process, scoring functions should be further investigated.

SUBMITTER: Zelei T

PROVIDER: S-EPMC8611126 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Background: Increasingly, multi-criteria decision analysis has gained importance as a method by which to assess the value of orphan drugs. However, very little attention has been given to the weight (relative preferences) of the individual criteria used in a framework. Aims: This study sought to gain an understanding of the preferential weights that should be allocated in a multi-criteria decision analysis framework for orphan drugs, from a multi-stakeholder perspective. Method: Using key MCDA criteria for orphan drugs reported in the literature, we developed an interactive web-based survey tool to capture preferences for different criteria from a general stakeholder sample who were requested to assign weights from a reimbursement perspective. Each criterion could be assigned a weight on a sliding scale from 0 to 100% as long as the sum of all the criteria was 100%. We subsequently used the interactive tool with an expert focus group, followed up with a group discussion regarding each criterion and their perspectives on the weight that each criterion should be allocated when assessing an orphan drug. The expert focus group participants were then able to adjust their weights, if the group discussion had changed their perspectives. Results: The interactive tool was completed by 120 general stakeholder sample from a wide range of countries and professional backgrounds and an expert focus group of ten members. The results showed the differences in perspectives on the importance of criteria. Both groups considered Treatment efficacy to be the most important criterion. The general stakeholder sample weighted Treatment safety at 12.03% compared to the expert focus group's average of 20%. The results also demonstrated the value of the group discussion, which provided additional insights into the perspectives on the importance of criteria in assessing orphan drugs. Conclusion: This study aimed to contribute to the important aspect of preferences for different criteria in MCDA. This study sheds light on the important aspect of the preferences of the different criteria. All respondents agreed on the relative importance of Treatment efficacy and Treatment safety, criteria that are captured in conventional cost-effectiveness studies, but they also expressed the view that in addition to those, several disease-related and drug-related criteria should be included in MCDA frameworks for assessing orphan drugs.

Project description:BackgroundAdvances in targeted therapy development and tumor sequencing technology are reclassifying cancers into smaller biomarker-defined diseases. Randomized controlled trials (RCTs) are often impractical in rare diseases, leading to calls for single-arm studies to be sufficient to inform clinical practice based on a strong biological rationale. However, without RCTs, favorable outcomes are often attributed to therapy but may be due to a more indolent disease course or other biases. When the clinical benefit of targeted therapy in a common cancer is established in RCTs, this benefit may extend to rarer cancers sharing the same biomarker. However, careful consideration of the appropriateness of extending the existing trial evidence beyond specific cancer types is required. A framework for extrapolating evidence for biomarker-targeted therapies to rare cancers is needed to support transparent decision-making.ObjectivesTo construct a framework outlining the breadth of criteria essential for extrapolating evidence for a biomarker-targeted therapy generated from RCTs in common cancers to different rare cancers sharing the same biomarker.DesignA series of questions articulating essential criteria for extrapolation.MethodsThe framework was developed from the core topics for extrapolation identified from a previous scoping review of methodological guidance. Principles for extrapolation outlined in guidance documents from the European Medicines Agency, the US Food and Drug Administration, and Australia's Medical Services Advisory Committee were incorporated.ResultsWe propose a framework for assessing key assumptions of similarity of the disease and treatment outcomes between the common and rare cancer for five essential components: prognosis of the biomarker-defined cancer, biomarker test analytical validity, biomarker actionability, treatment efficacy, and safety. Knowledge gaps identified can be used to prioritize future studies.ConclusionThis framework will allow systematic assessment, standardize regulatory, reimbursement and clinical decision-making, and facilitate transparent discussions between key stakeholders in drug assessment for rare biomarker-defined cancers.

Project description:Gene-gene epistatic interactions likely play an important role in the genetic basis of many common diseases. Recently, machine-learning and data mining methods have been developed for learning epistatic relationships from data. A well-known combinatorial method that has been successfully applied for detecting epistasis is Multifactor Dimensionality Reduction (MDR). Jiang et al. created a combinatorial epistasis learning method called BNMBL to learn Bayesian network (BN) epistatic models. They compared BNMBL to MDR using simulated data sets. Each of these data sets was generated from a model that associates two SNPs with a disease and includes 18 unrelated SNPs. For each data set, BNMBL and MDR were used to score all 2-SNP models, and BNMBL learned significantly more correct models. In real data sets, we ordinarily do not know the number of SNPs that influence phenotype. BNMBL may not perform as well if we also scored models containing more than two SNPs. Furthermore, a number of other BN scoring criteria have been developed. They may detect epistatic interactions even better than BNMBL.Although BNs are a promising tool for learning epistatic relationships from data, we cannot confidently use them in this domain until we determine which scoring criteria work best or even well when we try learning the correct model without knowledge of the number of SNPs in that model.We evaluated the performance of 22 BN scoring criteria using 28,000 simulated data sets and a real Alzheimer's GWAS data set. Our results were surprising in that the Bayesian scoring criterion with large values of a hyperparameter called ? performed best. This score performed better than other BN scoring criteria and MDR at recall using simulated data sets, at detecting the hardest-to-detect models using simulated data sets, and at substantiating previous results using the real Alzheimer's data set.We conclude that representing epistatic interactions using BN models and scoring them using a BN scoring criterion holds promise for identifying epistatic genetic variants in data. In particular, the Bayesian scoring criterion with large values of a hyperparameter ? appears more promising than a number of alternatives.

Project description:BackgroundRare superficial vascular anomalies represent a wide range of diseases. Their management is difficult given the broad spectrum and the lack of clinical trials assessing treatment efficacy. A randomized clinical trial of vascular anomalies is difficult because of the rarity of the diseases and is enhanced by the population of interest often being children. Therefore, suitable designs are needed. We conducted a methodological systematic literature search to identify designs implemented for investigating the treatment of rare superficial vascular anomalies.MethodsWe conducted a literature search on January 25, 2021, of the PubMed, Cochrane Central Register of Controlled Trials (CENTRAL), Embase, ClinicalTrials.gov and European Union Clinical Trials Register databases. This systematic methodological literature search was registered at the Prospective Register of Systematic Reviews (PROSPERO: CRD42021232449). Randomized and non-randomized studies were included if they met the following criteria: were prospective studies of rare superficial vascular anomaly therapies, dealt with humans (adults and children) and were published in English from 2000. We excluded case reports/case series reporting fewer than 10 patients, reviews, retrospective studies, animal studies, studies of systemic or common vascular anomalies and non-therapeutic studies. We did not assess risk of bias in the included studies because our review was a methodological one focused on the design used. The review provided a descriptive analysis of relevant features of eligible research studies.ResultsFrom 2046 articles identified, we included 97 studies (62 reports and 35 ongoing studies): 25 randomized controlled studies, 7 non-randomized comparative studies, 64 prospective cohorts and 1 case series. Among the 32 comparative studies included, 21 used a parallel-group design. The 11 other studies used different designs such as cross-over, randomized placebo phase, delayed-start, within-person, or challenge-dechallenge-rechallenge or used a historical control group or an observational run-in period.ConclusionsOur systematic literature search highlights the lack of randomized control trials in superficial vascular anomalies due to the rarity of patients and their heterogeneity. New designs are emerging and can overcome the limitations of testing treatments in parallel groups.

Project description:BackgroundDemonstrating safety and efficacy of new medical treatments requires clinical trials but clinical trials are costly and may not provide value proportionate to their costs. As most health systems have limited resources, it is therefore important to identify the trials with the highest value. Tools exist to assess elements of a clinical trial such as statistical validity but are not wholistic in their valuation of a clinical trial. This study aims to develop a measure of clinical trials value and provide an online tool for clinical trial prioritisation.MethodsA search of the academic and grey literature and stakeholder consultation was undertaken to identify a set of criteria to aid clinical trial valuation using multi-criteria decision analysis. Swing weighting and ranking exercises were used to calculate appropriate weights of each of the included criteria and to estimate the partial-value function for each underlying metric. The set of criteria and their respective weights were applied to the results of six different clinical trials to calculate their value.ResultsSeven criteria were identified: 'unmet need', 'size of target population', 'eligible participants can access the trial', 'patient outcomes', 'total trial cost', 'academic impact' and 'use of trial results'. The survey had 80 complete sets of responses (51% response rate). A trial designed to address an 'Unmet Need' was most commonly ranked as the most important with a weight of 24.4%, followed by trials demonstrating improved 'Patient Outcomes' with a weight of 21.2%. The value calculated for each trial allowed for their clear delineation and thus a final value ranking for each of the six trials.ConclusionWe confirmed that the use of the decision tool for valuing clinical trials is feasible and that the results are face valid based on the evaluation of six trials. A proof-of-concept applying this tool to a larger set of trials with an external validation is currently underway.