Dataset Information

A Review of Issues Affecting the Efficiency of Decision Making in the NICE Single Technology Appraisal Process.

ABSTRACT:

Background

Escalating demands upon the National Institute for Health and Care Excellence's (NICE's) Single Technology Appraisal (STA) programme require a 2.5-times increase upon 2015 capacity by 2020. This additional strain on committee resources threatens to compromise the rigour of the STA process. In 2018, NICE introduced changes to the appraisal process, aiming to expedite final decisions, including consultation opportunities prior to the company's evidence submission, a 'Technical Engagement' stage prior to the first committee meeting, and powers for committee chairs to recommend technologies without a second formal meeting.

Objective

This study reviews recent STAs and aims to understand why appraisals require multiple meetings, and whether recent reforms can address the underlying issues.

Methods

NICE STAs published between January 2010 and January 2018 were reviewed, excluding updates or re-considerations. Data on cost, clinical, and decision-making outcomes from 146 appraisals were extracted and analysed thematically.

Results

Drugs for advanced cancers were least likely to be recommended (28/43 [65.1%] vs 71/74 [96%] for non-cancer) and took longer (2.36 meetings for a final decision vs 1.97 for non-cancer). The academic review increased upon the company's base-case incremental cost-effectiveness ratio by a median of 32.7%. Eighty-four technologies (57.5%) received a negative preliminary recommendation, deferring a final decision by an average of 142 days. Of these, 85.1% were not considered cost-effective. Uncertainty in economic (34.3%) and clinical (22.3%) data also prevented a positive decision. The majority (72.6% [61/84]) of negative preliminary decisions were overturned following further committee discussion; important considerations were Patient Access Schemes, decision optimisation, and the Cancer Drugs Fund.

Conclusions

Value considerations are the primary driver of negative preliminary recommendations. It is unclear if new opportunities for additional interaction between NICE, review groups, and manufacturers will meaningfully improve the efficiency of the appraisal process, particularly given the proportion of technologies requiring further committee discussion for decision optimisation or admission into the CDF.

SUBMITTER: Walton MJ

PROVIDER: S-EPMC6710310 | biostudies-literature | 2019 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Review of Issues Affecting the Efficiency of Decision Making in the NICE Single Technology Appraisal Process.

Walton M J MJ O'Connor J J Carroll C C Claxton L L Hodgson R R

PharmacoEconomics - open 20190901 3

<h4>Background</h4>Escalating demands upon the National Institute for Health and Care Excellence's (NICE's) Single Technology Appraisal (STA) programme require a 2.5-times increase upon 2015 capacity by 2020. This additional strain on committee resources threatens to compromise the rigour of the STA process. In 2018, NICE introduced changes to the appraisal process, aiming to expedite final decisions, including consultation opportunities prior to the company's evidence submission, a 'Technical E ...[more]

PMID: 30617953

Similar Datasets

Project description:The National Institute for Health and Care Excellence (NICE) invited Alimera Sciences, the company manufacturing fluocinolone acetonide intravitreal implant (FAc) 0.19 mg (tradename ILUVIEN®), to submit evidence on the clinical and cost-effectiveness of FAc for treating recurrent non-infectious uveitis. Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre?+?, was commissioned to act as the independent Evidence Review Group (ERG). This paper contains a summary of the clinical and cost-effectiveness evidence submitted by the company, the ERG's critique on the submitted evidence, and the guidance issued by the NICE Appraisal Committee (AC). The company submission (CS) was mainly informed by the PSV-FAI-001 trial in which FAc was compared with (limited) current practice [(L)CP], which was not considered to be representative of UK clinical practice by the ERG. There was no comparison of FAc to any treatment listed in the final scope, and especially to the dexamethasone intravitreal implant (dexamethasone), which was considered to be a relevant comparator by the AC. The primary outcome of the PSV-FAI-001 was recurrence of uveitis in the treated eye. Most of the events for the primary outcome were imputed during the PSV-FAI-001 trial, which probably led to an overestimation of the number of recurrences of disease, and a biased estimate of the relative effectiveness of FAc versus (L)CP. Finally, the place of FAc in the treatment pathway was not clearly defined by the company. Substantial uncertainty surrounded the cost-effectiveness results due to the shortcomings of the clinical evidence. Additionally, the quality of life of patients was not measured during the PSV-FAI-001 trial and long-term effectiveness data of FAc were lacking. The ERG adjusted several issues identified in the CS and added dexamethasone as a comparator in the decision analytic model. The ERG presented multiple analyses as base-cases because several elements of the assessment remained uncertain. The fully incremental ERG results ranged from dexamethasone (extendedly) dominating FAc (when assuming a hazard ratio of 1 or 0.7 for dexamethasone versus FAc) to an incremental cost-effectiveness ratio (ICER) of £30,153 per quality-adjusted life-year (QALY) gained for FAc versus (L)CP [when assuming a hazard ratio of 0.456 for dexamethasone versus (L)CP]. The ICER of FAc versus (L)CP ranged from £12,325 to £30,153 per QALY gained. After a second AC meeting where alternative company scenarios comparing FAc with dexamethasone were considered by the AC, the AC concluded that "the results of the company's analyses ranged from the fluocinolone acetonide implant being dominant (that is, it was more effective and costs less), to an ICER of £29,461 per QALY gained, and most of the ICERs were below £20,000 per QALY gained". Therefore, the AC recommended FAc as a cost-effective use of National Health Service (NHS) resources for treating recurrent non-infectious uveitis affecting the posterior segment of the eye in the final TA590 guidance (published July 2019).

Project description:The National Institute for Health and Care Excellence (NICE) invited the manufacturer (Celgene) of oral azacitidine (ONUREG), as part of the Single Technology Appraisal (STA) process, to submit evidence for the clinical effectiveness and cost-effectiveness of oral azacitidine for maintenance treatment of acute myeloid leukaemia (AML) after induction therapy compared with watch-and-wait plus best supportive care (BSC) and midostaurin. Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre+, was commissioned to act as the independent Evidence Review Group (ERG). This paper summarises the company submission (CS), presents the ERG's critical review on the clinical and cost-effectiveness evidence in the CS, highlights the key methodological considerations and describes the development of the NICE guidance by the Appraisal Committee. In the QUAZAR AML-001 trial, oral azacitidine significantly improved overall survival (OS) versus placebo: median OS gain of 9.9 months (24.7 months versus 14.8 months; hazard ratio (HR) 0.69 (95% CI 0.55-0.86), p < 0.001). The median time to relapse was also better for oral azacitidine, and the incidences of TEAEs were similar for the two arms. The company excluded two of the comparators listed in the scope, low-dose cytarabine and subcutaneous azacitidine, informed only by clinical expert opinion, leaving only best supportive care (BSC) and midostaurin for the FLT3-ITD and/or FLT3-TKD (FLT3 mutation)-positive subgroup. An ITC comparing oral azacitidine to midostaurin as maintenance therapy in the appropriate subgroup demonstrated that the OS and relapse-free survival (RFS) HRs were favourable for oral azacitidine when compared with midostaurin. However, in the only available trial of midostaurin as maintenance treatment in AML that was used for this ITC, subjects were not randomised at the maintenance phase, but at induction, which posed a substantial risk of bias. The revised and final probabilistic incremental cost-effectiveness ratio (ICER) presented by the company, including a commercial arrangement, was £32,480 per quality-adjusted life year (QALY) gained for oral azacitidine versus watch-and-wait plus BSC. Oral azacitidine was dominant versus midostaurin in the FLT-3 subgroup. The ERG's concerns included the approach of modelling haematopoietic stem cell transplantation (HSCT), the generalisability of the population and the number of cycles of consolidation therapy pre-treatment in the QUAZAR AML-001 trial to UK clinical practice, and uncertainty in the relapse utility. The revised and final ERG base case resulted in a similar probabilistic ICER of £33,830 per QALY gained versus watch-and-wait plus BSC, but with remaining uncertainty. Oral azacitidine remained dominant versus midostaurin in the FLT-3 subgroup. After the second NICE appraisal committee meeting, the NICE Appraisal Committee recommended oral azacitidine (according to the commercial arrangement), within its marketing authorisation, as an option for maintenance treatment for AML in adults who are in complete remission, or complete remission with incomplete blood count recovery, after induction therapy with or without consolidation treatment, and cannot have or do not want HSCT.

Project description:It has widely been accepted that aggregating group-level decisions is superior to individual decisions. As compared to individuals, groups tend to show a decision advantage in their response accuracy. However, there has been a lack of research exploring whether group decisions are more efficient than individual decisions with a faster information-processing speed. To investigate the relationship between accuracy and response time (RT) in group decision-making, we applied systems' factorial technology, developed by Townsend and Nozawa (Journal of Mathematical Psychology 39, 321-359, 1995) and regarded as a theory-driven methodology, to study the information-processing properties. More specifically, we measured the workload capacity CAND(t), which only considers the correct responses, and the assessment function of capacity AAND(t), which considers the speed-accuracy trade-off, to make a strong inference about the system-level processing efficiency. A two-interval, forced-choice oddball detection task, where participants had to detect which interval contains an odd target, was conducted in Experiment 1. Then, in Experiment 2, a yes/no Gabor detection task was adopted, where participants had to detect the presence of a Gabor patch. Our results replicated previous findings using the accuracy-based measure: Group detection sensitivity was better than the detection sensitivity of the best individual, especially when the two individuals had similar detection sensitivities. On the other hand, both workload capacity measures, CAND(t) and AAND(t), showed evidence of supercapacity processing, thus suggesting a collective benefit. The ordered relationship between accuracy-based and RT-based collective benefit was limited to the AAND(t) of the correct and fast responses, which may help uncover the processing mechanism behind collective benefits. Our results suggested that AAND(t), which combines both accuracy and RT into inferences, can be regarded as a novel and diagnostic tool for studying the group decision-making process.

Project description:The National Institute for Health and Care Excellence (NICE), as part of the institute's single technology appraisal (STA) process, invited the company that makes obinutuzumab (Roche Products Limited) to submit evidence of the clinical and cost effectiveness of the drug in combination with chemotherapy, with or without obinutuzumab as maintenance therapy for adult patients with untreated, advanced follicular lymphoma (FL) in the UK. Kleijnen Systematic Reviews Ltd (KSR), in collaboration with Erasmus University Rotterdam, was commissioned to act as the Evidence Review Group (ERG). This paper describes the company's submission, the ERG review, and NICE's subsequent decisions. The clinical evidence was derived from two phase III, company-sponsored, randomised, open-label studies. Most evidence on obinutuzumab was based on the GALLIUM trial that compared obinutuzumab in combination with chemotherapy as induction followed by obinutuzumab maintenance monotherapy with rituximab in combination with chemotherapy as induction followed by rituximab maintenance monotherapy in previously untreated patients with FL (grades 1-3a). Long-term clinical evidence was based on the PRIMA trial, studying the benefit of two years of rituximab maintenance after first-line treatment in patients with FL. The cost-effectiveness evidence submitted by the company relied on a partitioned survival cost-utility model, implemented in Microsoft® Excel. The base-case incremental cost-effectiveness ratio (ICER) presented in the company submission was <£20,000 per quality-adjusted life-year (QALY) gained. Although the ERG concluded that the economic model met the NICE reference case to a reasonable extent, some errors were identified and several assumptions made by the company were challenged. A new base-case scenario produced by the ERG suggested an ICER that was higher than the company base case, but still below £30,000 per QALY gained. However, some ERG scenario analyses were close to or even above the threshold. This was the case in particular for assuming a treatment effect that did not extend beyond trial follow-up. These results led to an initial negative recommendation by the appraisal committee. Subsequently, the company submitted a revised base case focusing on patients at intermediate or high risk of premature mortality. Simultaneously, a further price discount for obinutuzumab was granted. In addition to the company's revised base case, the ERG suggested a restriction of the treatment effect to 5 years and implemented biosimilar uptake and cheaper prices for rituximab. All of these adjustments did not exceed £30,000 per QALY gained and therefore the use of obinutuzumab for patients with advanced FL and a Follicular Lymphoma International Predictive Index (FLIPI) score of two or more could be recommended.

Project description:ObjectivesTo assess how long the UK's National Institute for Health and Clinical Excellence's (NICE) Technology Appraisal Programme has taken to produce guidance and to determine independent predictors of time to guidance.DesignRetrospective time to event (survival) analysis.SettingTechnology Appraisal guidance produced by NICE. DATASOURCE: All appraisals referred to NICE by February 2010 were included, except those referred prior to 2001 and a number that were suspended.Outcome measureDuration from the start of an appraisal (when the scope document was released) until publication of guidance.ResultsSingle Technology Appraisals (STAs) were published significantly faster than Multiple Technology Appraisals (MTAs) with median durations of 48.0 (IQR; 44.3-75.4) and 74.0 (IQR; 60.9-114.0) weeks, respectively (p?<0.0001). Median time to publication exceeded published process timelines, even after adjusting for appeals. Results from the modelling suggest that STAs published guidance significantly faster than MTAs after adjusting for other covariates (by 36.2 weeks (95% CI -46.05 to -26.42 weeks)) and that appeals against provisional guidance significantly increased the time to publication (by 42.83 weeks (95% CI 35.50 to 50.17 weeks)). There was no evidence that STAs of cancer-related technologies took longer to complete compared with STAs of other technologies after adjusting for potentially confounding variables and only weak evidence suggesting that the time to produce guidance is increasing each year (by 1.40 weeks (95% CI -0.35 to 2.94 weeks)).ConclusionsThe results from this study suggest that the STA process has resulted in significantly faster guidance compared with the MTA process irrespective of the topic, but that these gains are lost if appeals are made against provisional guidance. While NICE processes continue to evolve over time, a trade-off might be that decisions take longer but at present there is no evidence of a significant increase in duration.

Dataset Information

A Review of Issues Affecting the Efficiency of Decision Making in the NICE Single Technology Appraisal Process.

Background

Objective

Methods

Results

Conclusions

Publications

A Review of Issues Affecting the Efficiency of Decision Making in the NICE Single Technology Appraisal Process.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets