How NICE there’s an app for that!
By Jo Watts-James, MBA
In 2019, the National Institute for Health and Care Excellence (NICE) published its Evidence Standards Framework (ESF) for digital health technologies (DHTs). This helps identify technologies that are likely to offer benefits to users and the healthcare system, establishes a classification system to assign them to one of the tiers of standards within the framework, and provides a set of standards to help developers understand what good levels of evidence for DHTs look like.
As a result of the explosion in new technologies coming to the market, particularly digital ones, in 2022, NICE announced a new early value assessment (EVA) approach to assess MedTech (digital products, devices, and diagnostics) that are most needed and in demand. EVA, specific to MedTech, allows rapid assessment of technologies for clinical effectiveness and value for money. The overall time frame of an early value assessment is approximately 6 months.
What's different about EVA?
EVA is an evidence-based approach that provides the National Health Service (NHS) with guidance about the value of MedTech technology, including an option for a recommendation for use while further evidence is generated. The main difference to the current NICE Medical Technologies Evaluation Programme is that technologies selected for EVA are not expected to have a complete evidence base before they’re recommended.
The initial topics selected for consideration include:
- Mental health: priority clinical areas
- Cardiovascular: predicting risk of heart failure
- Early cancer detection
- Medical technologies that boost healthcare capacity.
NICE states that whilst digital technologies play an important role in EVA, they are not intended to be the main focus; EVA is for all MedTech including devices, diagnostics, and therapeutics. However, of the initial 18 EVAs listed to date (October 2023), over 80% (15 EVAs) are for DHTs (Table 1). This is because the initial focus for topic selection was on rapidly identifying DHT's that would address system priorities in cardiovascular disease, mental health, and post-COVID system recovery.
Table 1. Type of technology assessed in EVAs listed to date by therapy area
| Non-digital | Digital | Total | |
|---|---|---|---|
| Antibiotic usage | 2 | 2 | |
| Cardiology | 1 | 1 | |
| Mental Health | 1 | 5 | 6 |
| Oncology | 4 | 4 | |
| Pain | 1 | 1 | |
| Respiratory disease | 2 | 2 | |
| Weight management | 2 | 2 | |
| Total | 3 | 15 | 18 |
Key: EVA – early value assessment.
Outcomes of 10 EVAs have been published to date, and 7 of them recommended all technologies be evaluated (Table 2).
Table 2. Outcomes for published EVAs
| Outcomes | Published EVAs | EVA topics |
|---|---|---|
| Recommended all the technologies assessed | 7 | - 2 Digital tools for anxiety - 1 Digital tool for depression - 1 Radiotherapy data cloud-based storage system - 1 Kit to test for genetic variant for antibiotic use - Multiple digital technologies for measuring vital signs in patients with acute respiratory infections - 1 Device to measure QT intervals in patients on antipsychotics |
| Recommended some but not all of the technologies assessed | 1 | - 12 Point-of-care technologies that can be used for testing in people with suspected UTIs |
| Did not recommend the technologies assesses | 2 | - Medical imaging analysis software that uses artificial intelligence (AI) to analyze images from CT coronary angiography (CTCA) - Artificial intelligence software to analyse X-rays for suspected lung cancer |
| Total | 10 |
Key: CT – computed tomography; EVA – early value assessment; UTI – urinary tract infection.
An 11th EVA for DHTs to manage back pain has also been recommended; guidance is being drafted and publication is expected in December 2023.
How is value assessed?
EVA is a new process that is still under development. NICE’s role will be to:
- Identify what additional evidence is needed for NICE to support using the technology in the NHS, and identify system partners who may be able to support the design and delivery of an evidence-generation plan
- Determine what should be included in an evidence-generation plan, including guidance on suitable approaches to generating evidence
- Develop a proportionate and pragmatic approach to evidence generation with consideration for the burden on patients, healthcare professionals, and the wider system
There is acknowledgement that evidence will be less robust than standard technology appraisal due to the timing of the assessment and the fact that MedTech is not subject to the same standards of evidence generation as pharmaceuticals. Despite this acknowledgement, EVA submissions are currently still expected to follow required evaluation standards as set out in the NICE process and methods manual for health technology evaluations (PMG36). NICE provides advice on how this can be applied to EVAs in the interim process and methods document for EVAs (PMG39).
Scoping
NICE states the evidence considered should be relevant to the evaluation in terms of patient groups, comparators, perspective, outcomes, and resource use as defined in the scope wherever possible. Scoping is therefore a key element of the evaluation.
Clinical evaluation
NICE interim guidance recommends the following for searches, ongoing studies, and evidence synthesis:
- Broad evidence mapping searches may need to be done to identify evidence on the technologies because studies may only be presented as conference abstracts, or may be published in less-well-known journals, so articles may not be well indexed. Unpublished data provided by companies and other stakeholders could be considered. To accommodate the timetable, pragmatic approaches are acceptable such as single-reviewer screening and data extraction.
- Searches for ongoing studies should also be done. Any ongoing studies that are identified should be checked to see if they will help to fill key evidence gaps. A full critical appraisal of studies using a validated tool is not needed, but there should be discussion on the potential biases in key studies and how the risk of bias could affect key outcomes. Comments on the generalisability of the results to clinical practice in the NHS should also be made.
- Evidence synthesis of the key findings should be provided in a simple narrative and descriptive format, a quantitative meta-analysis is not expected. The report (currently developed using the standard NICE template for single technology appraisal submissions during the pilot phase) should include a discussion of the evidence gaps and uncertainties in the identified evidence and suggestions of outcomes to focus on in future evidence generation.
Economic evaluation
EVAs are reviewed by the Medical Technologies Advisory Committee (MTAC) or the Diagnostics Advisory Committee (DAC) and aim to assess cost-effectiveness but with more flexibility than standard technology appraisals due to the lack of robust evidence at this early stage. NICE interim guidance says that the economic evaluation work that is likely to be most beneficial for committee decision-making will probably vary by topic, and that the external assessment groups doing the work will need to be flexible about what is presented.
NICE advises that proposed work should be discussed between the NICE team and the external assessment group from an early stage to inform development of the protocol for the assessment and that economic evaluation should as a minimum consider model structure(s) that would be needed for future analysis to support NICE committee decision-making.
The objectives of the economic evaluation are to:
- Identify likely impacts of using technologies (while further data is collected) on:
- People with the condition and, when relevant, carers
- The NHS and personal social services (including costs)
- Identify uncertainties that are likely to be key drivers of model results and uncertainties that would not be apparent from technology-related studies
The cost of reversing a decision to adopt the technologies while further evidence is generated may also be considered. This could include any costs that could not be recouped related to implementation such as fixed or up-front costs related to the purchase of equipment, training costs, or changes to organisation of care pathways.
Decision-making
When making decisions, the EVA interim guidance from NICE says the advisory committee should consider:
- The extent of the evidence that supports the likelihood of the technology addressing unmet need in the system
- The key evidence gaps where further data is needed to resolve uncertainty for future decision-making and whether further evidence generation will sufficiently resolve the key evidence gaps
- If any identified risks or uncertainties could be mitigated if the technology is used while further data is generated
- The likelihood and size of the impact of adopting the technologies for people using them (and their carers) and the NHS personal social services while further data is collected, in terms of both potential benefits and risks
DHT assessment: NICE vs other HTAs
NICE is not alone in Europe in assessing DHTs as France and Germany also assess them; however, only NICE carries out economic assessment. In Germany, the Digital Healthcare Act (Digitale-Versorgung-Gesetz [DVG]) came into effect on 19 December 2019 introducing the “app on prescription” as part of healthcare provided to patients. These apps can be prescribed by physicians and psychotherapists and are reimbursed by health insurers.
A prerequisite for this is that a DHT must have successfully completed the assessment of the Federal Institute for Drugs and Medical Devices (Bundesinstitut für Arzneimittel und Medizinprodukte [BfArM]) leading to a listing in a directory of reimbursable digital health applications (DiGA directory). The procedure is designed as a fast-track process: the BfArM has to assess the DHT within a 3-month period from filing of the complete application. A 2022 study showed that of the assessments completed since 2020 (N not stated), 30% achieved reimbursement, 7% had a negative outcome, and 63% of applications were withdrawn. Most were psychiatry interventions (53%). As of September 2023, 55 apps are listed in the DiGA directory.
In France, DHTs are assessed by the Medical Device and Health Technology Evaluation Committee (CNEDiMTS), using the same process as is used for all other medical devices. CNEDiMTS recommends medical device reimbursement by the National Health Insurance Fund and/or hospital use, through an assessment of their “actual clinical benefit” and likely “clinical added value” in comparison to available therapies. In the 2022 study mentioned above, CNEDiMTS reviewed 3 technologies and concluded one intervention had insufficient evidence, one provided moderate additional benefit, and one provided minor additional benefit.
Summary
Germany has assessed the most DHTs followed by the UK then France. In the UK, digital health technology evaluation is evolving; manufacturers should monitor the EVA process and learn how NICE assesses the value of MedTech so that they can generate the necessary evidence and gain rapid access to this ever-expanding market. It is likely these processes will evolve and that the evidence burden will increase for DHTs going forward.
DiGA. Cara Care für Reizdarm. Accessed 5 September 2023. https://diga.bfarm.de/de/verzeichnis
Federal Institute for Drugs and Medical Devices. The fast-track process for digital health applications (DiGA) according to Section 139e SGB V: a guide for manufacturers, service providers and users. Accessed 5 September 2023. https://www.bfarm.de/SharedDocs/Downloads/EN/MedicalDevices/DiGA_Guide.pdf?__blob=publicationFile
Gordon, W, Landman A, Zhang A, Bates D. Beyond validation: getting health apps into clinical practice. Digit Med. 2020;14. https://www.bfarm.de/EN/Medical-devices/Tasks/DiGA-and-DiPA/Digital-Health-Applications/_node.html
National Institute for Health and Care Excellence. Early Value Assessment (EVA) for medtech. Accessed 5 September 2023. https://www.nice.org.uk/about/what-we-do/eva-for-medtech
National Institute for Health and Care Excellence. Early value assessment interim statement. Process and methods. Published 15 December 2022. Accessed 20 September 2023. https://www.nice.org.uk/process/pmg39/chapter/a-new-approach-to-evidence-generation-for-medical-and-digital-health-technologies
National Institute for Health and Care Excellence. Evidence standards framework for digital health technologies. March 2019. Accessed 5 September 2023. https://www.nice.org.uk/Media/Default/About/what-we-do/our-programmes/evidence-standards-framework/digital-evidence-standards-framework.pdf
National Institute for Health and Care Excellence. Guidance, NICE advice and quality standards. Accessed 16 October 2023. https://www.nice.org.uk/guidance/published?sp=on
National Institute for Health and Care Excellence. Health technology evaluations: the manual Process and methods. Published 31 January 2022. Accessed 16 October 2023. https://www.nice.org.uk/process/pmg36/chapter/introduction-to-health-technology-evaluation Accessed 20th September 2023
National Institute for Health and Care Excellence. Seven technologies recommended to help people with non-specific low back pain. 13 October 2023. Accessed 16 October 2023. https://www.nice.org.uk/news/article/seven-technologies-recommended-to-help-people-with-non-specific-low-back-pain
Ofori A, Monnickendam G. POSC251: A comparison of the reimbursement of digital therapeutics in France, Germany and the UK. Value Health. 2022 January;S183.
