Response from the Multiple Sclerosis Society
http://www.mssociety.org.uk/nice/prov_appraisal.htm
11 September 2001
Introduction
1. At this next stage of the Committee’s reconsideration of the use of disease-modifying drugs in MS it is worth stating once again two stark realities facing people with MS:
Lack of reasoning in the PAD
3. The Appeal Decision of November
2000 was critical of the first FAD produced on beta interferons in a number
of areas, but a common thread was the lack of reasoning in that document.
Regrettably, there is a similar lack of reasoning in the current PAD in
relation to key aspects of the Appraisal Committee’s consideration of the
issues:
5. The successful appeal against
the original FAD has placed a clear onus on the Committee to show the reasoning
which has led to its views. If anything the need for clear explanations
of the Committee’s reasoning is even greater now than previously, because
the highly complex nature of the new economic modelling is in itself a
barrier to lay understanding of how the Committee has arrived at its decision.
It is therefore doubly disappointing to have to express the same points
on lack of reasoning that we made in our appeal against the original FAD:
Use of a ten-year time horizon
7. There is substantial class 1 evidence
of efficacy for these drugs over the medium-term. While there are
genuine difficulties in assessing the long-term benefits of treatments,
the approach to which the Committee has adopted is unduly rigid and harsh.
It discriminates against people who have long-term conditions: the Committee
would have them wait until decades of experience from other countries proves
the cost-effectiveness of treatments.
8. According to the PAD ‘In calculating
a cost per QALY using a 20-year time frame, the model had to assume that
benefits accrue over a period much longer than is supported by any available
evidence from clinical trials.’ The use of assumptions is inevitable
in considering the cost effectiveness of interventions in long-term conditions.
After discussion with the various interested parties the ScHARR consortium
considered the use of a twenty-year time horizon within the economic model
to be reasonable. In assessing whether the assumptions employed
are reasonable the Committee should consider two sets of issues:
9. The consortium’s use of a 20-year
time horizon was based on a conservative use of available evidence from
regulatory trials and from post-licensing studies such as the 4-year PRISMS
study and 6-8 year evidence on glatiramer acetate.
10. On top of this conservative approach,
in its consideration of the extrapolation of benefit we see once again
the Committee assuming that absence of evidence implies evidence of absence.
This is an intellectually unsound conclusion. There may be circumstances
where this is an appropriate assumption to make, but an assessment of long-term
use of drugs in a chronic condition is not one where such an assumption
should be regarded as automatic. On the contrary, such an approach
loads the dice against long-term conditions where it would be impracticable
and unethical to conduct trials of sufficient duration to provide the evidence
the Committee requires. Instead, the Committee should weigh all the
evidence to assess the balance of probabilities: is it more likely that
the benefits of the drugs continue to accrue over a longer period than
shown through evidence from clinical trials or more likely that they do
not?
11. Such a weighing of probabilities
must also take into account the very long-term nature of MS. The
assumption that benefit accrues only up to ten years has no connection
with the experience of people with MS, who will live with the disease for
thirty or forty years. They should not be denied access to the drugs
on the assumption that benefit accrues for only ten years, when successive
waves of evidence indicate benefit over ever-lengthening periods of time.
Assumption that benefits are not maintained
once treatment ceases
12. After discussion with the various
interested parties the ScHARR consortium considered the assumption that
benefit is maintained after treatment has ceased to be reasonable.
This assumption has been summarily overturned with no real explanation
save the assertion that the natural history dataset demonstrates otherwise.
13. Elsewhere in the PAD the Committee
has attached little or no weight to evidence which has not been generated
through Randomised Control Trials (‘non-RCT evidence’). The weight
attached to the natural history dataset is in stark contrast to this rigid
view on other non-RCT evidence. The natural history data set has
not been published or peer reviewed. It was collected at longer intervals
than in most RCTs; there was no blinding or controls to eliminate
investigator bias; there has not been any test of consistency between the
London Ontario assessors and those collecting data for other studies; and
the model involves the translation of DSS scores from the natural history
data set to EDSS scores. All of these aspects add to the level of
uncertainty about this data, but do not appear to have been treated by
the Committee in the same manner as it has considered other non-RCT data.
The Committee is apparently willing to depart from its earlier view of
non-RCT evidence in order to overturn the widely-accepted assumption used
by the consortium. Once again, this is done without adequate explanation.
Costs
14. The economic model’s narrow definition
of the costs of the MS underestimates its impact on daily life and
reduces the process to one interested only in how much the drugs save the
NHS rather than their real effect on people’s lives. This is particularly
so when considering relapses, the social and economic impact of which can
be far greater than the narrow costs used in the model.
15. The baseline ‘no treatment’ costs,
by reflecting current levels of service provision doubly penalise people
with MS. They suffer from poor service provision and suffer again
because other treatments appear expensive when compared with this.
Is the review of the clinical and cost-effectiveness,
and the wider implications for the NHS a reasonable interpretation of the
evidence?
16. The Committee’s view stands in
direct opposition to virtually all international opinion. While international
consensus is emerging on the additional benefits of early use of the drugs
England and Wales will move in the opposite direction – to no prescription
at all.
17. The points made above on data
omitted and the very conservative view on the value of extrapolation, taken
with the points made in this section, lead us to the conclusion that the
PAD does not represent a reasonable interpretation of the evidence. Essentially,
at each point where a choice has been made about use of evidence; about
measures of cost-effectiveness used and the weight attached to them; and
about what are reasonable assumptions, the Committee has opted for a choice
which minimises the likelihood of the drugs being viewed as cost-effective.
The economic data in particular has been manipulated in ways that challenge
expert consensus and experience from routine clinical practice. This
is done without any authoritative basis, but the outcome is regarded as
sufficiently robust to act as the foundation for a decision of immense
importance.
Use of Quality Adjusted Life Years
(QALYs)
18. Central to the Committee’s consideration
has been the use of the QALY. The QALY is designed for use in assessing
interventions in acute illness or preventative interventions, for example
cholesterol lowering drugs in people who have a myocardial infarction.
It is not designed to measure the efficacy of an intervention in a progressive
disorder where efficacy relates to a reduction in acute attacks of symptoms,
or to slowing progression. We consider the Committee’s decision to
have been made on the basis of a flawed measure of quality of life.
Use of the Expanded Disability
Status Scale (EDSS)
19. We have on a number of occasions
drawn attention to the limitations of the EDSS and its use in this appraisal.
Essentially our concerns are that the EDSS only partially captures the
effects of MS (as the PAD acknowledges, it focuses on mobility) but
has nevertheless been applied as if it captured the entire effects of MS
and benefits of the disease-modifying drugs. Paragraph 2.5 of the
PAD, which acknowledges the broad impact of MS, only serves to highlight
the narrowness of the Committee’s approach.
20. There is also a more specific
point on the way in which the EDSS has been employed in the economic model.
As we have said before, the assumption in the model that people move only
in one direction along the EDSS does not reflect real life. Indeed,
it is accepted in para 4.7 of the PAD which acknowledges the fact that
a small study had shown that 75% of patients ‘were unchanged or improved
[our italics] in terms of accumulation of disability after 8 years using
glatiramer acetate.’
Are the provisional conclusions of the Appraisal
Committee sound, and do they constitute a suitable basis for the preparation
of guidance to the NHS?
General points
21. For the reasons above the provisional
conclusions of the Appraisal Committee are unsound and do not constitute
a suitable and lawful basis for the preparation of guidance to the NHS
in England and Wales.
The position of those currently
on treatment
22. The guidance would leave those
who currently receive the drugs on therapy. The reference to the
continuing application of the stopping criteria in the ABN guidelines is
useful. There would, however, be significant downward pressure on
drug use and the role of the ABN guidelines in para 1.2 could be more clearly
expressed:
23. The position of those who have
been participating in clinical trials is not addressed. Article 30
of the Declaration of Helsinki states that ‘At the conclusion of the study,
every patient entered into the study should be assured of access to the
best proven prophylactic, diagnostic and therapeutic methods identified
by the study’. Those recruited to trials before the guidance comes
into force will have a legitimate expectation that they may continue on
therapy (or begin therapy if they have been on placebo) if the trial proves
successful. To remove this legitimate expectation discriminates against
those who have taken part in trials when they may otherwise have received
routine treatment.
24. In these circumstances, the likelihood
of future trials in England and Wales gaining ethical approval is limited.
This would have adverse knock-on effects for the development of clinical
practice and for the pharmaceutical industry in the UK.
Further research
25. The vague proposal for on-going
data collection does not effectively address the fundamental issue of how
the cost-effectiveness of interventions in long-term conditions may be
demonstrated. The proposal (in para 5.1) that health authorities
collect data on those who continue to receive treatment would be of little
value. The population would be skewed, would diminish in size, and
data could inevitably be captured only from part-way through their treatment.
No authoritative evidence would be produced by such a proposal.
26. In other appraisals the Institute
has made explicit recommendations for further research. We urge the
Committee to think more imaginatively about recommendations for further
robust research that would significantly narrow the uncertainty that has
made the appraisal of these drugs difficult. In addition, the fundamental
issues raised by this appraisal are not specific to MS and wider lessons
about the collection of information in the post-licensing phase of new
interventions (particularly those for long-term conditions), and about
the measures to adopt in future clinical trials, could be learned from
suitably wide-ranging research.
Implementation
27. The paragraph on implementation
sits oddly in the context of other guidance issued by the Institute, focussing
not on how the guidance should be applied, but on external factors which
may impact upon a re-appraisal of the issues. Its inclusion is a
profoundly disingenuous gesture as on the assumptions and measures adopted
by the Committee (which are also variables in this appraisal) it must be
obvious that to clear the £30,000 cost-per-QALY hurdle the drugs
would have to be provided at prices greatly below those charged elsewhere
in the world. As such it provides irrational underpinning for the
Committee’s decision as it holds out a spurious possibility of successful
re-appraisal.
28. To use unproven treatments when
proven and licensed ones are available will leave clinicians in an unenviable
ethical, moral and legal situation. The PAD contains nothing to guide clinicians
on the difficult choices they will face if their ability to prescribe glatiramer
acetate or beta interferons through the NHS is removed.
Other issues
29. It is clear from papers presented
to NICE’s Annual Public Meeting that £30,000 per QALY has effectively
become the benchmark of cost-effectiveness. There have been no directions
from the Department of Health or the National Assembly for Wales that they
consider this to be an appropriate test and there has been no public consultation
on the issue by the Institute. Instead the public is asked to acquiesce
to the idea that the custom and practice of the Appraisal Committee will,
of itself, deliver a test that is reasonable. We do not believe that
this is an appropriate or acceptable way to develop such a fundamental
aspect of public policy.
4. In addition, we have not had access
to other documents which might help us understand the Committee’s decisions:
we have not seen copies of any comment from the Institute’s staff to the
Committee which may have accompanied the report from the ScHARR consortium,
and the minutes of the 26 July Committee meeting have not yet been published.
This further hinders our ability to understand the Committee’s reasoning
and comment effectively on the contents of the PAD.
6. The lack of clear reasoning in the
PAD limits our ability to make effective input to the appraisal process.
As a consequence, while we set out below certain comments, we reserve our
rights to take such action as we believe appropriate.
i) the evidence upon which
the extrapolation of benefit is based
ii) the reasonableness of
the extrapolation in the context of that evidence and of the time scale
over which people are affected by MS.
It should then set out clearly its reasoning
to allow it to be properly understood and, if necessary, challenged.
‘It is likely that patients
currently receiving beta interferon or glatiramer acetate would suffer
loss of well being if their treatment was discontinued. Because of this,
patients may continue therapy until they and their consultants consider
it appropriate to stop. This decision should be taken in the context
of the criteria for withdrawal from treatment, established in the Guidelines
of the Association of British Neurologists (published January 2001).’
The position of those currently participating
in clinical trials