More MS news articles for Sep 2001

National Institute for Clinical Excellence: Second Provisional Appraisal Determination on use of beta interferons and glatiramer acetate in the treatment of multiple sclerosis

Response from the Multiple Sclerosis Society

http://www.mssociety.org.uk/nice/prov_appraisal.htm

11 September 2001
 
Introduction

1. At this next stage of the Committee’s reconsideration of the use of disease-modifying drugs in MS it is worth stating once again two stark realities facing people with MS:

2. The Prime Minister has repeatedly said that he wishes healthcare provision in the UK to match the standards of our European Union partners.  The Committee’s provisional conclusion would stand this desire on its head for people with MS.
 
Has all relevant evidence been taken into account?

Lack of reasoning in the PAD

3. The Appeal Decision of November 2000 was critical of the first FAD produced on beta interferons in a number of areas, but a common thread was the lack of reasoning in that document.  Regrettably, there is a similar lack of reasoning in the current PAD in relation to key aspects of the Appraisal Committee’s consideration of the issues:

  • There is no substantive explanation within the PAD of why the Committee chose to overturn the view of the ScHARR Consortium that the model should use a twenty-year time horizon for assessing benefit, simply that ‘The Committee considered that there was insufficient basis for this assumption’ (of long term benefit)[paragraph 4.12].The Committee needs to explain this clearly.
  • Similarly, the FAD summarily dismisses the proposition that benefit is maintained after treatment has ceased with the inadequate statement ‘Again, the Committee did not consider that this was justified.’ [paragraph 4.12].  The Committee needs to explain this clearly.
4. In addition, we have not had access to other documents which might help us understand the Committee’s decisions: we have not seen copies of any comment from the Institute’s staff to the Committee which may have accompanied the report from the ScHARR consortium, and the minutes of the 26 July Committee meeting have not yet been published.  This further hinders our ability to understand the Committee’s reasoning and comment effectively on the contents of the PAD.

5. The successful appeal against the original FAD has placed a clear onus on the Committee to show the reasoning which has led to its views.  If anything the need for clear explanations of the Committee’s reasoning is even greater now than previously, because the highly complex nature of the new economic modelling is in itself a barrier to lay understanding of how the Committee has arrived at its decision.  It is therefore doubly disappointing to have to express the same points on lack of reasoning that we made in our appeal against the original FAD:

  • that the Institute has a duty to demonstrate how its conclusions flow from its analysis of the issues and have not been pre-determined.
  • that lack of explanation denies interested parties the ability to effectively engage with the process.
6. The lack of clear reasoning in the PAD limits our ability to make effective input to the appraisal process.  As a consequence, while we set out below certain comments, we reserve our rights to take such action as we believe appropriate.

Use of a ten-year time horizon

7. There is substantial class 1 evidence of efficacy for these drugs over the medium-term.  While there are genuine difficulties in assessing the long-term benefits of treatments, the approach to which the Committee has adopted is unduly rigid and harsh.  It discriminates against people who have long-term conditions: the Committee would have them wait until decades of experience from other countries proves the cost-effectiveness of treatments.

8. According to the PAD ‘In calculating a cost per QALY using a 20-year time frame, the model had to assume that benefits accrue over a period much longer than is supported by any available evidence from clinical trials.’  The use of assumptions is inevitable in considering the cost effectiveness of interventions in long-term conditions.   After discussion with the various interested parties the ScHARR consortium considered the use of a twenty-year time horizon within the economic model to be reasonable.   In assessing whether the assumptions employed are reasonable the Committee should consider two sets of issues:

i) the evidence upon which the extrapolation of benefit is based
ii) the reasonableness of the extrapolation in the context of that evidence and of the time scale over which people are affected by MS.
It should then set out clearly its reasoning to allow it to be properly understood and, if necessary, challenged.

9. The consortium’s use of a 20-year time horizon was based on a conservative use of available evidence from regulatory trials and from post-licensing studies such as the 4-year PRISMS study and 6-8 year evidence on glatiramer acetate.

10. On top of this conservative approach, in its consideration of the extrapolation of benefit we see once again the Committee assuming that absence of evidence implies evidence of absence.  This is an intellectually unsound conclusion.  There may be circumstances where this is an appropriate assumption to make, but an assessment of long-term use of drugs in a chronic condition is not one where such an assumption should be regarded as automatic.  On the contrary, such an approach loads the dice against long-term conditions where it would be impracticable and unethical to conduct trials of sufficient duration to provide the evidence the Committee requires.  Instead, the Committee should weigh all the evidence to assess the balance of probabilities: is it more likely that the benefits of the drugs continue to accrue over a longer period than shown through evidence from clinical trials or more likely that they do not?

11. Such a weighing of probabilities must also take into account the very long-term nature of MS.  The assumption that benefit accrues only up to ten years has no connection with the experience of people with MS, who will live with the disease for thirty or forty years.  They should not be denied access to the drugs on the assumption that benefit accrues for only ten years, when successive waves of evidence indicate benefit over ever-lengthening periods of time.

Assumption that benefits are not maintained once treatment ceases

12. After discussion with the various interested parties the ScHARR consortium considered the assumption that benefit is maintained after treatment has ceased to be reasonable.  This assumption has been summarily overturned with no real explanation save the assertion that the natural history dataset demonstrates otherwise.

13. Elsewhere in the PAD the Committee has attached little or no weight to evidence which has not been generated through Randomised Control Trials (‘non-RCT evidence’).  The weight attached to the natural history dataset is in stark contrast to this rigid view on other non-RCT evidence.  The natural history data set has not been published or peer reviewed.  It was collected at longer intervals than in most RCTs;  there was no blinding or controls to eliminate investigator bias; there has not been any test of consistency between the London Ontario assessors and those collecting data for other studies; and the model involves the translation of DSS scores from the natural history data set to EDSS scores.  All of these aspects add to the level of uncertainty about this data, but do not appear to have been treated by the Committee in the same manner as it has considered other non-RCT data.   The Committee is apparently willing to depart from its earlier view of non-RCT evidence in order to overturn the widely-accepted assumption used by the consortium.  Once again, this is done without adequate explanation.

Costs

14. The economic model’s narrow definition of the costs of the MS  underestimates its impact on daily life and reduces the process to one interested only in how much the drugs save the NHS rather than their real effect on people’s lives.  This is particularly so when considering relapses, the social and economic impact of which can be far greater than the narrow costs used in the model.

15. The baseline ‘no treatment’ costs, by reflecting current levels of service provision doubly penalise people with MS.  They suffer from poor service provision and suffer again because other treatments appear expensive when compared with this.

Is the review of the clinical and cost-effectiveness, and the wider implications for the NHS a reasonable interpretation of the evidence?

16. The Committee’s view stands in direct opposition to virtually all international opinion.  While international consensus is emerging on the additional benefits of early use of the drugs England and Wales will move in the opposite direction – to no prescription at all.

17. The points made above on data omitted and the very conservative view on the value of extrapolation, taken with the points made in this section, lead us to the conclusion that the PAD does not represent a reasonable interpretation of the evidence. Essentially, at each point where a choice has been made about use of evidence; about measures of cost-effectiveness used and the weight attached to them; and about what are reasonable assumptions, the Committee has opted for a choice which minimises the likelihood of the drugs being viewed as cost-effective. The economic data in particular has been manipulated in ways that challenge expert consensus and experience from routine clinical practice.  This is done without any authoritative basis, but the outcome is regarded as sufficiently robust to act as the foundation for a decision of immense importance.

Use of Quality Adjusted Life Years (QALYs)

18. Central to the Committee’s consideration has been the use of the QALY.  The QALY is designed for use in assessing interventions in acute illness or preventative interventions, for example cholesterol lowering drugs in people who have a myocardial infarction.  It is not designed to measure the efficacy of an intervention in a progressive disorder where efficacy relates to a reduction in acute attacks of symptoms, or to slowing progression.  We consider the Committee’s decision to have been made on the basis of a flawed measure of quality of life.

Use of the Expanded Disability Status Scale (EDSS)

19. We have on a number of occasions drawn attention to the limitations of the EDSS and its use in this appraisal.  Essentially our concerns are that the EDSS only partially captures the effects of MS (as the PAD acknowledges, it focuses  on mobility) but has nevertheless been applied as if it captured the entire effects of MS and benefits of the disease-modifying drugs.  Paragraph 2.5 of the PAD, which acknowledges the broad impact of MS, only serves to highlight the narrowness of the Committee’s  approach.

20. There is also a more specific point on the way in which the EDSS has been employed in the economic model.  As we have said before, the assumption in the model that people move only in one direction along the EDSS does not reflect real life.  Indeed, it is accepted in para 4.7 of the PAD which acknowledges the fact that a small study had shown that 75% of patients ‘were unchanged or improved [our italics] in terms of accumulation of disability after 8 years using glatiramer acetate.’

Are the provisional conclusions of the Appraisal Committee sound, and do they constitute a suitable basis for the preparation of guidance to the NHS?

General points

21. For the reasons above the provisional conclusions of the Appraisal Committee are unsound and do not constitute a suitable and lawful basis for the preparation of guidance to the NHS in England and Wales.

The position of those currently on treatment

22. The guidance would leave those who currently receive the drugs on therapy.  The reference to the continuing application of the stopping criteria in the ABN guidelines is useful.  There would, however, be significant downward pressure on drug use and the role of the ABN guidelines in para 1.2 could be more clearly expressed:

‘It is likely that patients currently receiving beta interferon or glatiramer acetate would suffer loss of well being if their treatment was discontinued. Because of this, patients may continue therapy until they and their consultants consider it appropriate to stop.  This decision should be taken in the context of the criteria for withdrawal from treatment, established in the Guidelines of the Association of British Neurologists (published January 2001).’
The position of those currently participating in clinical trials

23. The position of those who have been participating in clinical trials is not addressed.  Article 30 of the Declaration of Helsinki states that ‘At the conclusion of the study, every patient entered into the study should be assured of access to the best proven prophylactic, diagnostic and therapeutic methods identified by the study’.  Those recruited to trials before the guidance comes into force will have a legitimate expectation that they may continue on therapy (or begin therapy if they have been on placebo) if the trial proves successful.  To remove this legitimate expectation discriminates against those who have taken part in trials when they may otherwise have received routine treatment.

24. In these circumstances, the likelihood of future trials in England and Wales gaining ethical approval is limited.  This would have adverse knock-on effects for the development of clinical practice and for the pharmaceutical industry in the UK.

Further research

25. The vague proposal for on-going data collection does not effectively address the fundamental issue of how the cost-effectiveness of interventions in long-term conditions may be demonstrated.  The proposal (in para 5.1) that health authorities collect data on those who continue to receive treatment would be of little value.  The population would be skewed, would diminish in size, and data could inevitably be captured only from part-way through their treatment.  No authoritative evidence would be produced by such a proposal.

26. In other appraisals the Institute has made explicit recommendations for further research.  We urge the Committee to think more imaginatively about recommendations for further robust research that would significantly narrow the uncertainty that has made the appraisal of these drugs difficult.  In addition, the fundamental issues raised by this appraisal are not specific to MS and wider lessons about the collection of information in the post-licensing phase of new interventions (particularly those for long-term conditions), and about the measures to adopt in future clinical trials, could be learned from suitably wide-ranging research.

Implementation

27. The paragraph on implementation sits oddly in the context of other guidance issued by the Institute, focussing not on how the guidance should be applied, but on external factors which may impact upon a re-appraisal of the issues.  Its inclusion is a profoundly disingenuous gesture as on the assumptions and measures adopted by the Committee (which are also variables in this appraisal) it must be obvious that to clear the £30,000 cost-per-QALY hurdle the drugs would have to be provided at prices greatly below those charged elsewhere in the world.  As such it provides irrational underpinning for the Committee’s decision as it holds out a spurious possibility of successful re-appraisal.

28. To use unproven treatments when proven and licensed ones are available will leave clinicians in an unenviable ethical, moral and legal situation. The PAD contains nothing to guide clinicians on the difficult choices they will face if their ability to prescribe glatiramer acetate or beta interferons through the NHS is removed.

Other issues

29. It is clear from papers presented to NICE’s Annual Public Meeting that £30,000 per QALY has effectively become the benchmark of cost-effectiveness.  There have been no directions from the Department of Health or the National Assembly for Wales that they consider this to be an appropriate test and there has been no public consultation on the issue by the Institute.  Instead the public is asked to acquiesce to the idea that the custom and practice of the Appraisal Committee will, of itself, deliver a test that is reasonable.  We do not believe that this is an appropriate or acceptable way to develop such a fundamental aspect of public policy.