More MS news articles for Jan 2002

Appropriate design and outcome measures in multiple sclerosis clinical trials

European Journal of Neurology
Volume 8 Issue 5 Page 503 - September 2001
Jeffrey I. Greenstein

Dear Sir,

The March 15, 2001 issue of the European Journal of Neurology contained an article by Khan et al. reporting the results of a small, open-label, unblinded study comparing the efficacy of IFNb-1a (Avonex, Biogen Inc., Cambridge, MA, USA), IFNb-1b (Betaferon, Berlex Laboratories, Richmond, CA, USA) and glatiramer acetate (Copaxone, Tevamarion Partners, Kansas City, MO, USA) for the treatment of relapsing-remitting multiple sclerosis (MS). The authors reported that both glatiramer acetate and IFNb-1b reduced relapse rate compared with an untreated group, whereas IFNb-1a had no effect. In addition, glatiramer acetate was reported to reduce the mean number of relapses compared with the no treatment control group during the second 6 months of therapy, whereas IFNb-1b and IFNb-1a were reported to have no effect. A secondary outcome variable, change in EDSS, was reduced for IFNb-1b and glatiramer acetate after 12 months, whereas IFNb-1a was reported to have no effect.

The findings of this study are questionable for a number of reasons. First, the results of this study contradict data from the randomized, placebo-controlled, double-blind phase III clinical trials of each of these agents (IFNB Multiple Sclerosis Study Group, 1993; Johnson et al., 1995; Jacobs et al., 1996). In their respective phase III trials, neither glatiramer acetate nor IFNb-1b had a significant effect on sustained EDSS (IFNB Multiple Sclerosis Study Group, 1993; Johnson et al., 1995), whereas IFNb-1a produced a significant reduction (37%) in the risk of sustained disability progression as measured by EDSS (Jacobs et al., 1996). In addition, none of the available agents for the treatment of MS has ever been shown to improve EDSS.

Secondly, there are several problems with the design of the Khan study that lead to bias. One shortcoming is that the patients were not randomized to treatment. Randomization ensures that subjects have an equal chance of being assigned to any one of the treatment groups. This process reduces the risk of selection bias because any known or unknown factors that may affect the dependent variable are distributed equally amongst treatment groups at the beginning of the trial. In the Khan study, consecutive patients were studied and patients were allowed to select one of three therapies or no therapy.

Another serious design flaw of the Khan study is that it was not blinded, and both experimenters and subjects were aware of which treatment subjects received. Subjects' and/or experimenters' knowledge of the conditions of an experiment may directly or indirectly influence the outcome of a study and invalidate the results. Blinding a study controls for experimenter and subject biases and increases the probability that these biases will not affect the study results. In MS clinical trials, it is important to have both a treating neurologist, who is responsible for assessment/treatment of adverse events and exacerbations, and an examining neurologist, who is responsible for neurologic examinations. This division of activities reduces the likelihood that the neurologist who is responsible for assessing the effects of treatment on the primary outcome variable will become unblinded to treatment. The Khan study is an open-label, non-randomized study that contains selection and subject biases, making the validity of any conclusions regarding efficacy questionable.

Thirdly, the Khan study evaluated the efficacy of each immunomodulatory agent (IMA; IFNb-1a, IFNb-1b and glatiramer acetate) against a 'no treatment' group. As described in the report, this no treatment group was comprised of patients who decided that they did not want treatment with one of the IMAs. When evaluating the relative efficacy among different therapies for MS, the effects of treatments should be compared against one another or a placebo control group. It is not enough to simply compare active treatment with no treatment because patients who received the study drug may have improved simply because of the placebo effect. In addition, because each of the IMAs was on the market for varying lengths of time, patients may have had preconceived notions regarding the efficacy of each IMA, and hence, the results of the study are biased. Patients who selected 'no treatment' may also have had milder disease and more confidence in not being on treatment.

Fourthly, the primary and secondary outcome variables were not clinically meaningful. The effects of IFNb-1a, IFNb-1b, and glatiramer acetate on relapse rate were assessed as the primary outcome measure, with relapses self-reported by subjects. Not only are patient self reports of relapse unreliable, but relapse rate alone is a poor primary outcome measure for clinical trials because it is independent of both the ongoing underlying disease activity and the clinical course of MS (Weinshenker and Ebers, 1987; Confavreux et al., 2000). Confavreux et al. (2000) recently performed a study to determine the relationship between acute relapses and the rate of irreversible disability progression in a large cohort of MS patients (Confavreux et al., 2000). Results showed that once patients reached a score of >= 4.0 on EDSS, further progression in disability was not affected by relapses. These data suggest that relapse rate should not be used as the sole primary outcome measure in clinical trials because treatments that have a short-term effect on relapses may have no effect on long-term disability progression. Because MS is a progressive disease, it is important to determine the effect of treatment on sustained disability progression.

The Khan study compared the effects of IFNb-1a, IFNb-1b and glatiramer acetate on mean change in EDSS as one of the secondary end-points. However, this end-point is not valid for several reasons. First, EDSS scores were not collected in a controlled manner because some EDSS scores were reported by patients over the phone without a visit to the investigating neurologist for verification. Secondly, mean change in EDSS score is not an appropriate end-point because EDSS scores are based on an ordinal scale and parametric statistical methods cannot be used to analyse changes in EDSS score (Wingerchuk et al., 1997). In the Khan study, mean change in EDSS was analysed using an analysis of variance, a parametric statistical test, making the results of this analysis invalid. Thirdly, the changes reported in EDSS scores were on a very narrow scale (a range of 0.2 improvement to 0.2 decline), which have questionable clinical significance.

The results of the Khan study are invalid because of the lack of appropriate experimental controls and outcome measures. Claims regarding the relative efficacy amongst treatments for MS can only be made based on the results of large randomized, double-blind clinical trials with well-defined end-points, including sustained disability progression as a primary end-point. Although phase IV trials may appear to represent the 'real world', they are only valid for assessing safety and tolerability, and cannot be used to make efficacy claims.

1    Confavreux C, Vukusic S, Moreau T, Adeleine P (2000). Relapses and progression of disability in multiple sclerosis. New Engl J Med 343: 1430-1438.
2    Jacobs LD, Cookfair DL, Rudick RAet al. (1996). Intramuscular interferon beta-1a for disease progression in relapsing multiple sclerosis. Ann Neurol 39:285-294.
3    Johnson KP, Brooks BR, Cohen JAet al. (1995) Copolymer 1 reduces relapse rate and improves disability in relapsing-remitting multiple sclerosis: results of a phase III multicenter, double-blind, placebo-controlled trial. Neurology 45:1268-1276.
4    The IFNB Multiple Sclerosis Study Group (1993). Interferon beta-1b is effective in relapsing-remitting multiple sclerosis. I. Clinical results of a multicenter, randomized, double-blind, placebo-controlled trial. Neurology 43:655-661.
5    Weinshenker BG&Ebers GC (1987). The natural history of multiple sclerosis. Can J Neurol Sci 14:255-261.
6    Wingerchuk DM, Noseworthy JH, Weinshenker BG (1997). Clinical outcome measures and rating scales in multiple sclerosis trials. Mayo Clin Proc 72:1070-1079.