European Journal of Neurology
Volume 8 Issue 5 Page 503 - September
2001
Jeffrey I. Greenstein
Dear Sir,
The March 15, 2001 issue of the European
Journal of Neurology contained an article by Khan et al. reporting the
results of a small, open-label, unblinded study comparing the efficacy
of IFNb-1a (Avonex,
Biogen Inc., Cambridge, MA, USA), IFNb-1b
(Betaferon, Berlex Laboratories, Richmond, CA, USA) and glatiramer acetate
(Copaxone, Tevamarion Partners, Kansas City, MO, USA) for the treatment
of relapsing-remitting multiple sclerosis (MS). The authors reported that
both glatiramer acetate and IFNb-1b
reduced relapse rate compared with an untreated group, whereas IFNb-1a
had no effect. In addition, glatiramer acetate was reported to reduce the
mean number of relapses compared with the no treatment control group during
the second 6 months of therapy, whereas IFNb-1b
and IFNb-1a were
reported to have no effect. A secondary outcome variable, change in EDSS,
was reduced for IFNb-1b
and glatiramer acetate after 12 months, whereas IFNb-1a
was reported to have no effect.
The findings of this study are questionable
for a number of reasons. First, the results of this study contradict data
from the randomized, placebo-controlled, double-blind phase III clinical
trials of each of these agents (IFNB Multiple Sclerosis Study Group, 1993;
Johnson et al., 1995; Jacobs et al., 1996). In their respective phase III
trials, neither glatiramer acetate nor IFNb-1b
had a significant effect on sustained EDSS (IFNB Multiple Sclerosis Study
Group, 1993; Johnson et al., 1995), whereas IFNb-1a
produced a significant reduction (37%) in the risk of sustained disability
progression as measured by EDSS (Jacobs et al., 1996). In addition, none
of the available agents for the treatment of MS has ever been shown to
improve EDSS.
Secondly, there are several problems
with the design of the Khan study that lead to bias. One shortcoming is
that the patients were not randomized to treatment. Randomization ensures
that subjects have an equal chance of being assigned to any one of the
treatment groups. This process reduces the risk of selection bias because
any known or unknown factors that may affect the dependent variable are
distributed equally amongst treatment groups at the beginning of the trial.
In the Khan study, consecutive patients were studied and patients were
allowed to select one of three therapies or no therapy.
Another serious design flaw of the
Khan study is that it was not blinded, and both experimenters and subjects
were aware of which treatment subjects received. Subjects' and/or experimenters'
knowledge of the conditions of an experiment may directly or indirectly
influence the outcome of a study and invalidate the results. Blinding a
study controls for experimenter and subject biases and increases the probability
that these biases will not affect the study results. In MS clinical trials,
it is important to have both a treating neurologist, who is responsible
for assessment/treatment of adverse events and exacerbations, and an examining
neurologist, who is responsible for neurologic examinations. This division
of activities reduces the likelihood that the neurologist who is responsible
for assessing the effects of treatment on the primary outcome variable
will become unblinded to treatment. The Khan study is an open-label, non-randomized
study that contains selection and subject biases, making the validity of
any conclusions regarding efficacy questionable.
Thirdly, the Khan study evaluated
the efficacy of each immunomodulatory agent (IMA; IFNb-1a,
IFNb-1b and glatiramer
acetate) against a 'no treatment' group. As described in the report, this
no treatment group was comprised of patients who decided that they did
not want treatment with one of the IMAs. When evaluating the relative efficacy
among different therapies for MS, the effects of treatments should be compared
against one another or a placebo control group. It is not enough to simply
compare active treatment with no treatment because patients who received
the study drug may have improved simply because of the placebo effect.
In addition, because each of the IMAs was on the market for varying lengths
of time, patients may have had preconceived notions regarding the efficacy
of each IMA, and hence, the results of the study are biased. Patients who
selected 'no treatment' may also have had milder disease and more confidence
in not being on treatment.
Fourthly, the primary and secondary
outcome variables were not clinically meaningful. The effects of IFNb-1a,
IFNb-1b, and glatiramer
acetate on relapse rate were assessed as the primary outcome measure, with
relapses self-reported by subjects. Not only are patient self reports of
relapse unreliable, but relapse rate alone is a poor primary outcome measure
for clinical trials because it is independent of both the ongoing underlying
disease activity and the clinical course of MS (Weinshenker and Ebers,
1987; Confavreux et al., 2000). Confavreux et al. (2000) recently performed
a study to determine the relationship between acute relapses and the rate
of irreversible disability progression in a large cohort of MS patients
(Confavreux et al., 2000). Results showed that once patients reached a
score of >= 4.0 on EDSS, further progression in disability was not affected
by relapses. These data suggest that relapse rate should not be used as
the sole primary outcome measure in clinical trials because treatments
that have a short-term effect on relapses may have no effect on long-term
disability progression. Because MS is a progressive disease, it is important
to determine the effect of treatment on sustained disability progression.
The Khan study compared the effects
of IFNb-1a, IFNb-1b
and glatiramer acetate on mean change in EDSS as one of the secondary end-points.
However, this end-point is not valid for several reasons. First, EDSS scores
were not collected in a controlled manner because some EDSS scores were
reported by patients over the phone without a visit to the investigating
neurologist for verification. Secondly, mean change in EDSS score is not
an appropriate end-point because EDSS scores are based on an ordinal scale
and parametric statistical methods cannot be used to analyse changes in
EDSS score (Wingerchuk et al., 1997). In the Khan study, mean change in
EDSS was analysed using an analysis of variance, a parametric statistical
test, making the results of this analysis invalid. Thirdly, the changes
reported in EDSS scores were on a very narrow scale (a range of 0.2 improvement
to 0.2 decline), which have questionable clinical significance.
The results of the Khan study are
invalid because of the lack of appropriate experimental controls and outcome
measures. Claims regarding the relative efficacy amongst treatments for
MS can only be made based on the results of large randomized, double-blind
clinical trials with well-defined end-points, including sustained disability
progression as a primary end-point. Although phase IV trials may appear
to represent the 'real world', they are only valid for assessing safety
and tolerability, and cannot be used to make efficacy claims.
1 Confavreux C,
Vukusic S, Moreau T, Adeleine P (2000). Relapses and progression of disability
in multiple sclerosis. New Engl J Med 343: 1430-1438.
References
2 Jacobs LD, Cookfair
DL, Rudick RAet al. (1996). Intramuscular interferon beta-1a for disease
progression in relapsing multiple sclerosis. Ann Neurol 39:285-294.
3 Johnson KP,
Brooks BR, Cohen JAet al. (1995) Copolymer 1 reduces relapse rate and improves
disability in relapsing-remitting multiple sclerosis: results of a phase
III multicenter, double-blind, placebo-controlled trial. Neurology 45:1268-1276.
4 The IFNB Multiple
Sclerosis Study Group (1993). Interferon beta-1b is effective in relapsing-remitting
multiple sclerosis. I. Clinical results of a multicenter, randomized, double-blind,
placebo-controlled trial. Neurology 43:655-661.
5 Weinshenker
BG&Ebers GC (1987). The natural history of multiple sclerosis. Can
J Neurol Sci 14:255-261.
6 Wingerchuk DM,
Noseworthy JH, Weinshenker BG (1997). Clinical outcome measures and rating
scales in multiple sclerosis trials. Mayo Clin Proc 72:1070-1079.