52 in 52 – #35: ECASIII Reanalysis
- May 18th, 2023
- Brannon Inman
- categories:
Welcome back to the “52 in 52” series. This collection of posts features recently published must-know articles. This week we cover “Thrombolysis with alteplase 3–4.5 hours after acute ischemic stroke: trial reanalysis adjusted for baseline imbalances”, a reanalysis of ECASIII and tPA for stroke.
Author: Brannon Inman (Chief Resident, Emergency Medicine Physician, San Antonio, TX) // Reviewed by: Alex Koyfman, MD (@EMHighAK); Brit Long, MD (@long_brit)
Thrombolysis with alteplase 3–4.5 hours after acute ischemic stroke: trial reanalysis adjusted for baseline imbalances
Background:
In 1995, the National Institute of Neurological Disorders and Stroke (NINDS) conducted two studies (NINDS I & II) that were published together as one article in the New England Journal of Medicine. The results reported no improvement in 24 hours, but in the embedded NINDS II study, a less severe modified Rankin Scale (mRS) was observed. However, these studies had significant discrepancies in baseline characteristics and stroke severity, with a much more severely disabled placebo group.
The ECASS III study is the only study thus far to report any benefit in the 3-4.5 hour time window. Despite this noteworthy distinction, the study had significant discrepancies in baseline characteristics and stroke severity, with a much more severely disabled placebo group.
The ATLANTIS B trial examined the extension of tissue plasminogen activator (TPA) administration from 3-5 hours using a dose of 0.9 mg/kg. This trial was initially planned to examine the effect of TPA administration in the 0-6 hour time frame but was stopped early due to safety concerns in the 5-6 hour time frame. The trial was turned into a 0-5 hour trial, and the patients enrolled up until that point were considered a separate trial and reported as “ATLANTIS A”. After NINDS was published, showing a reported effect at 0-3 hours, ATLANTIS B stopped enrolling patients within the 0-3 hour administration and focused on the 3-5 hour group. The trial was stopped early as “there was not likely to be a benefit”. It is worth noting that the trial showed a trend toward increased mortality in the TPA group, though it did not reach the prespecified criteria for termination. The methods used in this trial are admittedly messy.
The ATLANTIS A trial examined the use of TPA from 0-6 hours using a dose of 0.9 mg/kg. However, this trial was stopped early due to harm.
Guideline recommendations supporting TPA administration up to the 4.5 hour window are based primarily on the ECASS III study.
Clinical question:
How do differences in the treatment groups affect the results of the ECASIII study?
Study design and methods for correction:
- This study obtained access to the original data from the ECASS III trial through ClinicalStudyDataRequest.com.
- This study attempted to reproduce the results from table 2 through table 5 from the original publication attempting to ensure that they had the same database and were using the same variables as those used for the publication by the original study team.
- This study specified seven outcomes for their reanalysis, including:
- Symptom-free status
- Disability-free status
- Dependence-free status
- Mortality at 7 and 90 days
- Symptomatic intracranial hemorrhage (ICH)
- Change across mRS 0–6 at 90 days.
- This study assessed baseline imbalances between the two treatment groups and identified baseline variables that were statistically significant.
- To test for the robustness of the results according to the statistical analysis applied, the study conducted three (3) analytical approaches to dichotomous outcomes and multivariable modelling to the ordinal shift analysis
- Ordinal shift analysis is a statistical method used to measure changes in an ordinal outcome variable (a variable that has a finite number of categories that are ordered in a meaningful way). For example, the modified Rankin Scale (mRS) is an ordinal outcome variable.
- In this case ordinal shift analysis involves comparing the distribution of mRS scores between two groups (such as a treatment group and a control group) to determine if there is a shift in the overall distribution of scores.
PICO:
Population:
- Included patients enrolled in the ECASIII Study.
- The authors initial attempts failed to replicate the previously reported adjusted analyses. However, they were able to reproduce both the previously reported adjusted analyses with three conditions:
- Exclusion of patients who had incomplete or missing baseline NIHSS scores.
- Treatment of the baseline NIHSS score as a categorical variable with five categories (0–5, 6–10, 11–15, 16–20, >20).
- Treatment of the time from symptom onset to treatment as a categorical variable with seven categories (15 min windows).
Intervention:
- Removal of baseline discrepancies
Comparison:
- Original ECAS III publication
Outcome:
Reanalysis of the unadjusted data:
- On reperforming the unadjusted analysis on the original data, 209 patients in the alteplase group (rather than 210 patients) had an NIHSS score of 0 or 1.
- This discrepancy resulted in the outcomes at 90 days of NIHSS score of 0 or 1 and the global outcome becoming no longer statistically significant.
- On reperforming the unadjusted analysis for ICH, 11 cases of vascular serious adverse events in the placebo group rather than 10 cases originally reported.
- The original analysis in ECASS III was based on the NIHSS score, including two additional distal motor function components, resulting in a range of 0-46.
- However, the original publication erroneously stated a range of 0-42 based on 15 components.
- Reanalysis using the standard 0-42 range found that all primary outcomes were not significant.
Analyses adjusted for baseline imbalances:
- Calculated NIHSS scores were based on the 15 component scores in the original NIHSS score definition.
- In case a baseline NIHSS component score was absent, it was filled in with a value of 0. Similarly, if there was a missing value for a 90-day NIHSS component score, it was replaced with the highest possible value for that particular item.
- The only 2 variables that were statistically significant as baseline imbalances were the NIHSS score and history of prior stroke.
- After conducting a best-case sensitivity (Table 4) analysis, it was determined that the outcome of mRS 0 or 1 had a small effect across most analysis methods. However, the results for mRS 0 and mRS 0-2 were inconsistent and varied depending on the method of analysis used.
- This analysis showed significant increased RR of ICH
- A worst-case sensitivity analysis (Table S4a,b), using either the baseline NIHSS score ranging from 0 to 42 or 0 to 46, showed none of the efficacy end points were statistically significant in either unadjusted analyses or in any analysis that was adjusted for baseline imbalances.
Take Aways:
- This study highlights several of the significant methodological flaws in the original ECASIII study. The authors first acquired the data and then attempted to recreate the results. There were several issues including discrepancies in the reported and actual number of patients in placebo group and lack of transparency in converting continuous data into ordinal data or ordinal data into dichotomous. In summation, recreating the data was difficult. Despite the authors’ best attempts minor changes in outcomes were observed in some outcomes despite the use of the same data.
- In the adjusted analyses, 5 out of 6 analyses showed a significant increase in ICH.
- In the most forgiving of the reanalysis there was a small observed benefit in a mRS of 0-1, vs > 1. This lumps together having a small paresthesia in your arm but living independently with no disability vs living independently with small disability + coma, severe disability and death (2-6). I would argue this is not a meaningful comparison.
- In the most conservative reanalysis there was no benefit, and as mentioned observed increased ICH.
- Using this many tests against the null, the original authors probably should have used a Bonferroni correction. The Bonferroni correction is a conservative means utilized to adjust the significance threshold for multiple hypothesis testing. In application this correction makes the p value required to deem an observation significant much less than 0.05. For example, in table 4, with 14 comparisons you would actually need a p value of 0.00357 for an observation to be significant.
My take:
- This reanalysis highlights several of the issues in the original ECASIII and makes me seriously question if TPA has any role in the 3-4.5 hr window. At the patient level, I will follow the guidelines of my hospital, but at the policy level we really need to be having hard conversations about if we should be giving this medication for this indication. To me, these data do not really provide robust support for the use of TPA.
Reference:
- Alper BS, Foster G, Thabane L, Rae-Grant A, Malone-Moses M, Manheimer E. Thrombolysis with alteplase 3-4.5 hours after acute ischaemic stroke: trial reanalysis adjusted for baseline imbalances. BMJ Evid Based Med. 2020;25(5):168-171. doi:10.1136/bmjebm-2020-111386
- Hacke W, Kaste M, Bluhmki E, et al. Thrombolysis with alteplase 3 to 4.5 hours after acute ischemic stroke. N Engl J Med. 2008;359(13):1317-1329. doi:10.1056/NEJMoa0804656
- https://www.nejm.org/doi/full/10.1056/NEJM199512143332401
- Hoffman JR, Schriger DL. A graphic reanalysis of the NINDS Trial. Ann Emerg Med. 2009;54(3):. doi:10.1016/j.annemergmed.2009.03.019
