On March 11th 2020 the World Health Organisation declared the novel coronavirus (SARS-CoV-2) outbreak as a pandemic (Covid-19). To control the spread of the devastating disease many countries have employed interventions such as case isolation, quarantine, closures of schools, universities, restaurants and gyms, restrictions on travel and wide scale social distancing. Despite these measures, health systems have been pushed beyond capacity, with many clinical researchers volunteering to be redeployed to cope with increased demand. The development of Covid-19 has presented a variety of new challenges for all clinical trials in progress during the pandemic period.
Subject to participant and investigator safety, regulatory authorities have recommended data collection continue for as long as feasible, and where possible remotely for trials already in progress (see FDA , EMA and MHRA). But, the pandemic will inevitably lead to higher rates of missing outcome data in trials and for non-standard reasons. For example, some participants may not be able to provide outcome data as they have been infected and hospitalised with Covid-19. Other more vulnerable participants may choose not to attend follow-up visits that cannot be conducted remotely in order to stay sheltered at home. Alternatively, some participants who are in good health may not attend follow-up visits as they do not feel the need to see a clinician and want to avoid any new Covid-19 complications.
In all trials missing data creates a problem. This is because, if we only consider the observed data we may be left with an incomplete picture of how effective the treatment under study is. Imagine a half completed jigsaw puzzle, you might have a good idea what the overall picture is from the pieces already put together, but if the unobserved pieces are quite distinct and important, this could radically change the overall picture. In trials, if the health outcomes of patients we don't have are different from those we observe — for example the unobserved patients are sicker and therefore their outcomes are worse — we might not get the correct answer for the treatment being tested by analysing only the observed data. This might have terrible clinical consequences. The presence of Covid-19 in society adds additional greater challenges due to the scale by which trials are being affected.
Unfortunately we can never recover missing pieces of information. So what do we do? Our only option is to make an untestable assumption about the missing data, and perform the trial analysis under this assumption. As the assumption can never be proved, different credible assumptions about the missing data should also be made and the results of each analysis compared in sensitivity analysis. We will hope trial results remain the same under different assumptions to rule out any impact of the missing data on trial conclusions. However, if the results differ under different missing data assumptions it is important to report this as part of the trial findings.
But where do we start when considering what assumptions are most appropriate to make for missing data in trials overlapping a pandemic? Cro et al. propose the following four-step strategy to help facilitate clear thinking about the appropriate assumptions and analysis for relevant questions of interest:
Step (1): Clarify exactly what treatment effect (or estimand) you want to estimate from the trial. That is, do you want to know the treatment effect in the ‘world including a pandemic’ or in a hypothetical ‘pandemic-free world’.
Step (2): Establish what data is missing and why this may be. This will be informed by the estimand of interest (Step 1) and will help decide on how the missing data should be handled in the analysis. For the ‘world including a pandemic’ treatment estimand, all participant data not collected pre-/during/post-pandemic is required for analysis and creates a missing data problem. For the ‘pandemic-free world’ estimand only data unaffected by Covid-19 is required for analysis (data affected by Covid-19 may be set missing for analysis); there may also be missing data for participants not impacted by Covid-19 but lost to follow-up pre-/during/post-pandemic.
Step (3): Think carefully about what the missing participant data would have looked like and perform primary analysis under the most appropriate missing data assumptions. Step (4): Perform sensitivity analysis under alternative plausible missing data assumptions.
Unfortunately there are no missing data assumptions that can be universally recommended for primary and/or sensitivity analysis (Steps 3 and 4), these will be trial and estimand specific. But alongside a detailed explanation of the four-step strategy Cro et al. provides structured guidance to help assist decisions on plausible assumptions (e.g. when a missing-at-random assumption or a missing-not-at-random assumption may be most appropriate). In some settings different missing data assumptions may be most plausible for different types of participants in the same trial, for example for those infected with Covid-19 versus those simply lost to follow-up during Covid-19 for the ‘world including a pandemic’ treatment estimand. Controlled Multiple Imputation is highlighted as an accessible tool for implementing this. In summary, it is vital that careful consideration is given to the most plausible and relevant assumptions to handle missing data in any clinical trial. This will be especially critical for trials being conducted during Covid-19 due to increased amounts of missing data and direct or indirect effects of Covid-19 on outcomes. Clinical trials are expensive and involve a great deal of participants, clinicians and investigators time. We want to be confident that the results obtained are correct and lead to the right treatment decisions being made in practice. We hope this four-step strategy will help to facilitate clear thinking about the appropriate analysis for relevant questions of interest and support statisticians and investigators to maintain the scientific integrity of their trials in the Covid-19 environment.