Randomized trials on non-pharmaceutical interventions for COVID-19: a scoping review

Introduction

As of November 2021, 5 million deaths associated with SARS-CoV-2 infection and almost 250 million confirmed cases have been reported worldwide.¹ Numerous non-pharmaceutical interventions (NPIs) were taken worldwide to contain the pandemic.² NPIs are considered crucial to prevent infections, in particular for non-vaccinated populations and new virus variants.³ Such interventions include drastic social distancing and lockdown measures impacting billions of people around the globe.⁴ The need for randomised evidence on NPIs has been highlighted on numerous occasions.^{5 6} There is a major debate on the comparative effectiveness of public health measures such as quarantine, school and workplace closures, travel restrictions and different testing regimens to prevent COVID-19.^{7 8} The evidence for the benefits and harms of NPIs is mostly based on observational and modelling studies.⁹ Authors of high-quality evidence syntheses have concluded that the evidence for quarantine and travel restriction measures is limited facing a lack of randomised real-world evidence and that most of the studies use mathematical calculations based on diverging assumptions and model parameters.^{10 11}

Given the extremely complex interrelationship and parallel introduction of different interventions in a situation with strong time-dependent changes, including those of the virus itself, it is difficult and almost impossible to clearly attribute outcomes to individual interventions without parallel controls.⁹ Furthermore, with no valid causal models capturing complex interrelationship between interventions and regional settings, environments, population, cultural, social and economic factors nor high-quality, high-granular data reflecting these factors and allowing for statistical adjustments, there is a substantial risk of confounding bias, even for comparisons with parallel controls.^12–14 Only a randomised trial may provide high quality evidence on comparative effects of interventions without the need to understand complex mechanisms and measure complex data.^14–16

Randomised evaluations have been used in the past to measure causal effects of a wide range of diverse and complex behavioural, social, environmental interventions and policy strategies. For example, such randomised evaluations helped to determine how health insurance affects health outcomes,¹⁷ assessed impact of school class sizes on academic performance and health (including mortality),¹⁸ and explored which human papillomavirus screening strategy for cervical cancer is favourable,¹⁹ which hygiene interventions reduce hand bacterial counts,²⁰ how financial incentives improve physical activity²¹ or which infection control measures reduce spread of respiratory infections in children.²²

Conversely, no randomised trials assessing any NPIs were initiated early in the COVID-19 pandemic²³ and it is likely their number have remained sparse or even lacking.^{10 11 24} We aimed to provide a systematic overview of the major characteristics and results of randomised trials assessing NPIs to prevent COVID-19. Mapping the trial landscape helps to get an overview of the current trials worldwide, what evaluations on which NPIs are being conducted and what are the unmet needs.

Methods

A scoping review was performed to collect the current randomised trials on NPIs and to summarise their status and characteristics. We did not register this study or published a specific study protocol. We used the ‘Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews’) framework to provide a comprehensive study report, where applicable.²⁵

Results

We identified 41 eligible randomised trials (figure 1, table 1).

As of August 2021, 12 trials were completed (29.3%) including 9 with published results, 17 ongoing (41.5%), 9 not yet recruiting (22.0%) and 3 terminated early or withdrawn (7.3%). Most came from the USA (n=11, 26.8%), UK (n=4, 9.8%) and France (n=4, 9.8%). Of note, none of the completed and terminated trials had made their results available on the registries.

The types of NPIs were diverse. Of the 41 trials, 11 trials (26.8%) assessed protective equipment such as various masks, face shields or goggles; 9 trials (22.0%) assessed diverse COVID-19-related information and education programmes (eg, text messages, newsfeeds, personalised health education, Facebook advertisements, nurse-led home visits); 5 trials (12.2%) assessed the impact of attending a musical mass event/concert under specific safety measures; 5 trials (12.2%) assessed diverse testing and screening strategies in nursing homes, schools, universities and the emergency department; 5 trials (12.2%) assessed multicomponent behavioural interventions on hygiene management in the community. The other six trials assessed access to fitness centres, school opening, special housing of homeless individuals, direct income support for individuals with COVID-19-related financial disruptions, yoga sessions for healthcare professionals and COVID-19-related symptom tracking in outpatients with cancer.

The trials planned to recruit a median of 1700 participants (IQR 588–9500, range 30–35 256 399) with a median planned duration of 8 months (IQR 3–14, range 1–24). The largest trial assessed the impact of Facebook advertisements to prevent travelling and family reunions during Thanksgiving and Christmas with >35 million participants,²⁸ the shortest trial assessed the audience infection rate 8 days after an indoor music event.²⁹ A typical randomised trial on NPIs randomly allocated individuals (n=23, 56.1%), rather than clusters (n=18, 43.9%), to two comparison groups (n=38, 92.7%) and assessed a COVID-19-related primary outcome (n=33, 80.5%) that was laboratory confirmed (n=30, 73.2%).

Discussion

Worldwide, 41 randomised trials assessing NPIs have been initiated with published results available to inform policy decisions for only 9 of them. The number of trials assessing NPIs to prevent COVID-19 is a tiny fraction of more than 4000 randomised trials on COVID-19 registered globally as of August 2021.²⁷ Given the unprecedented public health impact worldwide and the wide debates about benefits and harms of NPIs, this number seems disproportionate in light of other interventions that have been the focus of much more randomised trials.⁸ For example, over 300 randomised trials included the highly debated hydroxychloroquine in their intervention arm³⁰ which has been found to be associated with increased mortality in COVID-19 patients.³¹ The reasons for such an imbalance remain speculative and require urgent investigation. However, trials assessing NPIs face some challenges. One hurdle is the need for large sample sizes to detect an effect of public health measures in populations with low event rates. For example, it was shown that about 1000 schools in Norway would have been needed to participate in a trial assessing school closures to demonstrate a meaningful risk reduction in a situation with low incidences.³² However, this number is put into perspective when one considers that there are more than 80 000 primary schools in France, Germany, UK alone and more than 1.5 billion students were deprived of schooling worldwide in the early days of the pandemic.^33–36 The fact that 117 million students are still affected by full school closure as of September 2021 underlines the continuous need to determine best strategies.³⁷ A randomised assessment would have helped to find optimal ways for schools to safely teach millions of students globally. One other potential hurdle is costs. However, worldwide, governments have made astronomical funds available for preventive measures. For example, it has been reported that Germany made available 700 million Euro for mobile air filters.³⁸ Such implementation could have been directly assessed through randomised roll-outs providing access to the preventive measures all the while providing robust evidence on their effectiveness.³⁹ Another barrier may be that as vaccines become more available and accessible, the need for randomised evaluations of NPIs may be perceived to be decreasing.³² The fatal course of the disease, with the dramatic increase in deaths in the early days of the pandemic has probably also shifted the research focus from prevention to treatments for COVID-19 to the detriment of evaluating NPIs.^{9 30} Overall, the specific role and relevance of these factors is not clear and requires an evidence-based assessment with sound meta-research.

The list of available treatments for COVID-19 remains limited,^40–42 with vaccination rates plateauing in many countries⁴³ and increasing numbers of confirmed cases during the summer and autumn 2021,¹ NPIs and their effectiveness will remain a central debate.⁹ Our analysis clearly indicates that randomised trials are generally feasible to assess the benefits and harms of NPIs during a pandemic situation, in a very short time using the most reliable method to provide reliable evidence for optimal health policy decision making. WHO is committed to improve the evidence base on effectiveness of public health and social measures,⁴⁴ and this overview may inform further developments in this important area of evidence generation.⁹

This study is limited as our sample relies on accurate trial registration and reporting. We cannot exclude that some NPI randomised trials might have been missed because they were not registered, and outcomes related to SARS-CoV-2-infections were incorrectly or not reported in the registries. Searching the grey literature (eg, via web search engine or institutional searches) or contacting key informants outside academia could have retrieved eligible trials that were not registered nor published in an academic format. In particular, NPI trials with inconclusive results or not supporting their use may be at risk of not being published potentially causing a publication bias.⁴⁵ Non-published results can be expected since some trials should have been completed according to their completion date given in the registry. Furthermore, we could have missed eligible trials because of our single-reviewer screening approach. However, it is unlikely that we have been unaware of pertinent major results of further NPI trials, given their substantial impact on current debates and scarcity of the evidence.⁴⁶

To provide continuous guidance for policy and practice, we will continuously track randomised trials assessing NPIs and provide all information on their key design characteristics and their status, freely accessible via covid-evidence.org.²⁶

Conclusions

Overall, during the first 18 months of the pandemic, the worldwide clinical research agenda failed to provide urgently needed evidence determining best strategies to prevent COVID-19 in schools, workplaces, nursing homes and other settings substantially affected. A long-term research agenda including behavioural, environmental, social and systems level interventions is urgently needed to guide policies and practices in the current and future public health emergencies.

Data availability statement

All data relevant to the study are included in the article or uploaded as online supplemental information. Not applicable.

Ethics statements