Challenges in assessing COVID-19 vaccine effectiveness in resource-limited settings: Experiences from South Africa

Abstract

Evaluating the real-world effectiveness of vaccines, including COVID-19 vaccines, and various biomedical interventions is crucial to address gaps in evidence from randomised controlled clinical trials and inform the national rollout of vaccinations. In the context of COVID-19, these gaps may include vaccine effectiveness against variants of concern and in high-risk subgroups such as people living with HIV. Designing vaccine effectiveness studies is more complex than designing randomised controlled clinical trials as it requires the availability of reliable, routinely collected data. Effectiveness studies in low- to middle-income countries (LMICs) are essential for tailoring vaccination strategies, addressing high-risk subgroups, ensuring equitable protection, and contributing valuable data to global health efforts. However, fewer COVID-19 vaccine effectiveness studies have been conducted in LMICs, including on the African continent, compared to high-income countries. Through our experience, it has become clear that national health data systems, resources and infrastructure, as well as adequate statistical capacity – which is crucial when conducting robust effectiveness studies – are lacking in LMICs. While each COVID-19 vaccine effectiveness study employed a specific study design and analytical approaches, none, to our knowledge, provided a rationale for their study design and statistical methods. Drawing from practical experiences, reflections and lessons learnt after designing a COVID-19 vaccine effectiveness study in a resource-limited setting, we present key considerations for data sources needed to run real-world effectiveness studies, for study designs, and for statistical modelling suitable for effectiveness studies. In the context of COVID-19, the study designs and statistical models are suitable for both prime and booster vaccines.

Significance:

• Substantially fewer COVID-19 vaccine effectiveness studies have been conducted in LMICs than in high-income countries.
• The lack of integrated national health data systems contributes to the lack of robust effectiveness studies in general and this was also observed during the COVID-19 pandemic.
• While each COVID-19 vaccine effectiveness study employed a specific study design and analytical approaches, none, to our knowledge, provided a rationale for their study design and statistical methods.
• Therefore, drawing from practical experiences, reflections and lessons learnt after designing a COVID-19 vaccine effectiveness study in a resource-limited setting, we present key considerations for study designs, data requirements and statistical modelling suitable for effectiveness studies.