Establishing a Framework for Evaluating the Effectiveness of Vaccines Targeting National Vaccination Programs

As the number of vaccines and the complexity of immunization programs continue to increase, there is a growing need for a systematic approach to studying vaccine effectiveness (VE).1 While many countries have developed VE evaluation programs, studies for VE in Korea have been sporadically and there are no routine programs for vaccines included in the National Immunization Program (NIP) in Korea.2 Most studies were performed through academic research grants from the Korea Disease Control and Prevention Agency (KDCA), with the majority done for 1–2 year projects. Studies performed during the last 15 years included mumps, hepatitis A, varicella zoster virus infection, influenza, pneumococcus and coronavirus disease 2019 (COVID-19). Vaccine-related stakeholders across disciplines, from clinicians to public health officers to the pharmaceutical industry, require more precise definitions to facilitate VE research. This ensures that studies are appropriately designed to address pertinent public health questions. To fulfill these objectives, we propose establishing a framework for evaluating VE in Korea by exploring models from other countries and providing general methodologies for integration into the VE evaluation framework.

In this study, we aimed to consolidate experts’ collective opinions regarding establishing a VE evaluation framework in Korea to develop evidence-based vaccination policies.

This study aimed to provide insights into the perspectives of experts on the establishment of a VE evaluation framework in Korea. We found high necessity and urgency ratings for establishing a government-led VE evaluation system, indicating strong consensus among experts on the importance of government involvement in vaccine evaluation. Public health needs to reorganize laws and regulations to underscore the need for a supportive regulatory framework to facilitate effective VE evaluation.4

Countries have implemented various methods to assess VE tailored to specific public health contexts, and some have introduced innovative approaches. VE studies are performed through governance of the national public health systems for NIP vaccines in many countries. In the US, VE studies are organized by VE Program Teams under the divisions of the National Center for Immunization and Respiratory Diseases (NCIRD).5 Each division includes branches which performs various surveillance platforms, epidemiology studies and laboratory work. VE studies are largely based on well-designed surveillance platforms which are conducted through contracts with public health departments, academic health institutions and medical centers. VE Program Teams organize, monitor studies, analyze and report the results in peer reviewed journals and are archived in the Centers for Disease Control and Prevention (CDC) website. CDC VE study members also participate in the Advisory Committee on Immunization Practices (ACIP) Work Group to plan, prepare and provide necessary and timely data for decision making policies.6 In England, VE studies are also planned and performed by members in the Immunization and Vaccine Preventable Diseases Division of the United Kingdom Health Security Agency (UKHSA).7 Members on the VE study teams include consultant clinical epidemiologists, epidemiologists, statisticians, database managers etc. VE studies are performed based on strong surveillance systems such as vaccination, hospital laboratories and clinical data which are linked with the National Health Service number.8 The UKHSA has permission as a public health agency to do data linkage without informed consent. During the COVID-19 pandemic in England, a national vaccine register, the National Immunization Management System (NIMS), was developed to collect COVID-19 vaccination data across various settings, enabling accurate monitoring of vaccine coverage and effectiveness, with high concordance between NIMS records and self-reported vaccination data.9 Japan developed its first COVID-19 vaccine assessment system, the Vaccine Effectiveness, Networking, and Universal Safety (VENUS) study, utilizing a multi-source database to actively monitor vaccination, COVID-19 cases, and polymerase chain reaction tests, achieving high data linkage rates and offering potential for population-based research on VE and safety.10 The preference for expanding and reorganizing responsible departments or forming a professional steering committee suggests a preference for collaborative approaches involving governmental and expert stakeholders. An effective VE evaluation framework is crucial for vaccination program managers to track progress and effectiveness against benchmarks; however, literature on monitoring frameworks and indicators used in national vaccination programs is limited and primarily focuses on routine childhood vaccination, highlighting the need for further documentation and analysis to inform future efforts.11

The majority of survey participants emphasized the importance of prioritizing influenza vaccine for effectiveness evaluation. Considering its seasonal nature, yearly variability, and the presence of antigenic shifts and multiple subvariants circulating simultaneously, assessing the challenges faced by the influenza vaccine is crucial. Influenza imposes significant public health burdens, particularly among infants, the elderly, and individuals with chronic diseases. Close monitoring of its annual vaccine effectiveness is essential for enhancing vaccination programs.

The survey highlighted the critical role of financial resources and human capital in establishing and sustaining a robust VE evaluation system. Including stakeholders from health authorities, academia, and industry underscores the importance of diverse expertise and collaboration in the evaluation process. The high ratings for establishing an integrated database platform and legal standards for data use indicate the importance of data infrastructure and privacy safeguards in VE evaluations. Experts have emphasized linking infectious disease reporting and vaccination data to enhance the accuracy and effectiveness of evaluations.

The perceived necessity and urgency of establishing and expanding surveillance systems for infectious diseases subject to vaccination highlight the importance of robust monitoring mechanisms for assessing VE. Quality control measures and compensation plans for participating institutions recognize the need for accountability and support within the surveillance framework. Equitable sharing of public health surveillance data, guided by principles outlined in an expert consultation from the UK, can effectively prevent or mitigate the impact of infectious diseases by: 1) building trust, 2) articulating value, 3) planning for data sharing, 4) achieving quality data, 5) understanding the legal context, 6) creating data-sharing agreements, and 7) monitoring and evaluation.12 The importance placed on managing and disclosing VE evaluation survey results reflects a commitment to transparency and accountability in the evaluation process. The need to expand the ways to disclose and utilize research data underscores a broader effort to maximize the utility of VE evaluation findings to inform policy and practice.

One limitation of this study is that it relied solely on expert opinions collected through online surveys, which may not fully capture the perspectives of all relevant stakeholders involved in VE evaluations in Korea. Additionally, the preliminary nature of the survey may have limited the depth and comprehensiveness of the proposed VE evaluation framework. Nevertheless, one strength of this study lies in its comprehensive approach to gathering expert opinions through a structured online survey, which allows for the systematic exploration of key considerations in establishing a VE evaluation framework for Korea’s NIP. In addition, by focusing on multiple domains derived from the initial survey results, this study provides a thorough analysis that can inform policymakers of the necessary steps for developing an evidence-based vaccination policy.

Overall, the findings suggest a strong consensus among the experts regarding the necessity and urgency of establishing a comprehensive VE evaluation framework in Korea. The identified priorities and preferences provide valuable guidance for policymakers and stakeholders to design and implement effective strategies for evaluating VE and enhancing public health outcomes.

In conclusion, the survey revealed critical insights into the urgent need for a systematic approach to VE evaluation in Korea. Key findings indicate a pressing demand for establishing a government-led VE evaluation system, reorganizing laws and regulations, securing adequate budgetary resources, and implementing an integrated database platform for epidemiological, clinical, and vaccination data. Moreover, the expansion of surveillance systems for major infectious diseases subject to vaccinations and efficient data management and results utilization emerged as pivotal areas requiring immediate attention. These findings underscore the imperative of collaborative efforts among policymakers, public health officials, researchers, and industry stakeholders to address the evolving landscape of infectious diseases and strengthen evidence-based vaccination policies in Korea.

The enhanced VE evaluation framework in this domain will empower policymakers to optimize decision-making regarding vaccination programs and ensure maximal health benefits from vaccines in Korea. Moreover, it is essential for clinicians to communicate the advantages of vaccines to the public effectively. However, to validate these findings, further research and collaboration are essential to address the identified needs and ensure the successful implementation of the proposed framework.