Monitoring and Supervising Immunization Services From Health Facility to National Levels

Abstract

• Name: Monitoring the Quality of Immunization Services and the Quality of a Management Information System from Health Facility to National Levels
• Purpose: In November 1994 a working group was appointed by the Ministry of Health in the Kyrgyz Republic to improve the Management Information System for immunization services. The primary task for the working group was to implement the practice of using information in managing health services at all levels. As a starting point, the working group began with information related to immunization services.
• Type of Design: The group developed a consolidated reporting form, worksheets, and graphs for monitoring not only immunization services, but also the quality of information reported. The monitoring process developed in Kyrgyzstan also includes analytical supervision and self assessment using a one page check list. The tools developed are applicable at health facility, district, and national levels. Although immunization services were selected at the outset, the long term objective in Kyrgyzstan is to incorporate other health services and disease surveillance into the monitoring process and management information system.
• Where Used: Although this work took place in Central Asia, the participatory and ownership process followed in executing this approach can be used in any country.

For information on the English version, please contact Robert Steinglass at BASICS II (rsteingl@basics.org).

Basic Information

Name: Monitoring the Quality of Immunization Services and the Quality of a Management Information System From Health Facility to National Levels

Origin: Ministry of Health, the Republic of Kyrgyzstan; BASICS

Source: BASICS

Basic Description: This tool uses information collected by routine reporting for monitoring the following: immunization coverage, contraindication rates, drop out rates, vaccine supply and usage, the cold chain (equipment and storage temperatures), vaccine usage, and the reliability of information reported, including population figures (denominators), and immunizations given.

Country Applications: Central Asia - Kyrgyzstan;. CIS - Ukraine; Africa - the approach used to improve the management information system in Kyrgyzstan has been applied to improve disease surveillance in Uganda.

Languages Available: Russian, Ukrainian, English translation.

Technical Scope: Routine monitoring and management information, particularly in the context of the reform of health care systems in the former Soviet Union.

Purpose: Monitoring, supervision, and quality assurance of immunization services and management information systems.

Type of Methods: Quantitative and qualitative.

Design: Routine reporting.

Frequency of Administration: Monthly, quarterly if desired, annual, and ongoing.

Key Users of Information
Officers in charge of health facilities, district and national program managers, statisticians, and epidemiologists.

Objectives and Scope of Tool

1. Simplify and operationalize management information systems;

1. Lower the cost and increase the sustainability of information systems;
2. Promote use of information at the collection point;
3. Monitor both health worker performance and the reliability of information;
4. Implement self assessment to help overcome deficiencies resulting from infrequent supervision;
5. Promote the concept of an information system whose purpose is information use at all levels (as opposed to routine use of forms and computers, and the generation of reports at the central level).

Key Monitoring Needs and Evaluation Questions Tool Seeks to Address

1. Ensure the availability of reliable immunization coverage data at all levels;
2. Monitor missed opportunities for immunization, such as contraindications;
3. Monitor the reliability of the cold chain;
4. Provide up to date and reliable vaccine balances;
5. Provide up to date and reliable information on the status of the cold chain (equipment and storage temperatures);
6. Assess the quality of information being reported.

Key Indicators

1. DPT3 coverage (performance and access)
2. DPT3 contraindication rate (quality)
3. DPT3 vaccine usage (efficiency)
4. DPT1 to DPT3 drop out rate (quality and continuity)
5. Measles immunization coverage (performance)

Research Design

1. Requires active and continuous participation by a working group composed of all parties and levels involved in the country's management information system (e.g. health workers, district medical officer, national program, epidemiologists, pediatricians, HMIS manager).
2. External technical assistance must maintain a secondary role for decisions on indicators, tools, and methods.
3. Requires modifications to existing methods, forms, and tools; not creating new ones.
4. Requires adaption to local and country information needs, capacity, and resources.
5. Requires a minimum of three months field testing and multiple revisions to ensure ownership and clarity of training materials and manuals.

Training

• Training should be conducted by those with experience in the country's management information and health care systems and by those who believe in the revised methods and approach.
• Step wise training works best, beginning at the district level. District level supervisors then train workers in health facilities.
• Requires two days of training, including practical exercises and discussion.

Implementation

1. The process should involve management, medical and statistical staff, and health workers at facility level.
2. National implementation requires a planned approach to ensure that all health workers receive adequate training.
3. Implementation should begin at the peripheral level, and finish at the national level.
4. The approach requires government funding. It is best to avoid initial dependence on donor funding for printing forms, worksheets, and training manuals and for conducting local, district and national training.
5. After training and implementation, nation-wide assessments should be made to ensure that new procedures and practices have been implemented effectively.
6. External assistance is required to introduce the concept of local use of data and for describing the implementation process.

Analysis

1. Implementing this approach should actually reduce the amount of time that health workers spend on data collection and analysis (depending on the current HMIS in the country).
2. Analytical and statistical requirements are minimal, and are generally within the capacity of a health worker with a secondary level education.
3. Data can be compiled and analyzed using any spreadsheet or statistical software.

Reporting

1. This approach promotes reporting from the "bottom up," from health facility to district to national level.
2. Data are presented in a consolidated reporting form, using simple graphs and worksheets.
3. As described above, the content of information relates to immunization, but can be expanded to include other health services and variables.

Dissemination of Results

1. Information is reported in a standard format, monthly, quarterly when desired, and annually.
2. Information is for health workers at all levels.

Manuals and Guidelines

1. Training manuals for all levels (health facility, district and national) are available in Russian, Ukrainian, and English translation from BASICS.
2. Training manuals and materials relate to the former Soviet health care system which exists in Russia and all of the former Soviet Republics. They may also apply to certain Eastern European countries.

Summary of Tables and Graphs

• Monthly Immunization Reporting Form (all levels)
• Child Register, children below 15 years
• Population by age report
• Annual Plan for Immunizations by antigen and dose
• Monthly Register of Immunizations given
• Contraindications Register
• Record on Vaccine Flow
• Monthly Refrigerator Temperature Record
• DPT3 cumulative coverage graph, children below age one year
• Monitoring the Timeliness of DPT3 coverage (Ukraine)
• Worksheet on the use of DPT Vaccine
• Contraindications to DPT3, cumulative by month, children below age 1 year
• Analytical Supervision and Self Assessment Check List, health facility, district, and national
• Questions for the Supervision Check Lists

Contact Person
Robert Steinglass
BASICS Project
1600 Wilson Blvd.
Arlington, VA 22209
703-312-6800