Name: Verbal Case Review
Origin: Derived from IMCI Health Facility Assessment Exit Interview instrument, with substantial modifications and local variations
Source: Robert S. Northrup and Youssef Tawfik, BASICS Project; Sarbani Chakraborty, Johns Hopkins School of Hygiene and Public Health (Baltimore); Retna Siwi Padmawati, Clinical Epidemiology and Biostatistics Unit, Gadjah Mada University Faculty of Medicine, Yogyakarta, Indonesia; and Stephen Luby, Department of Community Health, Aga Khan University School of Medicine, Karachi, Pakistan.
Basic Description: The Verbal Case Review (VCR) is a household survey instrument and methodology used for assessing the quality of clinical care being provided to sick under-five children (under five years) by the full range of practitioners and vendors who are consulted in a target population. Households with children under five years are screened to identify those with children who have been sick with either diarrhea, fever, or cough/respiratory symptoms during the previous two weeks. The mothers of these children are then interviewed using the VCR instrument which asks them to recall from whom they purchased medicines, from whom they obtained clinical care, and the actions or behaviors of those drug sellers and practitioners consulted for the child's illness. The WHO protocol for the Integrated Management of Childhood Illness (IMCI) is the clinical standard from which the questions regarding clinical quality in the interview instrument are derived. Practitioners' behaviors are compared to these standards to determine the gap between current practices and the quality standard.
In settings where initial assessment using the VCR has led to an intervention to improve the quality of care, the VCR has been used in ongoing longitudinal monitoring of care quality. The results of such monitoring have been fed back to the practitioners in monitoring cycles of two-to-three months to stimulate further improvements. Carried out by community representatives, these monitoring activities have also generated improved interaction between the practitioners and the community. They have also indicated to practitioners that the community is concerned with the quality of care being provided and is willing to accept certain changes in care (not giving injections, not giving anti-diarrheal medicines). In some monitoring applications, the VCR has been simplified to facilitate implementation by community members.
Country Applications: Indonesia, India, Pakistan, Bangladesh
Languages Available: English, Indonesian
Technical Scope: Integrated child health services, including nutrition.
Purpose: To assess or monitor the quality of care being provided to sick children by practitioners, either private or public, from the formal, informal, or traditional sectors.
Type of Methods: Quantitative, with the possibility of adding some qualitative components.
Design: May be either cross-sectional or longitudinal, formative or summative. In any format it is descriptive. It has been used to provide an isolated assessment of health care seeking behaviors by mothers; case management of sick children by practitioners; a baseline and/or post intervention evaluation; and ongoing monitoring data to be used in feedback to practitioners as part of a quality improvement intervention strategy.
Frequency of Administration: A baseline cross-sectional assessment may be followed with a subsequent cross-sectional evaluation assessment after sufficient time has been allowed for intervention activities to take place. In India, Indonesia, and Pakistan the interval between surveys was six to nine months.
When the VCR is used by a program for ongoing monitoring and feedback to practitioners, it is carried out continuously in target communities. The VCRs completed during the monitoring cycle (two or three month intervals) are summarized, and individual as well as group reports are prepared and fed back to practitioners.
Key Users of Information
Information on the quality of care provided to sick children, particularly with regard to care being provided by private practitioners, is of immediate interest and use to program managers and health providers in government, NGOs and donor agencies. Agencies which have mounted intervention programs based on VCR results have included local NGOs (India, Pakistan), and local government units (district and subdistrict health offices in Indonesia). Central government health offices and donor agencies have found the information about the private sector particularly useful in planning child health interventions (Planning office in Indonesian MOH, World Bank, USAID, UNICEF). The data has stimulated higher level decision makers to devote additional resources to private practitioners, rather than concentrating solely on the government health system.
Objectives and Scope of Tool
To assess a) health seeking behavior and the comparative usage of the private and public sectors; and b) the quality of care provided to sick children under five years by private and public sector practitioners.
The tool at present is limited to the key child health problems represented in the WHO sponsored IMCI package. In two applications it has also included assessment of the adequacy and effectiveness of care being provided to sick children in the home, prior to seeking external care. The principle of the tool a delayed exit interview aimed at making data gathering on specific types of illness and specific age groups of the population more convenient - may be readily adapted to other aspects of quality of care (e.g. maternal care, family planning, etc). The VCR has been used in one Indian setting to assess family planning and pregnancy related behaviors.
Key Monitoring Needs and Evaluation Questions Tool Seeks to Address
The VCR provides information for the monitoring and evaluation of the quality of care being provided to sick children. It also provides information on the rate of health care seeking outside the home, and comparative usage of different types of providers.
Key indicators
Access:
Quality:
Examination Practices for All Children
Examination of Children with Fever
Examination of Children with Cough or Respiratory Problem
Examination of Children with Diarrhea
Treatment and Counseling for Children with Fever
Treatment and Counseling for Children with Cough/ARI
Treatment and Counseling for Children with Diarrhea
Research Design
Standard protocol:
In assessment or evaluation mode, the standard VCR protocol is a cross-sectional survey of
households in the designated sample area.
Households with children under five years are screened to identify those with children who have been sick with either diarrhea, fever, or cough/respiratory symptoms during the previous two weeks. The mothers of these children are interviewed and are asked to recall the actions or behaviors of drug sellers or practitioners consulted for the child's illness. In some applications, this target has been further screened to select only those children whose parents bought medicines outside the house, or consulted a practitioner. To allow determination of rates of illness in the population, and rates of care seeking outside the home, records must be kept of the responses in each of the preliminary screening steps, either in registers or special screening forms which can be analyzed separately from the verbal case review questions.
Units of Observation and Analysis: Units of observation are individual children under five years, and individual practitioners. Analysis typically differentiates responses from drug sellers (who usually do not see the child), and responses from practitioners (who examine the child as part of the case management process). In most applications analysis provides data for practitioners in a targeted community/sample area, and differentiates among practitioners of different types doctors, midwives, nurses, unlicenced practitioners (quacks), drug sellers. Where appropriate, the analysis differentiates private and public sector providers, often by institution health center, private clinic, shop, etc.
In monitoring mode as part of an intervention, the units of analysis are individual practitioners or institutions (e.g. health centers, hospitals), and sample design must use the same incidence factors as those used in the cross-sectional approach. It should also use the same rate of use of individual practitioners or institutions in that population. Statistical significance has not been a priority for monitoring data, but convincing feedback to a particular practitioner must include at least three patients overall. Meaningful discourse on a particular illness (e.g. management of ARI) must include at least three patients with that type of illness, seen by the specific practitioner. In addition, consideration of how many interviews a volunteer community worker or health center staff member can be expected to carry out per month on an ongoing basis must be considered in determining the intervals (e.g. one month, three months) at which the data is collected.
Sample Size:Sample size calculations must take into account the prevalence of households with children under five years in the target area, and the expected incidence of each of the three targeted illnesses. In addition, researchers should consider: the degree to which statistical significance is needed in differentiating among different types of providers, or between private and public sector providers, or between formal and informal sector types of providers, as well as other available information (e.g. from preliminary key informant interviews or DHS surveys) regarding expected patterns of care seeking from particular types of practitioners. Seasonal factors influence the incidence of all three illnesses in many developing countries. In addition, if the VCR survey will be used as a baseline for an intervention that will be evaluated for changes in specific behaviors, calculations should consider the initial rate of performance of the particular targeted behavior (e.g. administration of ORS to only 10% of children with diarrhea seen by private sector comprehensive practitioners) to that expected to be attained after the intervention (e.g. an increase of 400% to 40% of cases seen by the same class of practitioners).
Target Population: The target population may vary depending on the purpose of the research. When it is used as a cross-sectional assessment tool with a randomly selected sub-set of households, the VCR could be applied to a large population (district or national) using a cluster sampling approach. In the majority of applications, however, the VCR has been used as part of an intervention effort, and the targeted population has been the catchment population of the institution implementing the intervention.
Lessons from experience:
Training
Standard protocol:
Interviewers have been trained for two to three days, with detailed explanation of the reasons for
the individual questions, role play in the training site, and observed interviews in the community.
The extent of training has varied. In Indonesia professional interviewers were borrowed from an
ongoing surveillance project and required less training. In India, village community health
workers (paid) or community women were selected on the basis of educational levels and
willingness to work outside the household. Thus, training was more extensive. To ensure
quality, it is necessary to conduct field level reviews of completed questionnaires, and repeat
interviews with 10% or 15% of respondents. Feedback to the interviewers should follow.
Lessons from experience:
It has proven particularly difficult to differentiate clearly between: a) use of drugs available at
home from a previous purchase; b) purchase of drugs from a drug seller who does not see the
sick child; and c) dispensing of drugs by a practitioner who has examined the child. This aspect
must receive adequate attention in training.
Implementation
Standard protocol:
The number of interviewers ranged from two over a two month period, to 25-30 for monitoring
VCRs during an intervention. Depending on the number of additions to the basic questions in
the survey instrument, the VCR takes from 15 to 45 minutes to administer.
Transportation requirements will vary with sample size.
Basic skills in carrying out surveys, managing forms and data entry into computers, and analyzing survey data are important to enable the process to run smoothly. Community organizations and district or sub-district health offices typically need technical support from a local institution.
Lessons from experience:
External technical assistance has been useful in the development phase of this instrument,
helping to establish a set of questions which are answerable based on mothers' recall, manage the
survey process and data management issues, and keep the analysis focused on the critical case
management issues amenable to intervention. Involvement of people with public health
experience and some knowledge of IMCI is very helpful.
Analysis
Standard protocol:
Computer hardware and software requirements: 286 computer running EPI-INFO in DOS is
sufficient. More sophisticated analyses are facilitated by higher power computers and use of
SPSS or other statistical software package.
Analytical requirements: Simple tabulations of data, and two-way comparisons using Chi square, are the basic outputs, showing rates of performance and incidence rates comparing different subgroups in the sample.
Time and labor requirements: depend on the cleanliness of the data coming from the field. Once the data is clean, preparation of the basic tables can be accomplished in a few days.
Lessons from experience:
Reporting Issues
Collective reports provide tables comparing rates of current performance of desired behaviors with the standards, or with individual targets set as part of individual behavioral contracts made during intervention activities. A software program has also been developed to produce a monitoring report for an individual practitioner based on VCR assessment of patients he or she has seen during the previous monitoring cycle.
Standard protocol:
Lessons from experience:
The standard report from the VCR provides simple tables for the indicators and rates of
performance mentioned above, and cites in the text the critical observed gaps between
practitioner behaviors and the IMCI based standards of practitioner case management.
Dissemination of Results
Lessons from experience:
The data from the VCR has been presented to practitioners in an intervention target area to
stimulate participation in the intervention. The data was presented showing tables which listed
desired case management actions and rates (in percent) of actual performance by the group of
practitioners. This led to discussion about whether the reported observations were consistent
with what the practitioners were doing (they were), and ultimately to agreement to improve those
behaviors.
The data has also been presented to government officials at district and subdistrict level to convince them that intervention was useful. In the Indonesian experience, presenting data showing that the private practitioners performed at a higher level of quality than those working in government facilities led the district health officer to demand that the subsequent intervention include the public sector, as well as the private practitioners.
Manuals and Guidelines
References
A paper has been submitted for publication describing the Bihar, India experience with the VCR and subsequent intervention approach
Validity
The VCR was derived initially from the instrument used for exit interviews as part of the IMCI health facility assessment process. It has been validated in India by Sarbani Chakraborty, using a comparison of directly observed actions performed by a practitioner with those reported by the mother of the child being cared for. The study compared reliability of the mother's observations at one, seven, and 15 days after the encounter. The results indicate substantial variation in reliability depending on the particular practice being recalled, some being less accurately noted by the mothers than others. Overall reliability was similar at one and seven days, and showed some decay at the 15 day recall period.
Contact Person
Beth Plowman
BASICS Project
1600 Wilson Blvd.
Arlington, VA 22209
703-312-6800