Assessment of ORS Commercialization Interventions Public/Private Partnerships for Child Survival

Abstract

Name: Assessment of ORS Commercialization Interventions
Purpose: The Assessment of ORS Commercialization Interventions (AOCI), a multi-study tool, was originally designed for the purposes of evaluating a multi-sectorial public-private partnership for the commercialization of oral rehydrat ion salts (ORS) in Bolivia. The tool measures knowledge, perception and behavioral data among consumers, pharmacists and doctors.
Type of Design: Descriptive qualitative studies and cross-sectional household studies.
Sample Size: Various
Where Used: Bolivia

Basic Information

Name: Assessment of ORS Commercialization Interventions

Origin: Sandhya Rao, Camille Saadé, Luis Alberto Quiroga, Marcela Revollo

Source: BASICS

Basic Description: A quantitative and qualitative multi-study methodology to assess and evaluate ORS commercialization interventions. The five parts of the tool include: 1) a knowledge, attitudes and practices (KAP) household survey with mothers of children under five years; 2) focus group discussions with consumers; 3) a simulated purchase study (SPS) in pharmacies; 4) in-depth interviews with doctors and pharmacists; and 5) a partnership process evaluation with members of an inter-agency task force consisting of international donors, Ministry of Health officials, pharmaceutical companies, advertising agencies and project managers.

Country Applications: Bolivia

Technical Scope: Child survival social marketing.

Purpose: a multi-study tool, was originally designed for the purposes of evaluating a multi-sectorial public-private partnership for the commercialization of oral rehydration salts (ORS) in Bolivia. The tool measures knowledge, perception and behavioral data among consumers, pharmacists and doctors.

Type of Methods: Qualitative and Quantitative.

Design: Descriptive qualitative studies and cross-sectional household studies.

Frequency of Administration: Every five years for evaluation is suggested; more frequently for assessment purposes.

Key Users of Information

In Bolivia, data was used by members of an inter-institutional task force including donors, MOH, project staff, pharmaceutical companies or other private sector institutions and advertising agencies.

Objectives and Scope of Tool

Impact: To measure change in knowledge, attitudes and practices of target groups (i.e., consumers, pharmacists, doctors, retailers) regarding diarrhea treatment.
Assessment: To gather in-depth data on the knowledge, perception and attitude of target groups toward generic and commercialized ORS; to determine appropriate messages and effective channels of promotion for each target group.
Process: To gather information on the potential sustainability, replicability and effectiveness of the ORS commercialization strategy; to provide an ecological context (social/financial/operational) to the implementation phase of the initiative.

Key Monitoring Needs and Evaluation Questions Tool Seeks to Address

How has the introduction of a commercialized ORS affected the market demand for ORS?
How is the new commercialized ORS perceived by the consumer?
How has the promotional campaign (mass media, interpersonal communication and point-of-purchase materials) affected consumers' knowledge and correct use of ORS and other treatments?
What is the effect of a promotional strategy and campaign on ORS recommendations by pharmacists?
Which aspects of the promotional campaign most influenced the target audiences? How was the promotional campaign perceived by the audiences?
How were prescriptions by doctors for children with diarrhea affected by the introduction of an inexpensive, branded ORS?
How did consumer knowledge and use of non-ORS treatments for diarrhea change after the introduction and promotion of inexpensive, branded ORS?
What are the key variables of the strategy that contribute to its sustainability, replicability and effectiveness, according to stakeholders?

Key Indicators

Knowledge of branded and unbranded oral rehydration salts.
Source of ORS knowledge.
Perception of ORS.
Attitude toward the use of ORS.
Recommendations of ORS by pharmacists to consumers.
Prescriptions of ORS by doctors.
Use of ORS among consumers.
Where ORS is obtained by consumers.
Knowledge of ORS preparation.
Perception of promotional materials.

Research Design

Standard protocol:

Household KAP survey with mothers of children under five: A systematic sample of respondents is taken from urban and rural areas. The sample should be representative of the area of intervention (district/region/nation). Topics include: socio-demographic data on age, income level, education; knowledge and use of branded and unbranded ORS; exposure to promotional campaign messages and print materials; knowledge and demonstration of ORS preparation.
Focus group discussions with consumers: FGDs are conducted in primary regions/districts of interest that contain mothers of children under five who have 1) been exposed to the promotional campaign; and 2) have used the branded ORS. The FGDs are stratified by socioeconomic level of the participants. Topics include: knowledge of and attitude toward ORS and management of diarrhea; perceived benefits and constraints of branded and unbranded ORS; perception of content and presentation of branded product; perception of promotional campaign.
Simulated purchase study: A random sample of pharmacies is taken from a list in each urban/rural area to be sampled (lists may not be available for rural areas). The pharmacies are visited by "mystery clients" who pretend to have a child less than two years of age with simple diarrhea for 24 hours. The interviewers (mystery clients) then ask the pharmacist, or person attending them, for treatment recommendations. After leaving the pharmacy, interviewers record notes of their interaction on a standard form. Topics include: the order and type of treatment recommendation; pharmacists' knowledge of branded/unbranded ORS and diarrhea; recommendations on preparation of ORS; availability of branded/unbranded ORS in pharmacies; existence and location of point-of-purchase materials.
In-depth interviews: Interviews are held with a random sample of doctors and pharmacists. Topics include: knowledge of diarrhea case management, including when to recommend ORS and how to correctly prepare the solution; characteristics of branded ORS; perception of patient demand; perception of promotional materials.
Process Evaluation: Interviews are held with a convenience sample of task force partners (stakeholders). Topics relate to stakeholders' perceptions of strategy, and implementation of the intervention, and the documenting of lessons learned. The purpose of the process evaluation is to understand how operational effectiveness may be improved to ensure sustained impact of the intervention.

Lessons from experience:

KAP study should be conducted on a sample of all socioeconomic levels such that differential impact on various socioeconomic groups can be assessed.
KAP study should use standard format for public health impact questions (i.e., WHO/CDD Household Study, UNICEF MICS survey, DHS).
Market research, epidemiological analysis and evaluation objectives should be clearly delineated prior to questionnaire development and selection of study design such that objectives can be balanced in the same questionnaire if resources are limited. If necessary, oversampling or additional sampling of certain areas may be considered in order to fulfill different objectives.
In countries with many linguistic and ethnic groups, attention should be paid to collecting a representative sample of all groups, and translation of questionnaires into the appropriate languages.

Training

Standard protocol:
All interviewers must be able to speak appropriate local languages, even if the questionnaire cannot be translated due to time or resource constraints. Interviewers for the simulated purchase study must resemble members of the target population, usually women of lower socioeconomic levels. Interviewers for in-depth interviews with stakeholders, doctors and pharmacists require a greater level of comfort with broader subject areas and ease with probing for deeper answers in situations of asymmetrical information (i.e., when one party, usually the interviewee, has more information about the subject area being researched.) Focus groups require moderators to facilitate and guide discussions as well as observers to take notes and operate audio-visual equipment. Preferably, moderators must have a higher level of knowledge of the subject area than focus group participants, and must be able to encourage all participants to speak during the sessions. Interviewers should be able to speak the requisite languages and/or dialects of selected respondents.

Lessons from experience:

Interviewers should be made aware of the research objectives of each study and the overall purpose of the multi-study research.
Interviewers and local researchers should be keenly aware of potential biases that may occur (and how to resolve them) during questionnaire design and implementation.
Simulated purchase study interviewers should be trained well using tricks and hints to better memorize discussions that take place in pharmacies. Exploration into the use of observational techniques (i.e., having a second interviewer enter a pharmacy simultaneous to the principle interviewer and observe the interaction between he/she and the pharmacist).
All questionnaires should be pretested with respondent cultural and/or linguistic groups.

Implementation

Standard protocol:
Several studies can be undertaken simultaneously if financial and human resources are adequate. The length of time needed for each study will depend on local logistics and resources. The Bolivia studies were implemented over a period of four months, including study and questionnaire design, sampling, pretesting, obtaining stakeholder approval and comments, data collection, analysis, report writing and presentation.

Lessons from experience:

Baseline and follow-up KAP studies should be undertaken during the same season to account for varying levels of diarrheal prevalence.
SPS interviewers should vary the route they take in selecting pharmacies to avoid entering more than one pharmacy within the small area in the same day.
SPS interviewers should be discreet and not give away their covert role by visibly displaying their trade tools (i.e., they should not carry tape recorders, display clipbboards or questionnaires to pharmacists, take notes in the immediate vicinity of the pharmacy or enter the same pharmacy more than once).
FGD participants should be chosen based on socioeconomic level. Preferably, none of the participants should know each other. Participants should all have a similar level of knowledge of the topic area.

Analysis

Standard protocol:
Quantitative data is analyzed using SPSS software. Questionnaires are coded prior to data collection and coding structures are created while data collection takes place.

Lessons from experience:

Database structure and coding of questionnaires (questions and responses) should be done prior to implementation of studies.
Skip patterns should be carefully checked with questionnaires prior to implementation and taken into account in the analysis phase.
Dummy tables should be created prior to data cleaning and analysis. If possible, graphics should be developed and selected for presentation prior to analysis.
Research hypotheses and objectives should be clearly stated at the outset. Objectives and hypotheses should be prioritized such that it is clear which objectives may be compromised if necessary.
A report outline should be prepared at the beginning of the study.

Reporting

Standard protocol:
Each study should have a separate report with detailed analysis of the research hypotheses and an overall summary of all results of the multi-study tool. Summary results should be reported in an integrated manner, categorized by topic area and reporting both qualitative and quantitative data.

Lessons from experience:

Quantitative data should be prepared using tables in the final report and graphics for the presentation.
Qualitative narratives should follow topic or content areas rather than question order.
Questionnaires and interview/focus group guidelines should be attached as an appendix to all reports.
Prior to implementation, thought should be given to the exact purpose and nature of reporting, the different types of interest groups who will read the report, and translation of the report into English or other relevant languages.

Dissemination of Results

Standard protocol:
Results should be presented to all stakeholders and interested parties both locally and at the international level. The presentation should integrate key findings from all five studies including any statistical comparisons made to baseline studies. Study designs, methodologies and analysis procedures should be included in the presentation.

Lessons from experience:

All audience participants should be given a summary report of results prior to attending the presentation. The summary report outlines study design, methodology, analysis and a brief review of key findings.
Those researchers who are most intimately involved with the analysis of the data should be present at all presentations if possible.
Presentations should always be made with relevant stakeholders allowing for sufficient question and answer time at the end of results reporting.

Manuals and Guidelines

Currently questionnaires, baseline and final reports are available for the Bolivia AOCI. These reports have not been developed into standardized guidelines as yet, since field tests in other countries have not been planned. If additional testing takes place, it will be possible to standardize questionnaires and guidelines for implementation and analysis. Almost all topic areas are common for ORS interventions throughout the world.

Contact Person
Sandhya Rao
BASICS Project
1600 Wilson Blvd.
Arlington, VA 22209
703-528-7474
sandhya_rao@jsi.com