SECTION IV:
IMPROVING ROLL BACK MALARIA MONITORING AND EVALUATION—THE WAY FORWARD
The data presented in this report illustrate not only the progress made in recent years in monitoring of malaria control but also identify several gaps and limitations in available data and challenges that remain in data collection efforts. This section first reviews the overall requirements for malaria monitoring and evaluation in different world regions. Recent progress is then highlighted and recommendations are made for improving data collection and reporting in the future at country, regional and global levels. Where relevant, reference is made to the ongoing work of the RBM Monitoring and Evaluation Reference Group (MERG), which is described in more detail in Annex 4.
1. Overview of Roll Back Malaria monitoring and evaluation
The goal of a national RBM monitoring and evaluation system is to provide reliable information on progress in controlling malaria that can be used at local and national levels and can inform regional and global efforts. The corresponding specific objectives are:
- collect, process, analyse and report on malaria-relevant information;
- verify whether activities have been implemented as planned to ensure accountability and address problems that have emerged in a timely manner;
- provide feedback to relevant authorities to improve future planning;
- document periodically whether planned strategies have achieved expected outcomes and impacts.
The basic monitoring and evaluation framework shown in Box 1 in the Introduction outlines the inputs, outputs, processes, outcomes and impact indicators that should be tracked in a good monitoring and evaluation system. However, in limited-resource settings, experience has shown that priorities must be established. The highest priorities include tracking:
- human and financial inputs;
- malaria control services delivered to those at risk of malaria;
- the coverage of the interventions;
- measures of mortality and malaria-associated morbidity.
2. Key Roll Back Malaria coverage and impact indicators, by region
Given the differences in malaria epidemiology, appropriate intervention strategies and the design and quality of HIS, appropriate RBM indicators also differ somewhat between regions. The major distinction is between Africa south of the Sahara and similar environments such as Papua New Guinea where malaria is highly endemic throughout countries, and the rest of the world, where malaria is more unstable and focal in nature (Table 9).
Table 9. Examples of appropriate Roll Back Malaria impact and outcome indicators, by type of malaria endemicity   |
3. Recent progress in monitoring
In recent years, progress has been made in standardizing core indicators between countries and regions and in setting up sustained efforts for measuring these indicators regularly over time. This section focuses on household surveys, surveillance of drug resistance and procurement data.
• Household surveys
Community-based (household) surveys on intervention coverage are conducted in an increasing number of malarious countries (Table 10). The national-level MICS and DHS that are conducted at 5-year intervals now include questions on malaria, specifically in relation to the coverage of ITNs and on antimalarial treatment of fevers/malaria illness in young children (10, 11). Since 2001, these questions have been grouped into standard malaria modules that are included in surveys in all malarious countries, allowing valid comparisons of coverage levels between subsequent surveys within a country as well as between countries. In 2005–2006, approximately 46 malaria-endemic countries (of which 30 are in Africa) will have an MICS and an additional 29 malaria-endemic countries (of which 16 are in Africa) will have a DHS.
• Service delivery
The delivery of malaria-related services to populations at risk is being monitored by many NMCPs and other agencies involved in implementing control activities in countries. Indicators include the number of ITNs distributed or sold, ITN (re-)treatments provided, quantities of insecticides used for IRS and quantities of drugs supplied (Table 11). Between 2000 and 2003, the number of countries that reported the number of households or units using IRS increased.
• Surveillance of antimalarial drug resistance
Surveillance systems that monitor the efficacy of locally used drugs have been set up in most countries with endemic falciparum malaria (Table 12). Standardized, high-quality drug efficacy surveillance is being promoted through subregional initiatives in the Mekong (69), the East African Network for Monitoring Antimalarial Treatment (70), the Horn of Africa Network on Monitoring Antimalarial Treatment (71) and the Amazon Network for the Surveillance of Antimalarial Drug Resistance (RAVREDA) (72). In addition, many NMCPs are developing and strengthening national networks to monitor the efficacy of antimalarial drugs—including combination therapy—for the treatment of falciparum malaria, and, to a lesser extent, of vivax malaria. RBM support for these networks includes assistance in choosing appropriate sentinel sites, training and strengthening reference laboratories for quality control and data analysis.
Table 10. Survey availability on mosquito net possession and use, 1999–2004   NA = not applicable because ITNs are not part of the national malaria control policy of any North African country. Surveys are classified as national or subnational based on sampling frame design and in relation to the local distribution of malaria burden. National surveys include DHS (11) and MICS (10); subnational surveys include those conducted by NetMark (in Africa) (12) and PSI (13). |
Table 11. Number of countries reporting on status of key service-delivery activities, by national malaria control programmes, 2003  |
Table 12. Number of studies available of antimalarial drug efficacy against falciparum malaria that meet WHO protocol (9), by region, 1996–2004 
a Includes ACTs other than those listed separately in other columns in the table. Available results cover all countries with endemic falciparum malaria except Comoros, Djibouti, Sao Tome and Principe, all 10 Central American countries and Paraguay. |
4. Limitations in available data and recommended improvements
Table 13 lists a number of important limitations in the availability of data and in the interpretation of the data presented in earlier sections of this report. Based on these limitations, coordination among monitoring and evaluation stakeholders and capacity for the standardized collection of quality data should be improved. This is true for many levels but first and foremost at country level, where most of the data originate.
Disease burden and impact
In high-burden countries with poor access to health care and with inadequate disease surveillance systems—in particular in Africa—major investment would be required to improve the quality of both HIS and access to health services, before the utility of HIS case and death reports for monitoring malaria disease trends could be assessed. Malaria case reporting under the system of Integrated Disease Surveillance and Response is in various phases of implementation in 36 African countries (36); this system remains to be evaluated for its reliability and completeness.
Apart from access to care and information systems, an inherent problem of malaria case reports in high-endemic Africa is that the appropriate definition of what a case report consists of is not obvious. In the absence of laboratory capacity in those areas where malaria is most prevalent, most diagnoses and treatments occur presumptively (on purely clinical grounds); for the vulnerable group of children under 5 years of age, presumptive treatment is in fact recommended in order not to delay potentially life-saving treatments (32), although clinical malaria might not be the most appropriate definition for purposes of monitoring. However, even if all clinical diagnoses were confirmed by parasitaemia testing, the diagnosis would still not have optimal specificity, because asymptomatic parasitaemia is common, so that a fever accompanied by parasitaemia does not necessarily indicate a fever that is caused by malaria. Despite these problems, HIS data are useful for local programme planning, in particular for forecasting drug supplies needed for delivery through the public sector, in all countries.
For disease trend monitoring in high-endemic countries, population-level data are thus indispensable. To supplement available data on all-cause under-5 mortality, the prevalence of childhood anaemia and malarial parasitaemia are potentially useful survey-based indicators. Because under-5 mortality measured in crosssectional surveys refers to the mortality rate over the 5 years preceding a survey and thus lags behind for the detection of any trends that started less than 5 years before, anaemia and parasitaemia prevalence would allow for a more rapid detection of impact (20, 73). For surveys of parasite infection rates to be useful, these should be conducted during or immediately after the peak transmission season (Annex 4).
For African countries that are approaching the Abuja targets of 60% coverage with ITNs and prompt and effective treatment, evaluating the trend in malaria-specific mortality will also become relevant. This could be done in representative, small-scale sentinel demographic surveillance sites based on verbal autopsies (18) (Annex 4).
In areas where overall health-care systems are more developed, where the majority of patients with malaria access the formal health-care system, and where malaria diagnoses are generally laboratory-confirmed, malaria cases and deaths reported through HIS are important burden and impact indicators. Case reports split by age group are useful for forecasting drug supply needs in different dosages and formulations.
It is crucial, however, to understand the completeness of HIS reporting and how the completeness might change over time. Between 2000 and 2003, the global annual number of reported cases averaged 48.3 million. These case reports came from between 77 and 100 of the 107 malaria-endemic countries and territories in a given year (Table 2). Compared with WHO’s estimate of 350–500 million cases in 2004, HIS would detect globally 10–14% of actual malaria cases (2). However, this percentage would be the average in some countries where HIS overreports malaria and in most other countries where HIS detects much less than 10–14% of cases.
In comparison, of the 107 malarious countries and territories, 10 provided their own reliable estimate of HIS reporting completeness in 2003 (Annex 1). These estimates ranged from 20% to 100%, but the definition of completeness was not always specified and probably varied between countries.
WHO is planning to assist countries in establishing standard definitions and methods for assessing the completeness of HIS reporting. Such assessments should take account of the extent to which the national HIS covers malaria cases that are treated in the private and informal sectors. The number of districts or other relevant subnational units with malaria that reported on malaria cases each month should also be considered. Rapid diagnostic tests, as an additional tool for laboratoryconfirmation of malaria diagnosis, may in future years help to ensure the quality of malaria case and death reports (66).
Vital registration systems that record causes of deaths are an important complement to HIS data, and the coverage and quality of vital registration must be promoted (Annex 4).
Intervention coverage
RBM is working with WHO, UNICEF, Macro/DHS and other international survey agencies to coordinate household survey activities and to further standardize methods, questionnaires and analysis plans for assessing relevant malaria indicators. Planning and implementation of household surveys are being monitored through the RBM MERG for identification of countries that need assistance and financial support (Annex 4). To supplement the data collection from DHS and MICS, in 2004 the Malaria Indicator Survey (MIS) was developed for the standardized assessment of core RBM coverage indicators.5 The MIS package contains standardized, bestpractice survey methods, questionnaires and analysis plans. A MIS could be used to design malaria surveys in countries where no other surveys are being conducted, or to fill gaps within the 5-year intervals between subsequent DHS or MICS, for a more rapid detection of progress. A scaled-down version of MIS is also available, called the standardized “lean malaria module”, with standard questions on malaria intervention coverage that could be added to other planned household surveys.
Recent improvements in the questionnaires of DHS, MICS and MIS included the addition of standardized questions on promptness and dosages of antimalarial treatment. The next round of MICS, in 2005–2006 in around 46 countries with malaria, is therefore expected to provide the first multiple-country dataset allowing a valid assessment of the coverage of prompt and effective treatment of young children. Also, levels of household possession of ITNs, the most important ITN coverage indicator in countries outside Africa, will be routinely collected from 2005 onwards.
Available surveys and survey designs do not fully address the need for coverage data. First, there is presently no standardized tool for measuring the coverage of antimalarial treatment in Asia and the Americas. Unlike in Africa, survey data on the treatment of children with fever are not optimally informative in areas where only a small proportion of reported fevers are actually caused by malaria, and where children under 5 years of age are not the only or main risk group for malaria. In these settings, surveys should measure treatment-seeking behaviour in older age groups as well, and using “all fever episodes” as the denominator would be less appropriate.
Second, for IPT coverage, a control strategy that is still in its first few years of scaling up, facility-based surveys in selected areas where the policy has already been implemented may at present be a more appropriate measurement method than are national surveys. Because antenatal clinic attendance is high (> 80%) in many of the African countries where IPT is policy (Fig. 12), antenatal clinic attendees can be expected to be a representative sample of the population targeted with IPT. A further advantage of facility-based surveys above household surveys is that the former provide more timely data. This is because surveys typically have to rely on data about previous pregnancies, since the number of respondents being pregnant at time of the survey is small.
IRS delivery and coverage
Also urgently lacking is a standardized measurement and operational definition of IRS coverage, which is why this report did not present data on this issue. Several countries conducting IRS reported an estimated IRS coverage for at-risk areas to WHO regional offices, and many centralized IRS programmes maintain detailed household listings of targeted spray areas. However, the definition of IRS coverage is not yet standardized across the world. Countries and regions vary in whether to define “coverage” in terms of geographical area, numbers of houses or household structures sprayed or numbers of people living in sprayed houses. They also vary in whether populations at no or low risk are included in the denominator, in the definitions of population at risk and the source of population data used, and in whether to apply a minimum threshold frequency of IRS.
In the absence of data on houses sprayed, IRS coverage could alternatively be estimated from quantities of insecticide used for IRS, by assuming a specific application rate for each insecticide and an average sprayable area per house, e.g. 250 m2 (74). However, annual collection by WHO of country data on this service delivery indicator was very incomplete from 2000 to 2003 (63). In areas where spraying programmes are highly decentralized or where monitoring efforts at the national level are less developed, the inclusion of questions on IRS coverage in MIS might prove useful
.Drug efficacy
A challenge for drug efficacy monitoring, especially in countries not covered by the above initiatives, is to ensure appropriate documentation of studies to allow determination of whether study designs followed the recommended WHO protocol (9). In regard to the massive implementation of ACTs, the effectiveness of these therapies must be closely monitored. Reference laboratories must be set up that can coordinate with the NMCPs. Finally, if possible, countries should also use in vitro testing and molecular markers to study the resistance to each of the component drugs individually and as an early warning system that could detect the development of resistance earlier and with greater sensitivity than clinical testing.
Forecasting supply needs
The Malaria Medicines and Supplies Service, an initiative of the RBM Partnership established in 2004 to facilitate access to high-quality and affordable antimalarial medicines and other essential supplies, will set up a monitoring system for the manufacturing and global sales of drugs (64). Whereas the Malaria Medicines and Supplies Service now provides a unique oversight on pricing and supply management at global level, particularly with respect to drug production, at country level greater efforts are needed, especially in the area of monitoring drug usage, demand and regulation of drug supplies. An evidence-based, standardized approach to forecasting drug supplies should be developed. The forecasts should consider the needs for treatment services through the formal health sectors—public and private—as well as through channels such as home management.
Financial resources
Monitoring financial resources for malaria control activities is important for ensuring that adequate resources are committed and sustained, and that health budgets are allocated among districts and programmes proportional to disease burden. Raising the estimated annual US$ 3.2 billion necessary to support the minimum set of malaria interventions in the 82 most malarious countries (38) will require coordinating financial information from national governments, the GFATM, the corporate for-profit sector, bilateral agencies, NGOs, international foundations and multilateral development organizations.
At country level, it is particularly difficult to track out-of-pocket expenditures for treatment and prevention and public funding embedded in the provision of general public health services, including, for example, health centres and hospitals where malaria cases are treated. For African countries, finance monitoring should include tracking progress towards the target reaffirmed in the Maputo Declaration of July 2003: 15% of national budgets should be allocated to the health sector (59). For donors, monitoring of financial resources for malaria is essential to ensure that the pledged resources are in addition to current assistance levels (57). This is explicitly acknowledged in the mission statement of the GFATM, which “only finances programmes when it is assured that its assistance does not replace or reduce other sources of funding, either those for the fight against AIDS, tuberculosis and malaria or those that support public health more broadly”.6
Table 13. Selected issues related to the interpretation of available data on malaria monitoring presented in this report   |
