GMAP

[Full Table of Contents]
[Executive Summary]

[Part IV: The Role of the RBM Partnership] PDF version

  1. Introduction to the Role of the RBM Partnership
  2. Advocacy
  3. Resource Mobilization
  4. Policy and Regulatory
  5. In-country Planning
  6. Financing
  7. Procurement and Supply Chain Management
  8. Communication and Behavior Change Methodologies
  9. Monitoring and Evaluation
  10. Humanitarian Crises

 

Part IV: The Role of the RBM Partnership

9. Monitoring and Evaluation

Monitoring and Evaluation (M&E) has been central to the RBM Partnership’s work from the outset—both within countries at national, district and local levels to track and guide the work, and outside of the country to inform the global and donor community on progress and opportunities. Robust and reliable data are critical for monitoring progress toward achieving the global goals, including the RBM targets (2010 and 2015) as well as the malaria-specific target of the Millennium Development Goals.[34]The list of Roll Back Malaria goals and targets are available on the RMB webpage; MDG 6 focuses on combating HIV/AIDS, malaria and other diseases, and one of its targets is to have halted by 2015 and begun to reverse the incidence of malaria and other major disease. A full list of the Millennium Development Goals, including targets and indicators are available on the UN webpage.

Click for source

Monitoring has been defined as the routine tracking of program performance through record keeping, regular reporting, surveillance systems or surveys. In contrast, evaluation refers to the episodic assessment of a program’s effectiveness, and the extent to which a particular program intervention may be linked to a specific output or result.[35]Monitoring and Evaluation Toolkit: HIV/AIDS, Tuberculosis and Malaria, Second Edition. The Global Fund to fight AIDS, Tuberculosis and Malaria, 2006.

Click for source A strong, combined monitoring and evaluation system will help improve the performance of programs by assessing the degree to which a plan is implemented as planned and how successfully it has achieved its intended results.

The Roll Back Malaria Monitoring and Evaluation Reference Group (MERG) was established in 2003 in order to provide expert guidance on monitoring and evaluation for malaria at the global, regional and national levels.[36]Framework for Monitoring Progress and Evaluating Outcomes and Impact. Geneva, Roll Back Malaria, 2000.

Click for source To date, MERG has provided guidance on technical issues related to monitoring malaria control activities. Notably, this work has resulted in the development of a core set of indicators and standard data collection methods to ensure consistency and harmonization in malaria information reported through major national-level household surveys.[37]Guidelines for Core Population Coverage Indicators for Roll Back Malaria: To Be Obtained from Household Surveys. Calverton, Maryland, USA, Roll Back Malaria, MEASURE Evaluation, WHO, UNICEF, 2006.

Click for source Since 2000, data on these core set of indicators have been routinely collected through the Multiple Indicator Cluster Survey (MICS), the Demographic and Health Surveys (DHS) and the Malaria Indicator Survey (MIS). The MIS was developed by MERG and its partners. Its development informed the malaria sections of both the MICS and the DHS surveys. All surveys are now being implemented in numerous countries across Africa.[38]Malaria Indicator Survey: Basic Documentation for Survey Design and Implementation. WHO, UNICEF, MEASURE DHS, MEASURE Evaluation, CDC, 2005.

Click for source This data collection has allowed for a more comprehensive assessment of progress of key malaria control interventions.

MERG has also provided guidance on important technical issues, such as monitoring malaria-specific mortality[39]Assessing the Impact of Malaria Control Activities among African Children Under Five Years of Age: Guidance Note. Roll Back Malaria MERG, 2006.

Click for source and has supported partners in the development of major tools and reports, such as the Global Fund M&E Toolkit[40]Monitoring and Evaluation Toolkit: HIV/AIDS, Tuberculosis and Malaria, Second Edition. The Global Fund to fight AIDS, Tuberculosis and Malaria, 2006.

Click for source as well as global malaria reports, including the Africa Malaria Report 2003, World Malaria Report 2005 and the Malaria and Children 2007 report.[41]Africa Malaria Report 2003. WHO and UNICEF, 2003; World Malaria Report 2005, Geneva, World Health Organization, 2005; Malaria and children: Progress in intervention coverage. New York, UNICEF, 2007.

Click for source More information on the work of the MERG, as well access to these documents and reports are available at: http://www.rollbackmalaria.org.

As countries improve their control efforts, MERG continually reviews and updates monitoring and evaluation needs (including indicators and systems for information collection) for the evolving spectrum of malaria transmission and disease in existing and changing situations across countries in all regions.

MERG has developed a malaria M&E framework as a guide for “One M&E System for Malaria.” The framework will serve as a solid base for future M&E needs but may require modifications as programs move towards sustained control and elimination.

Figure IV.4 below outlines this M&E framework for disease control, and the different levels of data needed by program managers to assess program performance and to make mid-course corrections, if necessary, to reach intended goals. Importantly, M&E for malaria requires both a stable system that can be utilized to track malaria control programs over time, and a system that can be built on as new needs arise (e.g. monitoring for sustained control and elimination).

Key Challenges

Despite efforts, major challenges remain in monitoring the malaria situation in most endemic countries and for reporting on progress toward global goals. The new call for elimination of malaria places even further demands on these systems. This section highlights some notable gaps in malaria M&E in most endemic countries. For example, most endemic countries have weak vital registration and health information systems that greatly underestimate the number of clinical malaria cases and deaths in the general population. This is in large part because most patients with malaria do not seek treatment in formal health facilities, and many malaria-related deaths occur at home. These systems, however, are useful for informing local programs and may be used to help estimate disease incidence. In addition, there is an urgent need to strengthen the number of skilled malaria M&E staff at the national and sub-national levels, in particular, but also at the regional and global levels as well.

Importance of M&E to improve program management. A strong monitoring and evaluation system will help improve the performance of programs by assessing the degree to which an operational plan or design is implemented as planned and how successfully it has achieved its intended results. Thus, information is needed at different levels for different and specific purposes. Data for global or national tracking should augment efforts to strengthen good program monitoring for local action. In this context, malaria programs should be designed to develop sound routine and interval data collection as well as special focused studies to answer specific questions as they arise. The need for some special studies can be anticipated (e.g. ongoing tracking of intervention efficacy, quality, and safety); others will need to be developed as the need arises.

Strengthen M&E and data collection. Substantial work has been undertaken by the RBM MERG, national governments and country partners in order to improve information available on the malaria situation in endemic countries and to strengthen monitoring systems overall. Notably, this work has resulted in a wealth of new malaria data from DHS, MICS and MIS that has allowed for a more comprehensive assessment of progress in expanding coverage with key malaria control interventions. Routine reporting systems and special studies have been improved; however expanded work in additional countries and over time is required to fully track scale-up and sustained malaria control. For example, enhanced efforts are needed to strengthen routine surveillance systems, and skilled staff is needed at national and local levels, as well as at regional and global levels to collect relevant data, analyze data to present useable information and then support programs to actually make data-supported decisions.

Evolve M&E system from scale-up to sustained control to elimination. As programs evolve from initial rapid scale-up to high coverage, the frequency and focus of data collection should also evolve. As further experience is gained, the RBM partners through MERG will continue to provide guidance using the RBM website. Additionally, MERG will increase focus on developing indicators important for low transmission areas.

Determine efficacy and cost effectiveness through M&E. Studies are needed to clarify the efficacy and cost effectiveness of new interventions to establish their role in the package of malaria control interventions. Standard M&E systems can then be used to track coverage, overall benefit to populations at risk, and coverage shortfalls.

Strengthen funding and human resources. An estimated 5-10% of program costs (or a more carefully calculated budget using local information) should be allocated for the substantial work required in M&E. This funding is needed to support the development of the costed M&E plan, core staff, and the costs for national and local surveys, routine monitoring (including local training and supervision), transport, communications, administration and local and national reporting. A supportive human resources policy environment within the health sector will be required to assure that available finances can lead to quality staff to do the work. Linkage with other program M&E efforts (e.g. reproductive health, Expanded Program for Immunization, HIV/AIDS) will allow shared costs.

Monitor for resistance. Emergence of drug and pesticide resistance will have a major impact on the ability to sustain control efforts. Hence, resistance monitoring should be a priority in order to identify quickly where certain interventions will fail to work, so that new intervention strategies can be implemented where possible.

Priorities

While the core framework for malaria M&E is expected to be robust as countries move from scale-up to sustained control to elimination, some components will likely need additional attention and others may move to a different frequency. The following sections provide brief descriptions of priority issues for the stages from scale-up to sustained control and moving to elimination.

Priorities for scaling up. The RBM Partnership has prioritized scale-up for impact as a focus for all countries between now and 2010. Consequently, malaria M&E systems will need to be strengthened dramatically to document rapid scale-up and to track action in the delivery of interventions at local levels in the country and at the national level.

Priorities for endemic countries include:

  1. Develop and use a costed malaria M&E Plan. The plan should address national and local needs, specify key information and data collection needs, analysis and reporting responsibilities, and link to decision making. This work may draw on the RBM M&E framework and available tools such as the Check List for M&E Plan Development;[42]Monitoring and Evaluation Toolkit: HIV/AIDS, Tuberculosis and Malaria, Second Edition. The Global Fund to fight AIDS, Tuberculosis and Malaria, 2006.

    Click for source     the Global Fund Monitoring and Evaluation System Strengthening Tool,[43]Monitoring and Evaluation Systems Strengthening Tool. The Global Fund to fight AIDS, Tuberculosis and Malaria, 2006.

    Click for source     the Data Quality Assessment Tool,[44]Data Quality Assessment Tool. The Global Fund to fight AIDS, Tuberculosis and Malaria, 2007.

    Click for source     and the Attachment A from Global Fund proposals[45]Proposal Form Round 8, Attachment A. The Global Fund to fight AIDS, Tuberculosis and Malaria, 2006.

    Click for source     to summarize program goals, objectives, indicators, targets, and timelines
  2. Staff and build staff competencies to complete the M&E work within national and partner organizations (e.g. NGOs and community groups)
  3. Undertake national population-based surveys at sufficient frequency to track progress in intervention scale-up, utilization, children under-five mortality, as well as other relevant impact measures
  4. Strengthen health information and vital registration systems so that they may provide more robust and timely data to track intervention coverage at sub-national levels, including district and community levels, as well as to monitor changes in the number of malaria case and deaths. Importantly, these systems will need to incorporate data from private and public health providers
  5. Report on a regular and timely fashion to all stakeholders and share the information widely for global tracking of progress and advocacy
  6. Create a culture of reporting and evaluating relevant data
  7. Ensure regular monitoring and reporting of potential drug and pesticide resistance

Priorities for country partners and the RBM Partnership include:

  1. Support countries in each of the areas listed above with particular attention to capacity building at national and local levels
  2. Strengthen international mechanisms to compile relevant malaria information from endemic countries, including tracking progress and disparities in intervention scale-up and other health and socio-economic indicators. This information should be jointly reported by partners to monitor progress towards global targets
  3. Report on changes in malaria burden. While the long-term goal is to strengthen routine reporting systems in endemic countries, there is an immediate and urgent need to report on changes in the malaria burden since such information is critical for improved program management and to monitor global goals. To this end, a model has been developed that estimates malaria-specific mortality based on coverage estimates and their known efficacies. This model is now being developed into a user-friendly software for countries to use. There is an immediate need to implement and train NMCPs in the use of this software
  4. Support countries in their development of a costed M&E plan. Track resource allocation, actions and progress from the partners to assure that M&E needs and gaps are addressed as laid out in this plan

Priorities for sustained control and elimination. Following the rapid scale-up to high coverage that is to be achieved by 2010, the RBM Partnership has set priorities to help countries sustain systems for high intervention coverage and move toward elimination by 2015. Support for M&E to track these stages will require continued strengthening of systems noted above, and additional work to support specific sustained control and elimination needs. Several aspects of this “additional work” can be anticipated: it will need to draw increasingly on available technologies of mapping and communication to track and respond to intervention gaps and infections and illness; it will require increasing capacity at local levels (districts and communities) for information and for response.

Priorities for endemic countries include:

  1. Continue to strengthen all systems described above under the scale-up stage
  2. Establish or strengthen systems at district and community levels to optimally track and immediately resolve local gaps and needs in intervention coverage
  3. Strengthen local systems to undertake surveillance and active case detection of malaria infection and illness and to connect to immediate response systems to investigate and address local transmission.
  4. Staff and build staff competencies to support this enhanced work at local levels
  5. Report regularly to all stakeholders. If routine systems provide robust and timely coverage and impact data, then the need for model-based estimates of the malaria burden and frequent national population-based surveys would decline. As the country moves closer toward the goal of elimination, there may also be less need for routine facility-based monitoring systems to track the numbers of cases. However, the need to regularly report to all stakeholders and share the information widely for global tracking of progress and advocacy will continue.

Priorities for country partners and the RBM Partnership include:

  1. Continue to strengthen all systems described above under the scale-up stage
  2. Support countries in each of the additional areas listed above with particular attention to capacity building and technology support at national and local levels
  3. Increasingly move national support systems to the sub-regions to enhance timely response to national and local needs
  4. Develop core indicators for countries with low incidence as value of survey data declines

Organizational Implications

MERG has been responsible for providing M&E guidance to countries and the RBM Partnership. In the context of growing demands for information to track country progress, MERG may need to further expand its mandate to undertake the RBM Partnership’s work and to support the country work. This support includes helping countries develop costed national M&E plans, and training qualified workers. These needs will likely increase as countries transition from scale-up to sustained control to elimination, considering the financing, staff and capacity building required at district and community levels.

Table IV.10: Summary of Monitoring and Evaluation Activities

Reference to priority Major actions Completed by Coordinator (bold) / Sub-coordinatorsa
A, D, H, K Support the development of costed M&E plans through regional and country support using MESST, M&E plan checklist, the GF Attachment A and other tools and templates Ongoing MERG
B, F, O, R, S Examine and address M&E human resource needs at national, regional and global levels tbd RBM partners (via MERG)
L, M, N, Q, T Continue to strengthen M&E guidelines for scale-up and expand the scope to address sustained control and elimination tbd MERG
C, E, I, J, P Establish and maintain global and regional tracking of malaria control progress* Ongoing UNICEF, WHO, MERG

a) Main coordinating group / body in the RBM partnership indicated in bold. Closely linked contributors within the RBM partnership are also listed. RBM partners are not listed explicitly as their involvement occurs through the Working Group.
* The major decision by the RBM Partnership to effect the "Partnership-support for M&E" will be to determine what investment (financing and staff) the Partnership and partners are prepared to make for what specific outcomes. To date, the MERG as an advisory / technical group represents modest partner-donated staff time and travel for a specific set of guidance and oversight actions. The MERG members have worked hard, but they are not dedicated staff that can be assigned a partnership workplan. The partners have a clear thirst for information, often driven by internal requirements, but have not invested in a joint partnership system to support the core and shared information collection systems. Thus the resources (if they exists) are provided by individual grants or donor support or national systems at country level. This process does not easily translate to address the need for a comprehensive data / information system that tracks country and regional progress, intervention coverage, disease burden reduction, economic investment and return on investment, etc.