Basic Health: End of Program Evaluation

Background:
Plan Zambia is implementing a Child Survival Project with support of Plan Norway and funded by NORAD as part of the Framework Agreement (2008-2010): GLO 0742 – GLO 07/385. The core elements of this program include Safe Motherhood, Child Health i.e. in prevention and management of childhood illnesses, Water and Sanitation also as a key factor for fighting diarrhoea in children and HIV/AIDS prevention. The Plan Zambia Country Program on Basic Health is part of the Country Programs implemented within Plan Zambia Country Strategy - 2004-2009. It must be noted that the NORAD supported program is a sub set of this program under review, which was already on going at the time that NORAD support was sourced. The program goal for the Basic Health Program is to improve the quality of health among children and mothers. More specifically the aimed to:

• To reduce the incidence of preventable childhood illnesses
• To reduce water-borne diseases through increased access to safe and clean water and sanitation
• To increase the percentage of women of child bearing age with access to antenatal care
• To increase the percentage of adolescents taking appropriate measures to avoid HIV /AIDS /STDs

This evaluation summary will however focus on only the findings and recommendations of the program components that directly relate to the specific objectives /intentions of The Child Survival Program as covered within the Basic Health Program namely: ensuring a safe and healthy early childhood development as well as safe healthy adolescent sexual and reproductive health. Since this evaluation has been completed, Plan Zambia has already drafted new Country Programs within which will be the programs that will continue to carry through the aspirations of the NORAD funded Child Survival Program. A separate evaluation will be carried out for the Child Survival Program in its final year of implementation (2010). That evaluation will be more focussed on the goals, objectives and interventions of the NORAD support.

Purpose/objective:
The purpose of the evaluation was to provide Plan, its partners and stakeholders an assessment of the extent to which the objectives of the program have been achieved. Specifically, the evaluation aimed to ascertain the extent to which the program goal has been achieved. It also aimed at informing at development of the next program cycle. The evaluation reviewed the program and its activities that were implemented from July 2004 for their relevance, effectiveness, efficiency, sustainability and impact in relation to the issues that were identified and set objectives.

Methodology:
The evaluation used document review / desk study which among other things included review of program documents and field work which involved conducting of interviews with key informants at national, district, program unit, and community level and focus group discussions with beneficiaries at community level. The evaluation also conducted a Knowledge, Attitudes and Practices Survey with selected participants in the communities using a structured questionnaire.

Key findings:

1. In general, the evaluation observes that Plan has contributed to increased access to basic primary health care services partly achieved through health worker training and construction of community health facilities.
2. More specifically the evaluation concludes that:
   - Plan has contributed to improved access to child health services by increasing awareness about child health issues e.g. malaria (increased access to insecticide treated nets) and nutrition, training first line health volunteers for quick access to treatment, and constructing rural health centres. Extensive Immunization coverage is also reported to have improved though it was noted that coverage for measles is still low and yet this is critical for child survival.
   - There is increased access to clean and safe water for many members of the communities, water and sanitation coverage in the communities has increased. The incidence of diarrhoeal diseases was also reported to have reduced in the communities as perceived by the respondents. 
   - In the area of Safe Motherhood, the evaluation records that access to ANC and maternity services has been increased due to reduced distances to reach these services, access to preliminary assessments by TBAs who serve as referral agents has increased, newborn deaths have decreased, ANC attendance has increased and maternal mortality has reduced.

1. In the area of HIV/AIDS prevention, the evaluation observes that prevalence of HIV in all the communities was reported to have declined. The VCT uptake has increased and stigma and discrimination has reduced as evidenced by many people who are reportedly coming out in the open to declare their status and joining support groups. Awareness about PMTCT and PMTCT services is also reported to have increased.
2. The evaluation further notes that through Plan’s involvement in this program, there has been increased access to basic integrated Primary Health Care (PHC) services. However, it was observed that other areas of behavioral change are still lacking e.g. consistent ownership / use of latrines, use of safe water especially where the water taste /quality is perceived to be poor by the communities.
3. Lack of clear Monitoring and Evaluation Framework with objectively verifiable indicators to clearly measure progress towards targets, challenged program ability to measure its performance and effectiveness in quantifiable terms. As a result the basic source of progress monitoring were the meetings / discussions and descriptive quarterly reports.
4. In general the Evaluation observed that there is poor readiness of the communities and other stakeholders to take over and sustain the work that Plan has started / invested in the communities beyond Plan’s presence and support.

Recommendations:

1. In order to improve behavioural change in specific areas identified e.g. use of latrines and also in respect to HIV prevention, the evaluation recommends to Develop strategies to build on high awareness to cause actual behaviour change and changes in practices for implemented activities to minimise chances of regression to old behaviours and practices. A Study on borehole water quality and reasons for low uptake is also recommended for some programme areas.
2. The evaluation also recommends hiring of field level staff with clinical knowledge for enhancing the effectiveness of the Child Health and Safe Motherhood components of the program. The evaluation also observes that the current bulk of work load over the field based Health Coordinators may also have contributed to reduced effectiveness of the program based on the fact that the field health coordinators were charged with responsibility for overseeing interventions in water and sanitation, child health, safe motherhood and HIV and AIDS. The need to establish focal point persons for maternal and child health, HIV and AIDS, and water and sanitation at district office and community level has also been recommended.
3. The evaluation recommends the need to develop standardised performance indicators and targets aligned to national and MDGs performance indicators and conducting of program area specific baseline survey upon which future monitoring and evaluations will be benchmarked and tracked for progress.
4. In order to improve program sustainability beyond Plan presence, the evaluation proposes a clear exit strategy to be designed with communities / stakeholders.

Comments from the organisation, if any:

1. Comment on the overambitious targets that overlook the fact that programs like these succeed with high level political will and also availability of technical staff for delivering such interventions.
   1. This observation from the evaluation has been considered and the next Country health program for Plan Zambia plans to focus on mobilising resources at all levels for specific technical interventions. In the event of available grant funding, aspects for hiring competent personnel will be integrated in as much as such will lead to strengthened and more sustainable programs. The programs will also work more in strengthening Civil Society participation at various levels to lobbying and advocating for strengthened political will including ensuring increased budgetary allocation and retention of professional health workers by the government.
2. Comment on how to deal with the myths and stigmas e.g. regarding use of condoms, rusty water, and also women delivering at home because there are male midwives at the health facilities etc
   - “Rusty water” is due to ground water having a high iron and manganese content especially in certain parts of Mansa district. Plan under its water and Sanitation Program (WATSAN) plans to construct an iron removal plant alongside continuous awareness raising and health promotion to remove myths surrounding that.
   - Wrong perceptions regarding condoms use and women choosing to deliver at home because of male midwives being responsible for delivery at health facilities: Plan Zambia has integrated and addressed these issues in the new Country Strategic Plan using various strategies to increase awareness, inform, educate and communicate. Action groups for various peers groups will be strengthened including children groups, youth groups and community-based organizations in participatory advocacy skills in order to increase their capacity to demand for health services, including sexual reproductive health services. Plan will conduct in-depth research to facilitate effective and efficient programme design and delivery.

3. Comment on low Prevention of Mother to Child Transmission (PMCT) and lack of youth friendly Adolescent Sexual and Reproductive Health (ASRH) services (p39/90)

1. Limited access to and utilization of sexual reproductive health information and services especially in the hard to reach areas:  There is a big demarcation between the urban and rural areas where access to sexual and reproductive health services is concerned. Knowledge of HIV and its transmission is still low among women and men who have less education and live in rural areas[1]. Contributory factors include inadequate information, poorly trained health personnel to handle the increased disease burden related to HIV and AIDS, non-integration of STI services into other HIV and AIDS and reproductive health services, and inadequate quality youth friendly health services.  Young people do not have the freedom and confidence to access and demand for health services they need espercially in the rural areas. Other factors include male dominance, stigma and discrimination, negative cultural and traditional practices coupled with negative peer pressure and poor economic status of women prevents young women them from demanding and negotiating safer sex. Plan Zambia has put strategies in the next Country Program to address these observations.