External Evaluation of ACT Alliance Response Typhoon Haiyan - PHL 131

Background:

There were five members of the ACT Alliance that made a joint appeal for the Typhoon Haiyan, including NCA and our local partner, the National Council of Churches in the Philippines (NCCP). NCA implemented WASH activities; water systems, latrine construction, hygiene promotion, hygiene kit, in cooperation with NCCP and its member churches and other ecumenical partners.

Purpose/objective:

• To assess to what extent the results planned were achieved.
• Review ACT members’ appeal funded response against OECD-DAC criteria (excluding impact).
• To identify lessons learnt and best practices, including innovations/new systems developed for continuous programme improvement of the ACT Philippines Forum and the ACT Alliance.
• Determine how resilience to disasters has been increased in programme communities.

Methodology:

The evaluation was carried out by a team of three external evaluators. The team visited 21 project sites, sampled by the evaluation team in co-ordination with the requesting members. Data was collected mainly through review of internal and external documents and interviews with key stakeholders. The majority of interviewees were beneficiaries.

Key findings:

• The relevance of the planned and implemented activities was found to be high and in line with beneficiaries’ priorities. Food and non-food items were regarded as of good quality and were distributed in quantities according to humanitarian standards.
• Timeliness was good in the initial phase, while later activities experienced significant delays. These were sometimes caused by government procedures or land issues, sometimes by logistics or market shortages and were in other cases due to inflexible internal systems or staff inexperience with large scale responses.
• Effectiveness was generally high for activities that were implemented according to plan; WASH activities contributed to better hygiene, DRR activities provided a sense of safety for future disasters etc. However, there were examples of materials being of poor quality, either as a result of insufficient knowledge of project staff regarding quality control, or due to improper storage.
• The ACT response brought value added in terms of in-depth contextual knowledge and heavy investment in community relations. Pre-existing networks added value by allowing a rapid scaling-up of activities, drawing value from relationships with external stakeholders and all levels of government. The members’ multi-year experience of development programming in the affected areas increased connectedness and the likelihood of outcome sustainability.
• External coordination: The ACT response invested significantly at all levels in coordinating with government and cluster system structures. This includes significant investment in coordinating with local government, regular attendance in national and local cluster meetings and an awareness of and coordination with existing technical support structures.
• The team found multiple examples of increased resilience to disasters in programme communities, such as increased awareness of and willingness to use evacuation centres as well as structural improvements in shelters.

Recommendations:

• Develop a strategy to adjust programming to available capacity. For example, if technical skills are lacking in a particular field, such as construction, programming should avoid this field and focus on areas where there is a comparative advantage such as community-based approaches (for example avoiding latrine construction and instead focusing on hygiene promotion).
• Members should accept full responsibility for the quality of programming implemented with funds supplied by them, whether this is done by the member itself or a local partner. This implies a duty to limit programming funded to realistic levels. It also implies an organisational responsibility to appropriately resource monitoring and follow-up structures involved in local partnerships and to strategically plan for exits.
• Develop a clear monitoring and evaluation system that complements that of their implementing partners. Clear roles and responsibilities in Monitoring and Evaluation between ACT members and implementing partners should be defined.

Comments from the organisation:

There are several observations in the evaluation report that NCA will take into consideration for future humanitarian response. Some of the lessons learned that NCA notes:

• The evaluation confirms that Act Alliance's extensive local networks, contextual knowledge and heavy investment in community relations brought significant value added to the response and in increased the potential sustainability of outcomes.
• According to the evaluation several of the implementing partners were overstretched, partly caused by overambitious programming and a mismatch between implementing partner ambitions and ACT member accompaniment resources. Project proposals have been accepted despite being unrealistic in terms of reach and/or complexity and in some cases there.
• The evaluation suggests that NCA focuses on transferring capacity in a more systematic and conscious manner to partner organizations such as NCCP

NCA will enter into partnership according to capacity to meet humanitarian needs. Our operating model will always include close cooperation and consultation within the forums of the ACT Alliance as well as local partners. When the humanitarian imperative demands us to set up operational responses we will do so to fill gaps and meet immediate humanitarian needs. We believe that strengthening of local partners will need to begin with emergency preparedness planning, as capacity building during first phase of disaster response has proved difficult.

When we enter into partnership with a local partner during an emergency response (like in the Philippines), we will aim to have realistic programming based on their implementation capacity as well as set clearer objectives for capacity development and follow up.