Endline evaluation of NRK funded Telethon WASH project in Districts Sanghar & Thatta, Sindh, Pakistan

Background

Norwegian Church Aid (NCA), with support from NRK Telethon funded and implemented a Water, Sanitation and Hygiene (WASH) programme to strengthen community resilience to flood, and improve access to safe drinking water, and provide adequate sanitation and hygiene for targeted communities in Sanghar and Thatta, Sindh province, Pakistan. The five-year long programme was implemented by NCA’s local partner Research and Development Foundation (RDF) which ran from May 2015 to Dec 2019. GLOW Consultants, a specialised research firm, was commissioned by NCA to conduct a third-party end-line evaluation of the programme.

Purpose/ Objective

The main purpose of the evaluation is to understand the extent to which the proposed objectives and outcomes were achieved in terms of DAC criteria which includes relevance, effectiveness, efficiency, and sustainability as defined below:

• Relevance: is the extent to which key stakeholders were involved during programme implementation.
• Effectiveness: is the extent to which programme coverage, whether access to water, sanitation, and hygiene services increased during these ventures or did not increase.
• Efficiency: is the extent to which the targeted communities effectively participated in programme implementation.
• Sustainability: covers three important themes: social, environmental, and operation and maintenance. Of these themes, WASH maintenance most affects the sustainability of the intervention. While due attention was to be given to institution building to institutions that operate, maintain and manage the WASH facilities.
• Cross-cutting issues: that covered Gender Analysis, Disaster Risk Reduction, and Grievance Redressal System

The objective of the end-line evaluation was to assess programme design and implementation with emphasis on results and achievements as compared to the findings of the baseline study conducted in the year 2015 and, other assessments conducted in the area along with village profiling done each year.

Methodology

The evaluation methodologies were broadly guided by the Terms of Reference (ToR) and refined by the evaluation team in consultation with NCA at the inception stage. The key features of the methodology are:

1. Review of Relevant Documents / Secondary Data
2. Field Visits for primary data collection
   a) Key Informant Interviews
   b) Focus Group Discussions (FGD’s)
   c) Transect Walks

Key Findings

The following are the key findings of the evaluation exercise:

• The NCA programme was implemented in the overall context of competing humanitarian priorities in the targeted districts as well as shrinking space for the INGOs in the country. The approach adopted by NCA allowed to successfully navigate both these challenges. NCA’s partner was successfully able to gain and retain access through close engagement with relevant government offices.
• Access to beneficiaries, including women and minority groups in general, was not an issue. This allowed better engagement with women and minorities as compared to other parts of Pakistan including areas within Sindh, where access to these groups has been more limited. However, despite these achievements it was challenging for women to assume a more proactive role in the delivery of programme activities. They were generally reluctant and were asking the men in the community before sharing their own opinions. This reflects the need for more comprehensive community mobilization framework which may use men as agents of change for bring a behaviour change in women.
• The programme districts experienced frequent changes in government officials. This meant that the programme team had to re-start their conversation whenever there was a change. To cope with the situation, the programme team focused on second tier staff who were less likely to be changed frequently, however they had limited decision-making power.
• There were some area-specific challenges which affected programme implementation. For example, it was difficult to find a constant ground water source as water quality changed its quality with seasonal changes (e.g. in monsoon season or in dry seasons), and the water table often fell due to less rains in both districts. Similarly, in desert areas, communities also faced shortage of water for construction, where lack of sufficient water slowed down the construction of latrines. Waterlogging was also an important issue that affected availability of suitable sites for latrine construction. Such factors have to be considered in future designs.
• Although most of the programme staff was part of WASH programmes for years, it would add value to their knowledge if refresher workshop/trainings on Pakistan Approach to Total Sanitation and other related topics were to be conducted as part of this programme. Relevant government officials can also be invited to take part in these trainings.
• Water tariffs were introduced to ensure sustainable access to clean drinking for communities, however there were not many people who were willing to pay for water. This was mainly linked with low affordability and the possibility that in these communities safe drinking water was not prioritised over other economic needs. The engagement of local decision makers may positively affect this situation.
• Due to poverty and other related issues such as availability of space, not all beneficiaries were able to build a pits/septic tank for themselves.

Recommendations

Based on the evaluation findings, the following key recommendations were received by NCA from the consultant.

• It would bring added value to future interventions of a similar nature to explore installation of water meters to regulate solar powered water supply schemes and to ensure that there is no over extraction of water. Carrying out ground water re-charge study can also be beneficial in assessing the actual situation and designing the interventions.
• There is a need to further invest in the capacity building of water management committees and to consider including new members as part of the committees such as schoolteachers, elected members and, staff of PHED in the committee.
• As under-ground water quality is a major issue, NCA may explore potential innovative practices from similar contexts (e.g. Bangladesh for areas with high water table / high turbidity) to explore new ways to ensure provision of clean drinking water for the targeted communities.
• As latrines are also utilized for bathing purposes especially by women and girls. It might be a good idea to consider bathing needs and separate provision for draining the wastewater generated through bathing activity in the design as well.
• Linkages can be further strengthened with line departments by providing them more opportunities to be involved in the programme activities.

Comments from Norwegian Church Aid

Through management response, NCA responded to the finding / observations of the consultant by responding with our agreement, partial agreement or disagreement. Following are the response provided by NCA.

• Since NCA is not providing piped water to every household therefore, installation of water meters for HH is not possible.
• NCA agrees that over harvesting of water is discouraged to save water resources which is why only a small motor of 2 horsepower is installed at the facility with limited operational time i.e. 2 hours a day. The water collected is stored in the overhead storage tank and only used for drinking purposes.
• NCA can also look to study ground water re-charge capacity if there is a strong evidence that ground water re-chargeability will be affected by our interventions.
• In year 2016, NCA has implemented a project on solar desalination stills in the same area of district Sanghar. However, the results were not encouraging. Furthermore, many organizations had tried this technique particularly in desert area of district Sanghar but it didn’t yielded good results.
• NCA recommends rehabilitating perennial sources of water i.e. dug wells and in months where there is short fall of water Rainwater Harvesting Tanks can be the alternative.
• Smart design of sanitation facilities which can accommodate bathing space to be used by women and young girls.