Country Strategy Evaluation 2010-15 and Organization Assessment

Background

Lutheran World Service India Trust (LWSIT) is a not-for-profit non-governmental organization in India. LWSIT has been working in the areas of disaster relief and integrated development for more than four decades. Its National Office is located in Kolkata and it currently works in the states of Assam, Odisha, West Bengal and Jharkhand.

Three projects have been evaluated within the overall framework of the Country Strategy Plan (CSP) as per the ToR, each of which represents three distinct contexts and theories of change. We have summarized the evaluation on the organisation and the third project “Development Support Program for the Stone Quarry Workers” (DSSQC) implemented in Birbhum district of West Bengal and Dumka of Jharkhand. The project addresses the problems faced by the communities living in the vicinity of the illegal mines and stone quarries in the area and is being funded by Normisjon. The phase evaluated of the project was from 2013 to 2017.

Purpose/objective

The document presents the findings of the evaluation of LWSIT's Country Strategy Plan 2010-15 from the perspective of:

• Assessing the relevance of the Country Strategy to the priority focus groups and in the context of changes in the national and international contexts.
• Determining the extent to which Strategic Objectives of the LWSIT Country Strategy 2010-2015 were met.
• Reviewing the Normisjon/ NORAD supported Development support programme for the Stone Quarry Affected Communities of Birbhum district- West Bengal and Dumka district - Jharkhand) as well as two other projects supported by other donors.
• Assessing the organizational strengths, limitations and challenges of LWSIT.

Methodology

The evaluation adopted a consultative approach that included the participation of various stakeholders; the methodology followed was based on a qualitative analysis of the available literature and primary data collected during the evaluation. A detailed evaluation framework was prepared along with a tentative sample and field plan which was consolidated in the inception report that was subsequently shared with LWSIT.

The evaluation team visited seven project locations (including the National Office), with evaluation teams of at least 2 members visiting each of the project locations. The primary research covered a total of 120 community based organizations spread across project locations and interactions with close to 1500 community members. The final report was made after the interim report was discussed among the partners, board of Trustees and unit level team members.

Key findings

1. It was found that strategic priorities, objectives and approaches mentioned in the CSP are very much relevant to the project areas across projects. However, deficiency in supply side engagement, especially in terms of the engagement of duty bearers and advocacy with government line departments was found to be a major weakness in the implementation of the strategic objectives and the related modes of operation.
2. Organizational capacity was perceived to be a highly important requirement particularly in the context of the DSSQC project location, as expected results would be achieved only when the field team is able to understand and articulate the theory of change and the subsequent action points. The DSSQC still remains at a hand-holding stage.
3. The Project team across locations lacked clarity on the different approaches articulated in the CSP. Adoption of the Rights Based Approach (RBA) has been a strategic decision; however, it is still at an initial stage of implementation.
4. As a result of activities under the Stone Quarry project, SHG members were able to avail free medical check-ups and extra coaching for children which has been greatly appreciated among the rights holders. Moreover, the awareness of possibilities of access to credit and various government services, as well as awareness about the mandate of the duty bearers has increased though the project still has a long way to go before creating an environment where the right holders are able to access their rights and entitlements.
5. With regards to the impact of lobbying and advocacy work on the political level, it has been negligible as work on these areas itself has not taken place across the three projects.
6. The evaluation team wishes to highlight that the overall level of engagement of the partner communities in the planning process was visible, exerting a positive impact on the overall effectiveness of the projects.
7. There is still a heavy dependence on the ED/Senior Management Committee to provide guidance and set the organizational agenda. Consequently, voices of change and reaction from the field units may be lost. However, a key strength is LWSIT‘s dedicated, motivated staff.

Recommendations

1. While LWSITs ideology and strategic framework have been captured and articulated in the CSP 2010-15, several concepts in the Country Strategy have not been defined or explained thus possibly impacting uniformity in interpretation and consequently, implementation. Additionally, the situational analysis in the CSR is pertinent, broad-based and relevant; it is suggested that going forward, whilst developing its next country strategy, LWSIT leverage its experience, expertise and understanding of community needs to develop a more contextual situational analysis that can be, in turn, also reflected in Project documents.
2. The RBA is a new concept to LWSIT and is therefore still evolving. While developing its new country strategy LWSIT should effectively engage its different stakeholders (internal and external) to assess and review its progress and position on continuing with the rights based approach. There is a need to improve the field staff‘s understanding of RBA in its totality and for the project to begin generating evidences of change at the field level.
3. It is strongly recommended that LWSIT develops a five year operational plan which is aligned with its next strategic plan (2017-22), along with clear milestones and measurable indicators that can be used to monitor the strategy.
4. It is recommended that standardized training modules be developed and further translated into local languages for standardization, efficiency and effectiveness.
5. A viable theory of change for its projects with an intervention logic that is clearly verifiable would assist with programme attribution.
6. LWSIT currently uses the same approach and methods for CBOs of different categories and length of association. This, with the lack of a clearly defined exit plan, may affect effectiveness and sustainability. A robust M&E system needs to be developed to ensure project progress and to be able to adequately assess the same.
7. It is recommended that LWSIT prepare a logical framework for each project unit with clear, achievable and measurable indicators; while defining indicators on project results, its attribution to the project activities should be clearly established.

Comments from the organisation

The key findings and recommendations have been the subject of conversations with LWSIT during Normisjons project visits.