Annual Report 2007 Evaluation of Norwegian Development Cooperation

"Continuous Tough Evaluation" from a "Strong Independent
Evaluation Department"

This is what Erik Solheim, the Minister of the Environment
and International Development, expects of Norad's evaluation
work (The Trondheim daily Adresseavisa, 31 January
2008). By virtue of a clear mandate, the evaluation department
has wide authority to implement independent and
critical evaluations of how the Norwegian development
budget is spent. This remains the case whether the responsibility
for implementation is vested in the Ministry of Foreign
Affairs, in Norad, in the multilateral system, or in Norwegian
organisations.

We work systematically to cover all important parts of the
development budget. In 2007 we completed sector evaluation
of long-term aid in the petroleum area and for hydropower
development, and in 2008 we will evaluate the work
in the fisheries sector and assistance in preserving cultural
heritage. In 2007 we performed the first country evaluation
for a long time of the total Norwegian aid to one country,
in the form of the Zambia study. In 2008 we will get to
grips with aid to the Western Balkans, that is, the countries
of the former Yugoslavia.

We are systematising the evaluation of the work of Norwegian
non-government organisations (NGOs). In 2007 we
have looked at Guatemala and in 2008 we shall deal with
Uganda. The emphasis will be on results.
In the past year we have also performed two evaluations of
Norwegian humanitarian aid, one of the use of former military
trucks in emergency relief and the other of NOREPS -
the Norwegian Emergency Preparedness System. In 2008
and 2009 we will continue with evaluation of peacebuilding,
using as our point of departure the Guidance for evaluation
of peace-building and conflict prevention that the Evaluation
Department has helped to develop for OECD/DAC. Evaluation
of assistance in promoting human rights is next in line.
Are we then capable of living up to the Minister's expectations?
Everyone who has worked in evaluation knows how
difficult it is to give clear answers. We are facing challenges
on practically every front. Some of this is purely a matter of
methodology; as a rule, our data are unreliable. When aid is
harmonised with that from other donors it becomes difficult,
and often impossible, to evaluate the impact of Norwegian
funds and Norwegian input. Even as regards independence
and roles, however, we may encounter problems: the people
who know most about aid in a particular area often suffer
from a conflict of interest and cannot take part in the evaluations.
This means that the consultants must spend a lot of
time getting acquainted with what they are to evaluate. The
fact that foreign firms often get the assignments helps to
strengthen the independent evaluation, but it also creates
distance. Another factor is that the consultant industry is
often accused of toning down its criticism for fear of losing
future commissions. We do not believe this is the case, but
the perception may be real enough. The expertise of those
taking the assignments may also vary widely.

Our response to this is to be clear about roles and demand
a high quality. We are very strict on conflicts of interest and
regard it as our job both to encourage critical independence
and to protect the consultant against what can easily be perceived
as pressure from the objects of the evaluation. We are
laying increasing emphasis on having the results of the aid
documented. We think that we have largely succeeded in this,
as in the Zambia study, the Guatemala evaluation and in the
evaluation of the hydropower aid. In the future, however, we
shall be looking more closely at the use of money and at
cost-effectiveness than we have so far managed to do.

The Zambia evaluation is one example of a "tough" evaluation,
in which fundamental questions were asked, and in
which good results were obtained as regards institutionbuilding,
at the same time as we found reason to enquire
whether the aid was sufficiently poverty-oriented. There
may be more such evaluation projects in the future, where
we also make use of other approaches to evaluation than
those we have traditionally employed.