Evaluation of Climate Adaptation Projects
\label{evaluation-of-climate-change-adaptation-projects}The concept of climate change adaptation cannot be easily and
unequivocally defined. Measures as diverse as land use planning,
infrastructure design, community development and farming practices might
all include considerations related to climate change, but it remains
difficult to isolate and evaluate the individual adaptation elements.
Furthermore, activities which have a clear adaption focus require
comparison against a counter-factual baseline which is often difficult
to establish. It should also be considered that adaptation strategies
considered as successful within a short time frame might actually
exacerbate longer-term vulnerability. By way of illustration, poorly
designed coastal defenses may in the short-term lower the level of
vulnerability, thereby encouraging among others population growth and
development. In the long-term however vulnerability may increase if
extreme weather events exceed the design threshold of the defenses.
These complexities need to be considered when designing and implementing
adaptation activities, and also when evaluating them, as evaluation is a
key stage of any process of public intervention (OECD 2011).
Evaluation activities can be categorised by their timing: a) ex-ante
evaluations take place prior to the implementation of an initiative and
are forward-looking assessments of the likely future effects (this may
include vulnerability or climate change impact assessments), b) mid-term
evaluations are conducted during the implementation phase of projects
and intend to improve performance, and c) ex-post evaluations normally
serve the purpose of a summative evaluation since they are undertaken at
the end of the implementation phase of projects, and are directed at
investigating how well the initiative served its aims, at assessing the
sustainability of results and impacts and at drawing conclusions that
may be relevant to similar initiatives in the future (UNDP 2007, GEF
2008, UNFCCC 2010).
There are mainly three ways to carry out ex-ante evaluations of climate
change adaptation. One is to apply a Cost-Effectiveness Analysis (CEA)
where the societal costs and benefits of adaptation projects are
determined, and thus to arrive at a prioritized ranking of the most
cost-effective adaptation measures (ECAWG 2009). This results in a ‘cost
curve’ showing measures that are cost-negative (i.e. create savings),
measures with a cost-benefit ratio below 1 (i.e. benefits outweigh
costs), and measures with a cost-benefit ratio above 1 (i.e. cost
inefficient). Cost-Benefit Analysis (CBA) is often used to assess
adaptation options when efficiency is the only decision making criteria.
A CBA involves calculating and comparing all of the costs and benefits,
which are expressed in monetary terms, thus providing a basis for
prioritizing possible adaptation measures. Multi-Criteria Analysis, the
third evaluation approach, allows for different adaptation options to be
assessed against a number of criteria, where each criteria is given a
weight and an overall score for each adaptation option is obtained. A
recent review of approaches for assessing costs and benefits of
adaptation options (UNFCCC 2010) shows that most approaches focus on
either adaptation costs or vulnerability/risk management. Only two
studies actually compare costs and impacts (Moench et al., 2009, ECAWG
2009). Another study incorporates in the analysis costs and multiple
aspects and benefits for the prioritization of adaptation measures for
disaster risk and vulnerability reduction (Haque et al., 2012).
In terms of ex-post evaluation, such economic-based analyses (i.e. CBA,
CEA and MCA) are often contested and characterised by thorny
methodological issues, particularly in relation to uncertainty,
valuation and equity (Grafakos and Olivotto 2012). Uncertainty
surrounding future climate change dynamics and impacts and future
socio-economic development constrains the identification of optimal
adaptation options. The valuation of costs and benefits, both from a
financial and economic point of view, inevitably includes elements of
subjectivity. Last but not least, it is important to consider not only
the level of net benefits of adaption project but also their societal
distribution, for example by giving weights to different costs and
benefits according to who receives the benefits and who bears the costs
(UNFCCC 2010).
The focus of the chapter is on the methodological challenges of ex-post
evaluations of climate change adaptation interventions. In relation to
this, it should be pointed out that adaption initiatives may be
implemented at different levels. This realization leads to the necessity
to measure the results of adaptation interventions at different scales
and therefore to develop methodologies able to assess national policy
level, community level and project level related adaption measures. The
framework of choice depends on the type of adaptation effort undertaken
(e.g. reduction of community vulnerability, implementation of adaptation
investment portfolio of actions, mainstreaming adaptation in national
processes) and by whom (e.g. NGO, donor or bilateral agencies, national
governments). As project-related evaluation systems are widely used
among international donor agencies such as DFID, USAID and the World
Bank but also among international climate funds like Global Environment
Facility (GEF), the Adaptation Fund, the Green Climate Fund (GCF) and
the Climate Investment Funds (CIF/PPCR), this chapter focuses on this
particular level of adaptation interventions. In need to justify
spending by assessing results locally and aggregating information to the
portfolio level, the evaluation systems at this level are usually built
around evaluation criteria, expressed through logical framework models
(input, output, process, outcome), result-based frameworks and in some
cases supported by a theory of change.