Impact assessment has been defined by a well recognized manual as an activity “intended to determine more broadly whether the program had the desired effects on individuals, households, and institutions and whether those effects are attributable to the program intervention.” 1 Attribution in the context of impact assessment relies upon the identification of one or more “counterfactual” scenarios, which attempt to project the evolution of metrics of interest in the absence of the assessed activity. In order to appraise whether net benefits are sufficient, it is generally necessary to quantify their magnitude and distribution. Such analysis may be historical (ex post) or prospective (ex ante).
In the context of CIFOR, impact assessments are studies that attempt to attribute the consequences of the use or influence of research products resulting from CIFOR and partner research, in terms of ultimate intended goals. Ultimate intended goals for CIFOR are dictated by the mission of the Center and the mission of the broader CGIAR umbrella. The Center’s mission is “is to contribute to the sustained well-being of people in developing countries, particularly in the tropics.” The CGIAR has a mission “to achieve sustainable food security and reduce poverty in developing countries.” As a result, metrics of ultimate impact should be directly related to the present and future well-being of poor populations in developing countries.
As CIFOR principally conducts policy-oriented research2 , the primary impact pathway for most of the Center’s research involves policy influence, so as to improve the welfare effects of development interventions and regulatory frameworks implemented by other agencies. Thus, the impact pathway for the Centre’s activities can be generalized as in Figure 13. The Centre conducts research activities (principally in partnership), which lead to research outputs, including publications, manuals, models, presentations and professionals with enhanced capacity. In turn, research outputs are intended to reach decision-makers and policy processes through indirect and direct means of dissemination. When novel ideas and improved understanding embedded in the disseminated research outputs lead to an alteration in the actions of decision-makers, the outputs create influence, which is an outcome. Shifts in policy regimes that result may, if implemented, cause alterations in programmatic effectiveness and market efficiency. As a result, resources may be managed in a more productive, sustainable, or equitable manner. When such shifts lead (both indirectly and directly) to changes in the welfare of target populations, impact results.
Click here to enlarge
Figure 1. Generalized impact pathway for CIFOR policy oriented research.
In the strictest sense of the term, impact assessment only refers to evaluation that encompasses this last sequence of the figure, by ascribing changes in welfare to a policy shift, which is attributed to the influence of a research output to which CIFOR contributed. However, measurement of human welfare is notoriously difficult, and reliable attribution of the full causal chain from outputs to influence and well-being at wide scale is frequently not possible. Thus, it is well accepted among stakeholders and the impact assessment community that there are a number of reasonably unambiguous proxies to the welfare of the poor that can be employed for impact assessment. Gross economic surplus, in some cases aided by consideration of distributional effects, is a frequently employed proxy, and in fact occupies the bulk of the research impact assessment literature. Thus, attribution of direct proxy metrics for welfare, such as economic surplus, indices of health and nutrition, rural income levels, and indices of the conservation of important environmental resources can all be reasonably termed impact assessment.
For policy-oriented research, some of these metrics are often particularly difficult to attribute convincingly, due to difficulties in tracing the origins of policy improvements, as well as the fact that the pathway from influence to benefits can be particularly protracted. As a result, most studies of policy oriented research produced to date under the mantra of “impact assessment,” in fact, only reach as far as “dissemination” or “influence.” To the extent possible, at CIFOR, we will attempt to go further down the impact pathway.
-----------