Impact Assessment of Livelihood Interventions

Many livelihood approaches in the last three decades had their bases in the definition given by Chambers & Conway (1992), the DFID Sustainable Livelihoods Framework (DFID, 1999), and other similar frameworks.

‘A livelihood comprises the capabilities, assets (including both material and social resources) and activities required for a means of living. A livelihood is sustainable when it can cope with and recover from stress and shocks and maintain or enhance its capabilities and assets both now and in the future, while not undermining the natural resource base’. (Chambers & Conway 1992).

Given this framing, livelihood programs comprise a variety of interventions aimed at protecting, enhancing and promoting livelihood strategies of people and improving their livelihood security and well-being in the long-term. These include supporting on-farm and off-farm income generating activities, skill development, enhancing capabilities to access and engage in alternative means of livelihood, capability building and financial support for self-employment activities, market support and so on. Livelihood security of a household can be improved by providing access to productive resources (includes replacing and rebuilding them when required), diversifying income generating activities, creating and protecting household assets and reserves, and a combination of any of these strategies.

In particular, during unforeseen events such as natural disasters or economic shocks, it is important to prevent erosion of productive assets and build households’ ability to cope and maintain their livelihoods. Such efforts are paramount to ensure the livelihood security of households and to help them minimize their vulnerability to external shocks (Chambers 1989).

The key objectives and expected outcomes of livelihood interventions may include improved food security of the household, enhanced productivity, increased days of employment, increased income and creation of assets, enhanced economic self-resilience and reduced vulnerability, poverty reduction, and overall well-being of the household. Apart from some of these outcomes, many interventions also result in an intended or unintended impact on areas such as education and health. Such unintended consequences (either positive or negative), are further amplified by the interdependence of livelihood activities (for example, agriculture and cattle rearing) and diversification of households’ livelihood activities.

Furthermore, there are multiple external socio-economic and political influences on the livelihoods of people, their claims to resources, and community and institutional support systems that moderate the potential impact. Given the vast scope of interventions and the possible range of outcomes, practitioners often have to make conscious choices and trade-offs about the objectives, type of interventions, implementation models, intended beneficiaries, sectors, and so on.

The Theory of Change and Program Theory The theory of change (TOC) is the foundational element of any evaluation. TOC maps the outcomes to the intervention through the processes of the program (Figure 1). The TOC for livelihood interventions broadly explains how specific interventions lead to immediate and long-term outcomes (Figure 2).

In case of livelihood interventions, the theory of change can broadly be illustrated as follows:

For a robust theory of change, it is important to establish a strong causality between interventions and outcomes, and making sure that the outcomes are realized only because of the interventions and nothing else – this indicates internal validity. Similarly, it also needs to take into account any unintended outcome that the program might have resulted in, and accommodate for the same in the program evaluation process.

The role of theorizing the program – both its processes and impacts – is an important step before assessing the impacts. Program theory helps in establishing the causal link between the interventions and the outcomes, which can be either intended or unintended. Let’s take the example of MNREGA, where the intended outcome is to create employment opportunities for men and women in rural areas.

With both the adult members employed, the responsibilities for household chores may fall on children, particularly girls, which may cause their dropping out of school. Hence, one of the unintended outcomes of MNREGA could be increased drop-out rates of girl children. Thus, it is important to theorize the impact of MNREGA in the context of the labor market, where child labor could be an unintended consequence.

Livelihood Impact Assessment Approaches

Conventionally impact assessment for any program is done to assess whether the program is able to meet the desired, predecided outcomes. The findings of such an evaluation is largely used for making a decision about continuation of the program. Thus, such evaluations are objective in nature and often deal with tangible economic measures. However, livelihood impact assessment should deal not only with assessing the outcome, but also improving practices (Van Rijn et.al., 2012). In other words, the impact assessment of livelihood projects should not stop just at evaluating economic gains of the households. Rather, it should also assess the changes that the interventions have brought about in the lives of people.

There have been many approaches to evaluate the impact of livelihood interventions. The conventional approaches often adopt quantitative methods to look at pre and post intervention data on the outcomes, and assess the effectiveness of the program based on the differences between the two. Given the objective nature of this approach, it often misses out on the ‘other factors’ that might be instrumental in bringing about changes.

Another prominently used approach is participatory impact assessment, which is often an alternative to the conventional quantitative method. One of the key aspects of the livelihoods framework is the emphasis on the centrality of the people themselves and their perspectives, needs, priorities and strategies (DFID 1999, Scoones 2009). This framework helps not only in identifying indicators for assessing output of the interventions, but also in identifying desirable outcomes as defined by the people themselves. In a participatory impact assessment approach, the progress and the outcome parameter on which the program will be evaluated, is decided by the stakeholders of the project, i.e., beneficiaries, authorities, facilitators etc. It is a collective effort and hence is susceptible to power dynamism.

Yet another approach of doing livelihood impact assessment is to use the sustainable livelihoods framework to evaluate the effect of programs on the work and lives of people. This tries to address the limitations of the above two approaches, by bringing in both quantitative and qualitative data.

Monitoring, Evaluation and Impact Assessment and Their Purpose

Assessment of a livelihood program can be done at multiple levels. It can take place at the implementation stage. It may be undertaken at the stage when initial level of outcomes are being realized, usually in the short-term. It may also be done after a longterm running of the program, to assess its wider impact.

Illustration

The Deendayal Antyodaya Yojana-National Rural Livelihoods Mission (DAY-NRLM) was launched in 2011 with the objective of reducing rural poverty by creating institutional platforms of rural poor and enabling access to self-employment and wage employment opportunities.
(https://aajeevika.gov.in/en/content/mission)

This translates into three broad categories of interventions – collectivizing rural poor into self-help groups (SHGs) and SHG federations, building their capacity through training, and providing access to financial services (Barooah et al 2019).

A simplified TOC (Figure 4 in this page) describes how these interventions result in intermediate and long-term outcomes. Building grassroots institutions of the poor and providing financial literacy and business training results in multiple intermediate outcomes. These include access to financial services for the underbanked and investments in income generating activities.

Some of the long-term dividends are envisaged to be increased household incomes and assets and improved participation of women in household decision making. Of course, there are many assumptions underlying this theory of change – that the groups are functional, training is relevant, timely and adequate, groups are connected to external sources of finance (Barooah et al 2019), there are investment as well as marketing opportunities, and so on.

Livelihood programs such as the DAY-NRLM impact multiple dimensions directly and indirectly related to livelihood promotion/ protection as well as social and political dimensions in the long-term. Thus evaluation and impact assessment for such programs can be complex and resource intensive.

Some impact assessments may take several years to conduct, extending well-beyond the intervention period. However, a well-defined theory of change, monitoring, evaluation and impact assessment are crucial to ensure that interventions actually lead to outcomes and to inform program and policy decisions by providing evidence for programs that are beneficial.

References

1. Barooah, B, Chinoy, SL, Dubey, P, Sarkar, R, Bagai, A and Rathinam, F. 2019. Improving and sustaining livelihoods through group-based interventions: mapping the evidence, 3ie Evidence Gap Map Report 13. International Initiative for Impact Evaluation (3ie). Available at: https://doi.org/10.23846/EGM013
2. Chambers, R. 1989. Editorial Introduction: Vulnerability, Coping and Policy. IDS Bulletin 20(2): 1-7
3. Chambers, R. and G. Conway. 1992. Sustainable rural livelihoods: practical concepts for the 21st century. IDS discussion paper, 296. Brighton: IDS.
4. DFID, 1999. Sustainable Livelihoods Guidance Sheets. Department for International Development, London, UK.
5. Scoones, I. 2009. Livelihoods perspectives and rural development, The Journal of Peasant Studies, 36:1, 171-196
6. van Rijn, F. C., Burger, C. P. J., & den Belder, E. (2012). Impact assessment in the Sustainable Livelihood Framework. Development in Practice, 22(7), 1019- 1035.