Forest protection is among the most effective approaches we have to mitigate climate change. At the same time, agricultural land and forests provide food, livelihoods and fuel for billions of people globally, particularly in low and middle-income countries (L&MICs).
At the same time there are concerns that large-scale forest protection programming will have negative knock-on effects on food security and other aspects of human well-being. On the other hand, some suggest these kind of programmes could be a win-win solution for both poverty reduction and environmental goals. So, what high quality evidence do we have to answer these key policy questions?
To address this question, 3ie, with the support of the Children’s Investment Fund Foundation, published an evidence gap map (EGM) that provides an overview of the evidence on the effects of forestry and other land use programmes on environmental and human welfare outcomes.
3ie’s evidence gap map shows existing evidence is very limited
Our EGM covers five broad intervention areas: area protection and management, law and policies related to forests and other land use, incentive programmes such as payments for environmental services (PES), training, education and information, and infrastructure. We looked for studies that measured effects along the causal chain, including intermediate outcomes, such as uptake of new agricultural or forest practices and environmental outcomes related to greenhouse gas emissions, including proxies such as forest coverage, and human welfare outcomes.
The evidence gap map shows that key policy questions remain unanswered. The main policy question motivating our study was whether there are trade-offs or potential synergies between programme effects on environmental and human welfare outcomes. But less than 10 per cent of studies address this question. Most studies measure effects on either environmental or human welfare outcomes, but not both. The existing evidence base does not tell us enough about whether there are win-win solutions, or whether programmes that target global environmental protection may have adverse effects on the welfare of a local population.
The evidence base is inflated by multiple studies of the same type of programme. In total, we included 241 experimental or quasi-experimental impact evaluations assessing the effects of different agriculture, land use and forestry programmes and 11 completed systematic reviews.
This focus is partially explained by the existence of long standing public forest protection programmes and availability of data in these countries. For example, we identified 13 studies evaluating the impact of the Grain for Green PES programme in China. Out of the 47 countries that are part of REDD or REDD+, we only identified evaluations of forestry programmes in approximately half of these countries.
The distribution of studies is also uneven between interventions. Four intervention areas are studied relatively extensively: protected areas, payment for environmental services, decentralised or community-based forest management and agricultural extension and training. For other areas, like infrastructure and monitoring and enforcement of forestry and land-use policies, there are few studies available.
The majority of studies use quasi-experimental methods. Only 18 per cent of the studies used a randomised design. Evaluations most effective at reducing bias use matching methods and panel dataincluding baseline data points.. However, many of the quasi-experiments we found use weaker study designs with a high risk of bias. Combining these methods with process evaluation and qualitative research could better inform future programme design and evaluation.
We identified only a handful of high-quality systematic reviews. The findings suggest that PES and decentralised forest management can reduce deforestation rates. But, the effects appear to be modest. The impact on human welfare outcomes is not clear. These findings raise questions about the cost-effectiveness of these programmes.
But, the certainty we can place in the findings from these reviews is limited by the availability of primary studies. The low number of high-quality studies also limits what systematic reviews can conclude in terms of the contextual factors (such as baseline deforestation or local capacity) and programme design features that may facilitate larger effects.
Evidence production is a smart insurance for reducing the risk of programme failure
After the Paris Agreement signed in 2015, 70 per cent of country plans include forests in their strategies to mitigate climate change. The total amount pledged by global funds dedicated to forestry programmes stands at US$8.1 billion, including US$3.48 billion pledged by the Norway’s International Climate and Forest Initiative alone.
Reducing emissions and ensuring food security requires significant investment. Without better evidence there is a high risk we waste funding on programmes that fail.
Make sure new evidence is policy relevant
Future studies should also be designed to address key policy questions. This would include assessing whether there are trade-offs between environmental and human welfare outcomes. We also urgently need studies that include data on costs, which in turn would allow for cost-effectiveness analysis to be carried out.
Existing high-quality systematic reviews of programmes such as PES and decentralised forest management, find relatively small effects on deforestation rates. The implication of this finding is that we need to understand how programmes can be better targeted at areas where effects may be larger.
We suggest researchers, implementers and funders consider the following steps to improve the evidence base and its value to decision makers:
- Incorporate prospective impact evaluations in new programmes, including those implemented under the auspices of REDD+, to address key policy questions
- Explore opportunities for replicating studies of the same programmes in different settings
- Use easily available existing data, such as high-resolution forest cover mapping tools, for example, Google Earth
- Design quasi-experimental studies informed by best practice approaches that have been shown to be most effective at reducing bias, such as matching methods and panel data including baseline data points
- Assess the effects of programmes on both human and environmental outcomes, including in food security and GHG emissions
- Investigate design and implementation issues, such as targeting and the moderating effects of different baseline conditions through mixed-method and theory-based impact evaluations
- Conduct or update systematic reviews as new evidence becomes available
Further reading
This work echoes pleas from experts in the environmental sector to incorporate impact evaluations into the many programmes funded each year. The following papers provide methodological resources and overviews of the discussion:
- Ferraro and Miranda (2017): Panel Data Designs and Estimators as Substitutes for Randomized Controlled Trials in the Evaluation of Public Programs
- Jones and Lewis (2015): Estimating the Counterfactual Impact of Conservation Programs on Land Cover Outcomes: The Role of Matching and Panel Regression Techniques
- Pattanayak (2009): Rough guide to impact evaluation of environmental and development programs
- Ferraro (2009): Counterfactual thinking and impact evaluation in environmental policy.
View the evidence gap map on land use and forestry programmes.
Download the evidence gap map report on land use and forestry programmes.
