Improving agricultural innovations and technologies in developing countries is of paramount importance to increase agricultural production and income sustainability. Although many agricultural technologies are available, adoption remains low among smallholder farmers. To identify and map the existing evidence on the effects of agricultural innovation interventions on outcomes related to productivity and the sustainable growth of smallholder farming, 3ie produced an in-house evidence gap map (EGM) which systematically outlined existing evidence from impact evaluations and systematic reviews. The EGM identified 308 completed impact evaluations, 6 systematic reviews, and 2 systematic review protocols charted on a matrix framework of 16 interventions and 15 outcomes, spanning across 58 L&MICs. Thus there is ample fodder to feed evidence-hungry decision-makers.
The map serves a second purpose of identifying existing gaps in the evidence, both in terms of interventions and in terms of outcomes of interest. In addition, the EGM report (discusses gaps in methodologies, geographies and population sub-groups.
Where is the evidence?
The most-studied intervention category is agricultural inputs (such as fertilizer, chemical and seeds) and practices (such as planting techniques) with 134 studies, while the least-studied intervention category is knowledge dissemination with 67 studies.
The mapping process also revealed that a high number of studies were using quasi-experimental methodologies; in particular propensity score matching (PSM) for interventions relating to inputs and practices. In studies where sub-group and population analyses were undertaken, the farmers’ sex was the most commonly measured indicator across all intervention types.
Geographically, the map shows that most studies are taking place in Sub-Saharan Africa and South Asia (225 studies). Most of these focus on inputs and practices-related interventions, while those in Latin America focus on institutional arrangement interventions (such as land titling, contract farming, and farmer certification).
Included impact evaluations by geographical location
What are the main gaps?
- Despite the fact that a large number of interventions were planned around activities that involve some form of education or training, few studies measured outcomes related to knowledge transfer. The largest part of the studies concentrates around interventions referring to the provision of inputs and practices, and outcomes related to productivity.
- There is very limited evidence on what the most cost-effective policies are. Only 7 studies conducted cost-effectiveness analyses.
- The use of randomised designs is scarce. The majority of the studies use quasi-experimental methods; particularly propensity score matching (162 studies), with only 66 studies using experimental methods.
How did we build the EGM?
EGMs are a tool developed to support evidence-informed policymaking in particular sectors or thematic areas.
The matrix framework of interventions and outcomes was developed drawing from relevant academic literature and consultations with key stakeholders; many of the categories in the EGM were informed by a consultation workshop including the International Fund for Agricultural Development and Food and Agricultural Organisation officials in 2016. 34,060 studies were preliminarily identified as pertaining broadly to the category of agriculture for development; after preliminary screenings 394 studies were identified as being suitable for full-text screening and coding, leading to the final count of 316 studies including both impact evaluations and systematic reviews.
Which are the areas where research should be heading?
The findings from this EGM highlight the importance for funding and research to go towards areas where there is both a dearth of evidence and a potential for inciting improvements.
Given the low adoption rates, it is quite surprising that there is so little evidence on what works in terms of knowledge dissemination, and this should be a high priority area for further research.
Although rural poverty is highly predominant in Sub-Saharan Africa, there is still a gap in knowledge on what interventions work to encourage agricultural productivity in those settings where agribusinesses and food industry are drivers of economic development.
More studies should explore heterogeneous treatment effects. Among the studies included in the EGM, only one third conducted any kind of sub-group analysis.
Areas where the EGM shows ample evidence are also areas that can warrant syntheses. Given the large number of impact evaluation studies related to interventions linked to the provision of inputs and practices (particularly seeds), and to schemes put in place to try to overcome the shortcomings of the rural financial markets, such as transfers and credit, systematic reviews in these areas are highly recommended.
Finally, given the relative lack of experimental studies uncovered in this EGM, we recommend that future impact evaluations in agriculture be planned hand in hand with programme implementation and design to allow for the use of more experimental methods.
