How should evaluators respond to a changing global landscape? Ideas from the World Bank IEG's expert panel

Paul Thissen, Marie Gaarder 15 April 2024

To respond to a rapidly-changing global context, the evaluation community needs to adapt its methods and approaches, according to an expert panel assembled by the World Bank’s Independent Evaluation Group (IEG).

The panel was part of a series of events marking 50 years of independent evaluation at the World Bank.

The world today looks starkly different than it did two decades ago, particularly in terms of challenges related to conflict, climate change, and humanitarian catastrophes, said Raj Kumar, Founding President and Editor-in-Chief of Devex.

“There was a consensus… that global development was important, that it should be funded,” Kumar said. “That has fundamentally shifted now. We are in a different time. …We are more likely in a period now where the world is going to be constrained for the very kinds of resources we need to do [development] work.”

To respond to this, Kumar called for evaluators to highlight the importance of development work.

“We need evaluators to resurrect the idea that development works,” Kumar said. “We need to point to examples where we’ve seen great positive results.”

To accomplish that goal, it’s essential to look at transformational change, said Juha Uitto, former Director of the Global Environment Facility’s independent evaluation office.

“For instance, we had a GEF-funded project of less than one million dollars … that was critically catalytic to create a wind energy market in Uruguay,” Uitto said.

One result of the project was that it created a regulatory framework and transparent market for wind power in the country, which was then taken up by the private sector, Uitto said.

“The beauty of this study that we did was [that] the factors we identified could readily be used to develop and screen future projects ex ante,” Uitto said. “We need to enhance our approaches to look at these bigger issues … We shouldn’t just focus on projects and check whether projects succeed in doing what they set out to do.”

Evaluators should inform stakeholders on what makes a process transformational, Uitto said, using a bigger lens to look at broader issues of transformational change.

Panelists also agreed that the development community needs to ensure its institutions are set up to learn from evaluation evidence.

“At 3ie, looking across organizations we have found that certain factors or levers are consistently important for building an evidence-based culture,” said Marie Gaarder, 3ie’s Executive Director. “We call this ‘TRIPS’: training, resources, incentives, processes, and signals.”

Those are the types of institutional changes that Andrea Cook said she was looking at in her position as Executive Director of the United Nations Sustainable Development Group System-Wide Evaluation Office to drive change across the United Nations development system underpinned by evidence to support the UN family to work better together in pursuit of outcomes that individual organizations cannot achieve alone to accelerate and leverage progress to achieve the ambition of Agenda 2030.

“It’s never been more important to build a stronger global and local evidence culture,” Cook said. “We want to make sure this evaluation evidence is available and considered across the UN system.” Moving beyond individual evaluations, and as the body of evaluation evidence grows, Cook also emphasized the need for evaluators to find ways to link with broader efforts to strengthen the evidence architecture and achieve a lasting step change in the world’s ability to use evidence to achieve the Sustainable Development Goals.

Evaluators must also make sure their work is relevant to those who make policy choices, panelists said.

“Evaluation methods and approaches need to be better tailored to meet the needs of policymakers,” Cook said. “It’s really critical to promote evaluation as a tool for learning and improvement … it requires a shift in mindset to emphasize the role of evaluators as facilitators and advocates.”

This requires a change from seeing evaluation as being only about providing evidence. Instead, evaluation needs to be seen as a process of actively supporting collaborative learning and the use of evidence by a range of actors throughout planning and implementation, explained Patricia Rogers, Founder of Better Evaluation and former professor of public sector evaluation, Royal Melbourne Institute of Technology. She called for evaluations to use theories of change that better address the complicated and complex nature of development, bringing together diverse evidence, and addressing equity and environmental sustainability in all evaluations.

“The good news is that there are a number of initiatives working on ways of addressing these issues… Together we can learn how to make these changes to evaluation practice,” Rogers said.

Watch the session recording here.

Key resources mentioned during the discussion: