https://bristoluniversitypressdigital.com/view/journals/pp/48/1/article-p89.xml

The article explores the potential benefits to public policy of combining traditional evaluative inquiry with insights developed dynamically in policy labs.

What possible synergies between evaluation practice and policy labs could help in addressing the three challenges of public policy: establishing what interventions work, explaining their change mechanisms, and using research findings?

This article assess the extent to which the purpose, structures and processes used in policy labs address three challenges: 1) establishing the causality and value of public interventions, (2) explaining mechanisms of change, and (3) utilising research findings in public policy.

Summary findings:

• This focus on synthesis and shared characteristics has limited the discussion of heterogeneity among labs
• Second, it is unclear how effective the analysed labs are in feeding their solutions into the actual policymaking and policy implementation process.
• we do not know how sustainable the labs are in the long run and what make them durable in organisational terms.
• Despite these limitations, through our analysis of the practices of policy labs around the world, we have identified a number of synergies between policy labs and evaluation that could be beneficial for both, and that could advance general public policy practice
• To conclude, policy labs are not an alternative development to established practices of public policy. They are rather a promising addition that, when combined with existing evaluation practices, can allow both entities to mutually benefit one another and create synergies to improve the craft of public policy.

—

Over the years, three broad challenges have emerged from global evaluation practice in relation to its contribution to public policies:

• Establishing what solutions work highlights the need of verifying the causal relation between public interventions and observed changes, and making a fair judgement about their value. This provides public policy with accountability for executed programmes, and indicates which interventions are worth financing in the future.
• Explaining why solutions work (or not) covers approaches and methods that provide insights into the complex nature of mechanisms responsible for socio-economic change. An understanding of these mechanisms enables the designing of more effective interventions.
• Transferring research findings into policy actions focuses on bridging the gap between the extensive production of policy studies and their limited utilisation in decisionmaking processes. Resolving this issue could substantially strengthen evidenceinformed policies.

We find that policy labs are not an alternative development to established practices of public policy. They are rather a promising addition. The structures, functions and processes of policy labs can help to address evaluation challenges mainly related to explaining why solutions work (focusing on users, unpacking behavioural change mechanisms, building and testing solutions in iterative REACT process) and translating research findings into decision making (involving policy stakeholders and providing platforms for safe learning through experimentation).

At the same time, policy labs may benefit from evaluation theory and practice in gaining a deeper understanding of challenges that relate to establishing what works, as well as valuing public interventions, incorporating systems thinking and complexity, and scaling-up solutions. We believe that these synergies, when turned into an exchange of practices, could eventually enhance the effectiveness of public policies.

Public policy is about triggering a behavioural change mechanism (Shafir, 2013; Weaver, 2015; World Bank, 2015). This recent perspective is very much in line with the public policy ‘classics’ stating that public policy is about changing the choices and behaviours of policy actors (Lasswell, 1951; Simon, 1997; Wildawsky, 2017). Evaluation literature follows this logic, arguing that public interventions (projects, programmes, policies and regulations) should be viewed as levers designed to activate certain change mechanisms among policy addressees, which in turn should lead to the desired effects (Astbury and Leeuw, 2010; Chen, 2004; Rossi et al, 1999).