How Can We Solve Animal Welfare's Evidence Problem? (Post 1)

Published on October 10, 2026 3:10 PM GMT TL;DR:  The movement has an evidence problem. To solve it, for each intervention we need to know: What types of evidence that intervention needs What types of evidence different sources can give us In this post, we outline a framework for evidencing an intervention through five stages: Figure 1: The five stages of the evidence pipeline. Each stage poses a different question, needing different types of evidence, and thus different methods (and potentially different organisations and experts). A follow-up post will cover practical next steps. Many thanks to those who reviewed our drafts. All mistakes are our own. 1. Introduction Four months ago, Matthes argued that animal welfare has an evidence problem. Discussion mostly focused on the specific examples she gave. Her wider claim received less attention: that the movement lacks enough well-evidenced interventions, and that we need to take ownership of the entire evidence pipeline to change that. Discussion so far has centred on interventions that are highly invested in, and whether their evidence is strong or weak (the top left of Figure 2). However, Most interventions are neither invested in nor well-evidenced. We want to add a second discussion point: how do we get more interventions to be well-evidenced (ideally, the top right of Figure 2)? Figure 2: Interventions by movement investment and strength of evidence. We agree with Matthes that there is an evidence problem. To solve it, we need to rethink how evidence generation works within the animal movement. We need to consider: What evidence do we need? What would a movement capable of generating it look like? This post is part one of a two-part response to Matthes' post. Here, we set out to answer the first question: To define what types of evidence we need and what types of evidence different sources can give us. Part two covers the practical next steps. 2. What Do You Mean We Have An Evidence Problem? Responding to Matthes, Krzysztof Wojtas argued that evidence-building is not "a linear process", with research finishing before implementation starts. Field implementation is itself part of the learning, and gives different types of evidence that are equally needed. We agree, and the point exposes a useful distinction: Evidence is not a monolith. Each intervention raises many different questions. Each question calls for its own kind of evidence, and each kind of evidence has its own best source. Implementation is one of those sources, and it answers some questions well and others poorly. Let's imagine an intervention helping fish in Brazil through improved biosecurity. Here are four questions it might need to answer, and whether its own monitoring data can answer them: The questionWhat would answer it?Can monitoring data answer it? Can farmers implement biosecurity practices in their context?Records from the farms in the programmeYes. How common are disease outbreaks across Brazil?A representative survey of farmsNo. The farms in the programme are not a representative sample. Does the intervention reduce disease outbreaks?A comparison of disease rates between fish that received the intervention (a treatment group) and those that didn't (a control group)Potentially. Depending on whether a control group was also being monitored. How much suffering does the intervention avert?A comparison of welfare indicators (e.g., cortisol levels) between a treatment group and a control group.No. Precisely measuring welfare typically requires sophisticated methods executed by experts, ideally within a controlled setting. So, monitoring data is an important part of an intervention's evidence sources, but it cannot be expected to fill all evidence gaps. Nor can implementers be expected to resolve all evidence gaps in an intervention. To solve the evidence gap, we will need diverse types of evidence and a diverse set of skills. The movement's evidence problem, then, is actually composed of many smaller evidence problems, each a kind of question the movement needs to be able to answer about its interventions. To solve it, we need to understand: 1a) What types of evidence an intervention needs 1b) What types of evidence different sources can give us 3. The Evidence Pipeline So what are the questions the movement typically needs to answer about its interventions? There are many ways to break that down. Here is our first attempt at a framework based on systems from tech development, clinical health, and global development:[1] Figure 3: The evidence pipeline. Each stage answers a different question about an intervention. As we invest in answering these questions for an intervention (y-axis), the strength of evidence (x-axis) increases.[2] The stages do not run in a strict order: findings at any stage can reshape the others. These five questions trace an intervention from idea to a well-evidenced scaled programme. The steps build on each other, and there is a rough logica