Collective Epistemics: Napkin Math on Independent Errors

Part of a larger series I want to put together on some of the basic equations and models of collective epistemics from the more mathy side of the social sciences.

LLM Status: Pictures + Picture descriptions are LLM-assisted.

Today we're going to do some napkin math on some fun little equations around collective epistemics.

You can see this as an exercise in trying to become more collectively rational. If we want to improve the epistemics of a community, is that the same thing as improving our own? How much should you explore versus exploit, given the role you have and the community you're in?

These are all good questions that I probably won't answer today. Instead I'll give you an introduction to some of the math behind the jury theorems, plus a hodge podge of simpler, approximate collective intelligence theorems. We'll then talk a little bit about tape readers, model selection and finally antifragility.

See it as a somewhat coherent buffet of different things within collective epistemics all pointing towards how it is likely better for you to create an inside view of a field compared to just delegating it if you care about the epistemics of the field you are in.

The first move is to look at ourselves as a collective agent.

Condorcet's Jury Theorem says that if you have n voters, each with probability p > 0.5 of being correct, and their errors are independent, then as n grows the probability that the majority is correct goes to 1:

Figure 1. Left: ten voters, each right more often than not, feed into a majority vote. The wrong votes get outnumbered. Right: the probability that the majority is right as the number of voters grows, for four values of p. Even p = 0.51 gets there in the end, it just needs a lot of voters.

The same logic applies to consensus formation on things that aren't votes.

The main point is that when you aggregate independent information, the errors wash out. Each person's idiosyncratic wrongness gets swamped by the collective's tendency toward truth.

The problem here is that word independent for the I.I.D assumption here is doing a lot of work. Correlated errors tend to stack up together.

Figure 2. Left: the same ten voters, but now their errors are correlated (the pink links), so when one is wrong the others tend to be wrong in the same way. The majority inherits the shared mistake. Right: what the correlation ρ between voters' errors does to the curve. At ρ = 0 you get the Condorcet curve from Figure 1. At ρ = 1 adding voters does nothing; the crowd is one voter with extra steps.

If everyone's errors are perfectly correlated, you effectively have n = 1 no matter how many bodies are in the room. A thousand experts who learned from the same masters, read the same texts and leave out the same variables might as well be one expert who is very sure of himself.

Another way to put it is that If everyone delegates to the same person, we end up with many correlated failures, this can be made more concrete in fact which is what we'll turn to next.

Scott Page talks about this in his book The Difference, and I think his framing makes the mechanism clearer.

Figure 3. Five predictions (blue) and the truth (red). Top: the predictions are spread out on both sides of the truth, so their mean lands close to it. Bottom: the same number of predictions, all bunched on one side. The average individual error is about the same in both cases. What changed is the diversity term, and with it the collective error.

If prediction diversity goes to zero, that is if everyone predicts the same thing, then collective error equals average individual error. The wisdom of crowds term has dropped out entirely. You've paid the costs of consulting many people (time, coordination, the appearance of epistemic humility) and received none of the benefits. You might as well have asked one person.

This is true when people don't have I.I.D information which happens to a larger extent when they choose the same type of model to approach the world through. This is true even if you take something like the outside view, for what is the outside view defined with respect to?

When you decide to "take the outside view," you're choosing a basis. What's the reference class? Who are you learning from? What methodology are you using? Model selection is a huge part of predicting the world, and it happens before the prediction starts.

We can think about this through Turing machines: tape readers that process inputs according to rules encoded in their heads. There's deep stuff connecting Bayesian learning and computation here, but the part we need is simple. Different machines read different kinds of symbols. You can take an economic perspective, an ecological model, a population model. Each one processes reality through a different set of primitives.

Figure 4. Reality as a tape. Three heads read the same segment, but each one only registers the symbols it was built to read. The population model sees α, β, γ and outputs "threat: high." The economic model sees $, ×, ÷ and outputs "threat: none." The ecological model sees the cycle symbols and outputs "cyclic." None of them read the cow in the middle.

This is basically what happened with Malthus. He took a pure population model, exponential growth with no sigmoid in sight, and projected it forward. It was correct from one angle, which was the exponential tape reader. But that reader couldn't see the symbols that would have told him about agricultural innovation, demographic transitions, or the fact that people change their behaviour when conditions change.

In theory there's a solution. Kolmogorov complexity defines the complexity of a string x as the length of the shortest program that produces it on a universal machine U:

Universal machines can read everything. Solomonoff induction turns this into a prior where we can weight every computable hypothesis by its complexity, update on evidence and then converge on the truth. The ultimate Bayesian reasoner to some extent.

It's also incomputable. Like AIXI (unless?...), you can't actually run it. Every real agent picks an approximation, something like a restricted hypothesis class, a specific set of programs the tape reader can execute. That choice is your prior, and it comes before Bayes gets involved. It's turtles all the way down, and at some point you have to stop and pick a turtle.

Which turtle you pick decides what counts as "simple."

Figure 5. The same situation compressed in two languages. In the language of population dynamics it takes a long program. In the language of folk farming practice it takes one sentence. Neither number is the "real" complexity. Each is the complexity relative to a choice of primitives.

(The cow situation refers to a quite strange example which I removed that involved a cast of 400 BCE greek people speculating about exponential cow population growth, no further comments.)

In the population dynamics language, the cow situation needs an elaborate program: growth rates, carrying capacity, differential equations, eigenvalue computation. K_A(S) = 847 bits. In the folk practice language, the same situation compresses to a single sentence: "Animals we farm stay at farming levels." K_B(S) = 52 bits (fully calculated numbers, trust me). Which description is "simpler" depends entirely on which primitives your language treats as basic operations.

I make a longer argument for this in The Atoms of Knowledge Aren't Universal. The TL;DR is that it's sometimes more useful to stop at a coarser description. The valence electron model of chemistry is "less precise" than quantum field theory, but it has higher effective information for predicting bonding behaviour.

So the choice of primitives decides what you can see, and the wrong primitives can make you blind to what's right in front of you. How do you make sure you're not all blind in the same way?

If correlated errors are the enemy of collective wisdom, then decorrelation is the remedy. In practic