Local Factor Graph Debate

Stephan Wäldchen, Dmitry Vaintrob, and many others have contributed excellent ideas to this post. More detailed statements and proofs can be found here. Codex and Claude generated all proofs, I've summarized the results and ideas here. I've been studying unstructured debate over factor graphs, following up on a previous formulation of this problem. I can prove that the error of unstructured debate decays as a power-law with respect to the length of the debate for certain families of weakly-coupled factor graphs. This kind of result is mathematically natural, thanks to a strong analogy with Ising models in statistical mechanics. Under this analogy, a judge's perception of the coupling between the truth values of two related statements is similar to the coupling between the spins of two adjacent atoms. The lattice Ising model is famous for predicting a phase transition in global behavior as the correlation strength between adjacent atoms changes: for weak coupling, the spins of atoms far away are effectively independent of each other and many small grains appear, while for strong coupling, the entire material becomes globally magnetized. Under this analogy, we should expect two regimes for argumentation. In the local regime, a question can be answered by a "small" set of arguments surrounding it. Programming problems are usually local: the correctness of a function is determined by the adjacent codebase. In the nonlocal regime, questions are entangled such that the best way to answer any one question is to answer all of them at once. Politics and philosophy are likely nonlocal: one's beliefs about corporate governance can be related to one's beliefs about immigration through shared questions of ethics, decision theories, and objective functions. Recent empirical research (El et al., Okawa) finds this statistical mechanics model useful for understanding multi-agent belief formation. I believe that debate research should distinguish between the local and nonlocal regimes. For narrow, local domains like programming, the problem size is bounded but the debate must resolve within a single training rollout: if an agent has written some code, the debate protocol should decide whether to reward this code or punish it. I view most existing debate literature (e.g. Irving et al.) as optimized for the local regime. For broad, intertwined domains like philosophy, the problem size can be overwhelming but the debate protocol is allowed to take far longer to converge: an agent's philosophical worldview may emerge slowly over the training process; we should not relitigate all of philosophy every time an agent needs to take a philosophically complicated action. I view coherence maximization (e.g. Qiu et al.) as operating in the nonlocal regime. By studying the simplest possible debate protocol in the local regime, I hope to demonstrate the kind of results this perspective can offer. Problem Formulation Suppose a judge is attempting to make a decision and two debaters are attempting to persuade the judge in opposite directions. For example, suppose an agent is being trained to perform tasks and a judge is trying to decide whether the model's actions should be rewarded. Trained debaters are assigned to argue for and against the claim that the agent is performing well. We assume that there is a vast, hidden space of claims the debaters could make. For example, the debaters could claim that the user would approve of a certain background color for a website, that the user explicitly asked for a dark background, that the background color is too generic, that generic color palettes lead to societal uniformity, that societal uniformity leads to societal stagnation, that the user prioritizes personal gain over societal stagnation, that the user is selfish, that environments with cooperative rewards shaped the user's evolutionary history, etcetera. We will model the debaters' claims as binary variables in a hidden factor graph. The factors in this factor graph encode the judge's perception of the dependencies between variables. For example, if the judge believes that "the user is selfish" implies "the user prioritizes personal gain over societal stagnation", then the factor graph should contain a factor penalizing global truth assignments in which "the user is selfish" is true but "the user prioritizes personal gain over societal stagnation" is false. The judge is also free to place unary factors on variables the judge knows to be true or false; for example, if a debater points to the user explicitly asking for a dark background color, the judge can attach a factor to the claim "the user explicitly asked for a dark background color" effectively forcing it to be true. During the actual debate, the judge will only see a small subset of the possible arguments the debaters could present. Let's model the judge's beliefs upon seeing this small subset by throwing away variables absent from the transcript and throwing away factors touching var