Safety Cases We Can Check Together

TL;DR. By Friday, any lab with a self-reviewed safety case can make the first move and put the relevant hashes out there. It is a six-rung ladder from testimony to replay that begins with a single hash. A safety case only commands belief when those most skeptical of the lab’s word are in a position to verify it. Self-audit is fine, but independent auditors with confidential access do better. And if we add a public replay of an open verifier or a retired checkpoint, we bring contestability and the standing to put new questions on the table. Epistemic status: I have been working on this mechanism-design argument since 2023 when we put Taiwan’s AIEC in place, and my review of SpaceXAI’s “For You” transparency release this August has given it further shape. Like the rest of the safety community, I am wary of frontier-level weight releases. The cruxes are at the end of this piece and I would welcome being proven wrong on them. What follows is a structure where every rung is secure, private, open and capture-resistant. Any release is put behind a capability gate. Policy will follow deployment decisions and responsible-scaling gates, which already rely on safety cases. That sort of thing warrants an inspection anyone can replicate. The first step is inexpensive: one hash. 1. From testimony to evidence There is substance to what is already in play. Take a frontier lab putting out a system card or risk report and incorporating its internal model-based review into the safety case. Anthropic’s Petri framework is the best public illustration of how to construct independence. Since Sonnet 4.5, Petri has figured in the alignment assessment for every Claude model. In May 2026 development was handed over to Meridian Labs, an independent nonprofit, in what Anthropic described as a way to keep the results neutral across the industry by making Petri “independent of any AI lab.” The UK AI Security Institute has put Petri to use in pre-deployment evaluations with a 3.0 prototype. Its “Dish” extension pits scenarios against the model’s actual production scaffold and prompt, meeting the pinned-configuration of §2 on its own terms. When two parties arrive at the same requirement from opposite ends, it is usually a real one. The harness is open and in the hands of external auditors. The hard part of establishing independence is over. What is lacking is the cryptographic layer. There is no attestation of the precise configuration, no public replay, and no hash commitments for the checkpoints in the system-card runs. In the absence of that hash, even a good-faith self-audit is unverifiable by design. There is nothing to measure up to: An outsider cannot be sure which checkpoint did the reviewing. One cannot tell if the published transcript is a complete record or has been curated. As for the run’s independence, it is impossible to prove or disprove. Testimony is evidence, but its force is limited by the testifier’s incentives. Reputation and liability provide some of that. Yet the ones who have to trust a safety case are the ones who cannot simply accept the lab’s version of events. Careful work should be subject to a check, not faith. We have come a long way from the scientific revolution’s Nullius in verba – take nobody’s word for it – and the demand for reproducibility. Institutional self-audit is still on the testimony side of that divide. Closing the gap takes one hash. 2. Identity is more than a hash A lab may be perfectly honest but still unable to show outsiders that the model in the report is the one that ran the audit, absent some machinery. A remote-attestation chain is not pure self-vouching, but it does require trust in the vendor’s hardware root. Public weights solve the identity problem in a way. But for an audit, identity is a larger proposition than a hash. The checkpoint is the core, but under deterministic inference you also need the full tuple: the seeds, quantization, kernels and the inference config (sampling parameters, system prompt). Put the same weights to a nondeterministic backend or a different prompt and you get different answers. With batch-invariant kernels in the release, cross-hardware bit-exact replay becomes arithmetic that can be re-run by all. The aim is to reduce the need for trust. 3. Many routes, one destination The point stands: a safety case does not have to live or die by the auditor’s secrecy. Whether it is the METR / AISI approach with confidential access or a protocol on an open model, the thesis holds. For the fidelity of the replay, the stronger course is a public release of a retired, capability-dominated checkpoint from the lab’s own line. An outsider’s check is meant to be of a piece with the lab’s own evaluation style, training lineage and safety framework. It is not some substitute the lab might dismiss as non-comparable. There are those in a lab who find no merit in a review that is lineage-matched; they can put open third-party models through committed-protocol audits if they wi