Proposal for tracking the effects of architecture on monitorability

Architectures that incorporate opaque recurrence or allow for agents to communicate with each other using latents could rapidly make it much harder to monitor chains of thought or communication (we’ll refer to this property as “monitorability” going forward).[1] As companies begin to explore such architectures, we believe it is important to transparently share evidence about how monitorability varies with architecture and training method. To inform the scientific debate on how to make tradeoffs between performance and monitorability, we believe AI companies should: Regularly report externally verified information about the degree to which their architectures may allow for latent reasoning or communication. Companies should publicly disclose enough information about architectures to allow external scientists to determine whether they could potentially enable models to perform much more complex reasoning without this reasoning appearing in the chain of thought (“latent reasoning”) or allow for latent communication between different instances of a model. Following GDM, we propose measuring opaque serial depth as a minimally-invasive proxy for the degree to which an architecture may enable latent reasoning, though companies could provide sufficient architecture transparency in other ways. We propose that companies work with third-party evaluators to produce independently verified reports of the rough distribution of opaque serial depth across all their near-frontier models.[2] External verification wouldn’t require third parties seeing the architecture of these models; interviews with employees along with a process for employees to whistleblow to the third party could be sufficient. By near-frontier models, we mean all general-purpose[3] internal models at least as capable as the best public models from six months ago.[4] Because it is important to understand the relationship between monitorability and architecture — and not just the monitorability of deployed models — the models covered in this survey should include purely internal R&D prototypes if they have near-frontier capabilities. Opaque serial depth can also serve as an incomplete proxy for latent communication[5], but companies should ideally share additional high-level information about the architecture and structure of communication between agents (e.g., whether AIs communicate in latents, whether shared ‘memories’ are stored in latents, whether agents receive parts of the KV cache of other agents). The relationship between well-elicited[6] no-CoT capabilities and overall capabilities could serve as an additional or alternative proxy measurement. More concretely, companies could build a dataset to measure no-CoT capabilities, project how much some change to architecture or training methods increases no-CoT ECI relative to how much that change increases overall ECI,[7] and work with third-parties to produce independently verified reports of the extent to which new methods might result in a disproportionately large increase in no-CoT ECI.[8] This reporting should have reasonable coverage over all near-frontier models. Regularly share evidence about monitorability and have third parties review this. Companies should publicly share their views about the monitorability of their near-frontier models and how this varies across architectures and training methods, along with any evidence or information they have about this.[9] Ideally, companies would run monitorability stress tests to see if models can be elicited or fine-tuned to perform reasoning without this being reflected in their chains of thought, or communicate with other models in ways that are hard to monitor.[10]  An initial sketch of what stress tests of CoT monitorability could look like is in an appendix below. Companies should give third-party experts access to the unredacted results from these experiments, and third-party experts should: a. Assess whether these experiments and other evidence are accurately describe b. Run their own experiments to gather more evidence about CoT monitorability and how this varies across training methods and architectures, and to gather evidence about any other important claims companies made. c. Report their findings to the public, with redactions for IP as appropriate. As above, this should have reasonable coverage over models at least as capable as the best public models from six months ago, including internal R&D prototypes if they are near the frontier. Evidence sharing and third-party reviews should occur regularly (e.g., every 6 months) and potentially whenever a new architecture or training method results in a near-frontier model with significantly increased opaque serial depth.[11] Publish a policy regarding architectures that could enable greater latent reasoning, and provide information to help assess compliance. Companies should publish[12] a policy document that clearly explains: a. Their views on what kinds of architectures could enable much great