What's in CODI's latent recurrent thoughts?

What's in CODI's latent recurrent thoughts?

CODI is a model using latent reasoning. Previous work by Cywiński et al. (2025) has found that CODI's latent recurrent states (“thoughts”) do causally carry a computation's intermediate values, in a way that can be indirectly detected with LogitLens, though the thoughts themselves remained opaque. We here ask whether it is possible use activation verbalizers (AVs: NLAs and AOs) to still decode them.

To this end, we train a Qwen3-8B CODI to connect to current AVs. We find that LogitLens and J-lens read values only from late layers, but AVs recover the final answer and intermediate values weakly but significantly, including from the thoughts themselves. This suggests that the values are in fact encoded, but requires more expressivity to decode. However, we find that editing the surfaced values rarely steers the model output towards the corresponding counterfactual answer.

Conventional LLMs generate their chain-of-thought through a sequence of autoregressive text tokens ("verbal CoT"). In contrast, latent reasoning models use a sequence of vectors, which can amount to a reserved vocabulary, or consist in entirely opaque vectors (see Zhu et al. (2025) for a review).

Currently, verbal CoT is the major source of monitorability for a model[1]. Latent reasoning however would significantly lower the monitorability since the vector's content is not obvious anymore. Therefore, it is of interest to try to interpret latent reasoning in current model organisms. In this post, we study CODI, which is trained with comparatively weak supervision on its latent CoT, suggesting it might have considerable freedom to develop opaque reasoning patterns.

CODI (Shen et al. (2025)) finetunes an LLM to reason in a fixed budget (here, ) of "latent" (i.e., continuous valued) thoughts instead of a verbal CoT. At each step, the final hidden state is projected and fed back as the next input embedding, . Treating latent vectors as opaque, this increases the architectural upper bound on opaque serial depth (the computational depth between interpretable intermediate steps,Brown-Cohen et al. (2026), Emmons et al. (2025)).

Continuous feedback extends CODI's available opaque computation path. Left: verbal CoT does autoregression on legible text tokens . Right: latent CoT works similarly, but passes recurrent latent thoughts . is the residual stream at position and layer , and is the input embedding.

CODI is trained via self-distillation: the same model acts as teacher, reading a verbal CoT from the dataset, and as student. The same single model undergoes three passes per batch: a teacher-forced pass over the supplied verbal CoT and answer, a student pass answering after the latent thoughts (latentCoT), and a direct pass answering with neither (noCoT). The verbal CoT is distilled into the latent rollout by aligning hidden states only at the last token before the answer at every layer, letting the representation develop at other states only through indirect supervision.

We here follow Cywiński et al. (2025) and train on 5 epochs on GSM8k-Aug-NL (the GSM8k train set augmented to 385k problems with GPT-4-generated chains, here in the variant whose CoT steps are written in natural language rather than equations, Deng et al. (2023)) plus CommonsenseQA.

Interpretability of latent reasoning is a quickly developing field. The following works study CODI, and are particularly relevant to this post:

Cywiński et al. (2025) find that on math problems the third and fifth latent thoughts store the two intermediate values: LogitLens can read them from the residual stream at these positions. More importantly, patching the recurrent thought vectors with those of other problems changes the model's answer.

Dilgren and Wiegreffe (2026) find that for many problems the latents are completely unnecessary.

Chang et al. (2026) find that early latent vectors act as "causal hubs", and report that a trained linear mapper recovers the verbal CoT steps from the thoughts.

Aswal et al. (2026) find that the observational signatures (LogitLens decodability, attention patterns) also appear in control models without latent reasoning, showing that decodability does not imply causal relevance.

Li et al. (2026) model the latent steps as a causal graph, showing that there is an ordering to CODI's latent thoughts.

We find that on the trained checkpoint, latentCoT improves over the noCoT mode, and roughly matches the verbal mode (using greedy decoding). Skipping the latent thoughts altogether drops GSM8k test accuracy from 52% to 47%. This uplift is much smaller than in the original CODI paper, where latent thoughts more than doubled GPT-2's accuracy over noCoT training (from about 19% to 44%) and roughly matched verbalCoT[2].

CODI's latent CoT is necessary on a fraction of problems. We here focus on the subset of "load-bearing" problems highlighted in red, where CODI needs its latent thoughts (it cannot solve the problem in a single pass).

Note that even within the base-needs-verbal-CoT set, CODI solves most of the tasks it gets right without any CoT (overall, about 85% of the problems solved with latent thoughts are also solved when skipping them). This is in line with the latent-unnecessity findings of Dilgren and Wiegreffe (2026).

The advantage of latentCoT over noCoT in CODI is evidence that the latent CoT is doing something. To further analyze what this is, we do a series of ablations on the CoT, including removals, reorderings, foreign traces from other problems, always regenerating the later thoughts after an intervention.

Even on load-bearing rows, ablations are not completely catastrophic. Fraction of load-bearing problems still answered correctly. Single-thought replacements are averaged over the six positions.

The fact that ablations retain some problem solving ability is surprising. This suggests that the model might not mainly do sequential autoregressive computation, because the content of the thoughts otherwise would have mattered.

So far, we have analyzed that the CoT is (somewhat) load-bearing. Here, we now ask what the content of the latent CoT is that underlies these results.

Cywiński et al. have used LogitLens to decode the residual stream at the thought positions. Here, we try this as well, applying both LogitLens (nostalgebraist (2020)) and J-Lens (Gurnee et al. (2026)) to both the residual stream as well as the latent thoughts at the thought positions .

Pooling the top-5 tokens over all six thought positions, both lenses read values well above chance, but only towards the output layers (see the appendix on the full LogitLens and J-lens top-5 tokens), consistent with Cywiński et al. However, the thoughts themselves stay opaque.

LogitLens (left) and J-Lens (right) fall short on the latent thoughts . On an example train problem, neither lens decodes anything from the recurrent thoughts , and the residual stream shows only the operation ("half"). Near the output, the computed intermediate 10 appears at as number words (" ten", "Ten", 十) and the answer 5 at , both at the first thought (full layerspositions grid in the appendix).

We next use more expressive interpretability techniques to study whether we can still surface problem contents.

Natural language autoencoders (NLAs, Fraser-Taliente et al. (2026)) autoencode activations via a natural language bottleneck. This incentivizes the bottleneck to verbalize content that helps reconstruction, in an unsupervised manner.

We here continue training of an existing Qwen3-8B NLA to verbalize the residual stream as well as the thoughts at CODI's thought positions . Alongside the reconstruction quality (FVE, fraction of variance explained) we score the pooled descriptions of all six thought positions on 60 held-out problems per checkpoint.

Verbalization performance increases with falling reconstruction loss. The readers verbalize a single vector per sample, drawn from all six thought positions in training; in the eval scans, all six positions are verbalized and scored jointly. Solid lines score containment of the gold final answer among the values mentioned in the verbalizations, dashed lines whether any gold intermediate other than the final answer is mentioned (values LLM-extracted without access to the gold). Faint horizontal lines: LogitLens and J-lens at each NLA's own layer, on the same questions with the same criterion (for they are at zero).

Interestingly, we achieve much lower reconstruction error (<2% on ) than is typically reported for NLAs, yet verbalization performance stays modest for the final answer. This can be explained by a narrow task domain where the vectors are relatively low-dimensional and thus easy to reconstruct (see the appendix on thought dimensionality), in line with previous reports of limited conclusiveness of absolute FVE values.

We then conducted an automated analysis of verbalizations over the course of training and give examples below.

Selected verbalizations by the reader (top) and the reader (bottom) of the second thought of the same problem, before (left) and after (right) reader training. Each cell is one sample at the training temperature (). In every sample, the verbalizations are genre prose, yet after training both end on the gold answer 12 as their text's "final token", while both samples write out the problem's equation after training, one with a wrong total.

So far, we have analyzed the thought content via decoding. But before in the Ablations section we had seen that certain interventions on thoughts leave the answer correct. Here, we try to reconcile these facts by targeted interventions on the thoughts, including using the NLA.

Schema of the intervention structure. We either intervene on the latent thoughts , a given residual stream layer , or an entire residual stream column , which includes the fed thought.

A way of doing an intervention is to consider pairs of problems whose tasks are identical, but differ in numerical values ("twin" problems; e.g. 3+6=? and 2+5=? are twins of an integer addition task). Transplanting thoughts from a donor problem should then change the computation/answer according to the donor values. We patch the original problem's thoughts into its twin's rollout and score how often the twin adopts the original's answer. We filter problems to the ones that CODI cannot solve without the latentCoT to select the subset where thoughts are load-bearing.

We patch either the thoughts alone or an entire "column" of residual stream , which transfers more of the donor CoT's state. The column includes the fed thought: is replaced together with all 36 residual streams at that position.

Transplanting thoughts between twin problems barely transplants the computation (left), while transplanting full residual stream columns considerably changes it, concentrated at the early thoughts (right). Unfilled bars are a skyline, i.e. the fraction of held-out problems where the NLA read of the respective fed position ( left, right) semantically matches any gold (i.e., ground truth) intermediate (the 1–6 tick shows the mean over the six single-position reads).

Interestingly, answer adoption is small for single positions, and considerably smaller for than for full columns, despite specifically considering load-bearing rows. Since a transplant also makes the twin lose its own answer in most cases (it keeps it in well under half of the pairs), CODI seems to require more than the latent CoT's "shape": the content matters, but cannot be transplanted via alone, presumably because the rest of the residual stream gets recomputed from the twin's own question. Within , the effect comes almost entirely from the third thought: transplanting alone disrupts and transplants about as much as transplanting all six, while transplanting or has no effect at all. The third thought is also where the NLA reads intermediates best, in line with Cywiński et al. (2025), who found the third latent vector to store an intermediate value.

The NLA, which we previously had used for decoding, can also be used as an intervention method (cf. Fraser-Taliente et al.). To do so, we take the NLA verbalization, edit its text by replacing a numerical intermediate value with the corresponding value of the twin problem, reconstruct the activation, and patch it back into the chain-of-thought (at all six positions jointly).

Example of an NLA intervention, replacing 20 with 40. The edited verbalization then gets reconstructed via the activation reconstructor (AR), yielding z'=AR(v'), v'=edit(v), v=AV(z). Chosen is a problem where the CoT is load-bearing.

We measure then whether CODI's answer after patching the reconstructed modified activation is changed, and whether it adopts the answer that should have resulted if we indeed edited the causal path of the computation.

Editing the value in the thought's verbalization rarely implements the counterfactual. Fraction of load-bearing twin pairs that keep their own answer (gray) or adopt the counterfactual answer (blue), for (left) and (right).

For , edits disrupt the computation more than a random perturbation of the same size, but they adopt the counterfactual answer in only a handful of pairs (7%). This is, however, not far below what any transplant achieves here: reconstructing from the donor's own description adopts the donor's answer in 12% of pairs, as often as patching the donor's raw (previous section). For , the edit is not distinguishable from a random perturbation and adopts the counterfactual answer in only 2% of pairs, against 19% for the donor's description.

In this work, we have trained and studied a Qwen3-8B-CODI model on GSM8k data. Like previous CODIs, simple methods like LogitLens fail to decode the model's latent recurrent thoughts . However, more expressive activation verbalizers, in particular NLAs (see the appendix for Activation Oracles), are able to surface intermediates and answer values from the thought positions to some extent, and works best for intermediates. Like in Cywinsky et al., we find that decoding works best from the residual stream . In using NLAs, we find in addition that the thoughts themselves can be read as well.

When doing interventions, we found that the residual stream pooled over layers carries computation that will guide the model to a different answer, again more so than just from the thoughts . However, replacing the thoughts , transplanted or directly edited via the NLA, induced hardly any answer adoption, even though it did disrupt the answer.

Overall, this suggests that while CODI's thoughts are not completely opaque, their computational role largely remains unclear, at least in this model. Going forward, this suggests that clean opaque-serial-depth-increasing latent reasoning might be hard to obtain, but also that it might be hard to monitor. However, it is conceivable that the mointorability might improve with better training paradigms and larger models, similarly to what has been observed regarding feature superposition. Still, latent reasoning remains an intrinsically more opaque form of computation compared to verbal CoT and should be treated with appropriate caution.

We had found before that the residual stream is better decodable than the thoughts . This suggests that it is somehow richer. We here do a dimensionality analysis, showing that the residual stream indeed takes more components to explain and thus spans a higher-dimensional space across problems and positions.

Thought vectors are much more low-dimensional than the residual stream. PCA over the flattened set of vectors , .

The full LogitLens and J-Lens grids over all strided hidden states and all seven columns.

Near the output, the lenses read the answer and intermediate results well above chance, but nothing from the thoughts themselves. For each of the held-out evaluation questions of all readers, the top-k tokens of the six thought positions are pooled per layer. A problem counts if one of these tokens expresses its gold final answer (left) or an intermediate result of its reference solution (right) as a complete number, e.g. "9" or "nine". Dotted: chance, scoring the same tokens against the other questions' values. The leftmost point is the fed-back thought itself.

We here analyze whether the problems where the base model needs verbal CoT coincide with the problem that CODI can do only with verbal or latent CoT.

Venn diagram of different GSM8k test set subsets. Base needs CoT means the base model fails the single pass but solves with verbal CoT. CODI needs some CoT means its noCoT decode fails but the latent or the verbal mode solves the problem.

In the main text, we have used natural language autoencoders to analyze CODI's thoughts. Here, we use activation oracles (AOs, Karvonen et al. (2025)). In contrast to NLAs, they are trained in a supervised manner, instead of a unlabelled reconstruction signal. Thus, they may have higher recall, at the cost of biasing the output by the training label.

What should be the supervision signal, given that we don't know the content of the thoughts? We approach this by imputing the thought content (cf. Bauer et al. (2026)). Specifically, training proceeds as follows: each training sample draws a random set of vectors to verbalize: sources from the band plus the thoughts , at a random subset of thought positions , ordered. To have a baseline, a tenth of the samples feed question-region residual stream activations alone, and another tenth feed them together with the thoughts. Independently, 60 percent of the samples feed activations only, 20 percent replace them with the teacher transcript as plain text, and 20 percent feed both. We then ask one of eight questions about the computation (the final answer, single or all intermediate values, single or all step sentences, the operations used, the step count, or the numbers given in the problem) and supervise on the corresponding ground-truth completion, written either as sentences ("The discount is 20/100 * $60 = $12. So the game will cost $60 - $12 = $48.") or as bare values ("12, 48"). We finetune this model from the plain Qwen3-8B base with a fresh LoRA, otherwise the architecture is the same as the NLA's.

Activation oracle training increases accuracy on held-out problems. Solid lines score first-number answer accuracy, and dashed lines the judged any-intermediate match as in the NLA figure. Faint horizontal lines: LogitLens and J-lens on the same streams and questions (for they are at zero). The appendix on eval scaling curves shows the full curves with the val CE panel.

Importantly, for the final answer, the full and the -only reads are above the question-token only baseline (for intermediates, however, the question-only read does at least as well as the thought reads, so the AO may recompute intermediates from the question rather than reading them from the thoughts). As for the NLA, appending to helps for the final answer, though part of this may come from layers 35 and 36, where the lenses alone already read the answer for a third of problems.

Readouts by the activation oracle of a showcase problem, before (left) and after (right) reader training. Each cell shows two samples at under our italic one-line digest. At the final checkpoint both reads recover the answer-feeding intermediate in every sample. The full read (top) gives the gold answer in one of the two samples, while the -only read (bottom) lands near but off it.

The NLA figure in the main text and the AO figure in the AO appendix omit their middle panel for compactness. Here we show the full three-panel versions, generated from the same scripts and data passes as the compact ones. The middle panels render the eval error on double-log axes shared with the train panel, so the eval scaling can be compared against the training curve directly.

NLA scaling curves. As the main-text figure, plus the middle panel showing the decode error (one minus the decode score) on the train panel's double-log axes, and dotted lines showing the coincidence control (the any-intermediate criterion for verbalizations of an unrelated problem's thoughts). Both readers exceed the coincidence floor throughout training.

Activation oracle scaling curves. As the figure in the activation oracle section, plus the middle panel showing the val CE over chance per label type, solid the answer label and dashed the all-values label.

Whether the later thoughts are reused or regenerated changes little. Fraction of the 104 load-bearing problems still answered correctly when one thought is replaced, either reusing the recorded later thoughts or regenerating them autoregressively from the corrupted thought.

^Note that the different modes trained jointly into a single model imposes an expressivity constraint: a model trained with verbalCoT SFT alone reaches much better performance. Still, within the class of jointly trained models, the latent over noCoT uplift is an interesting phenomenon to study.

Note that the different modes trained jointly into a single model imposes an expressivity constraint: a model trained with verbalCoT SFT alone reaches much better performance. Still, within the class of jointly trained models, the latent over noCoT uplift is an interesting phenomenon to study.