SFT Also Drives Safety Eval Results in Olmo 3

tl;dr Engels et al. 2026 found that most safety-relevant properties of Gemini 3.1 Pro and Gemini 3 Flash were already present after SFT: post-SFT and production models score nearly identically across their safety evals. The authors were surprised by this and were unsure whether it would hold for other model families. We find it holds for Olmo 3 32B Think. The released SFT, DPO, and RLVR checkpoints are within noise of each other on every Petri dimension, ODCV-Bench, StrongREJECT, ImpossibleBench, and Mask, while LiveCodeBench roughly doubles. Figure 1. Olmo 3 32B Think after each post-training stage. Higher is worse on everything except LiveCodeBench. Error bars are 95% CIs. Petri is the mean judge score (1-10) over all scenarios; full dimensions in the appendix. This replication was done as part of the Second Look Fellowship and supervised by Yixiong Hao and Zephaniah Roe. Our code can be found here. Ai2 releases every post-training checkpoint for Olmo 3 32B Think (SFT → DPO → RLVR), so rather than re-running SFT as Engels et al. did, we evaluate each released stage directly.[1] We use the same evals where an open equivalent exists (Petri, ODCV-Bench, StrongREJECT), add MASK and ImpossibleBench, and use LiveCodeBench as a capability control. Olmo 3's RLVR stage is small relative to frontier post-training, so this says little about heavily-RL'd models. However, we do believe this replication rules out  the hypothesis that the Gemini results are an artifact of Gemini's pipeline or SFT mixture. One limitation of this experiment is that flat evals don’t necessarily imply unchanged safety properties. A model can be badly misaligned and still look fine on some safety evals. For example, prior work found verbalized eval awareness  changes over these checkpoints, while our own Petri eval sees ~0 at all stages (see Figure 2). Appendix Figure 2: All Petri dimensions. Dimensions below the dashed line describe the transcript rather than the model's behavior. Benchmarks Petri: an auditor model runs 173 multi-turn scenarios and a judge scores each transcript 1-10 on each dimension. We use Sonnet 4.6 as the auditor and Opus 4.6 as the judge. ODCV-Bench: misbehavior rate as in Engels et al. StrongREJECT: unsafe response rate on harmful prompts. ImpossibleBench: LiveCodeBench tasks with unsatisfiable tests, either a single wrong test (one-off) or mutually contradictory tests (conflicting). MASK: lie rate of pressured responses that contradict the model’s stated beliefs. Tool calling: Olmo 3 Think’s chat template ignores the standard tools argument, so serving with it doesn’t expose any functions to the model. We edit the template to render tools as they do for the Instruct checkpoints and make vLLM’s Olmo 3 tool-call parser more tolerant of malformed calls. ^ Olmo 3 32B Think is the most capable model we know of with every post-training checkpoint public and a nontrivial RLVR stage. Discuss