analiza aiqbench
This arXiv preprint shows that a student model can pick up a judge's preference for an animal through RL post-training, even when it writes only invented syllables or numbers. The authors report transfer in 17 of 48 experimental cells at a 5% false-discovery rate and propose a four-stage chain (judge scores, reward-favoured strings, accumulation, transfer to answers) as the mechanism. For AI readers, the safety implication is that undesirable dispositions could move through reward signals that carry no visible semantic content, and the claim that pre-training measurements can anticipate where this happens still needs independent replication.
Subliminal Reinforcement Learning: traits travel through a judge's scores
arXiv:2610.41301v1 Announce Type: new Abstract: Subliminal learning, the transfer of a model's trait through data with no semantic trace of it, has been shown for students trained on a teacher's text. Post-training relies on group relative policy optimization with a language-model judge, where the student learns only from its own outputs and one judge's scores. We show this suffices: a student writes invented syllables or numbers, a judge made to prefer one animal scores them, and the student's preference moves toward that animal relative to a run whose judge prefers another. Of 48 design cells, two fail the validity gate and 17 establish the trait at 5% false-discovery rate, numbers-only cells included; elsewhere, our account gives the reason. Removing only the reward difference between judges removes transfer. We establish the principle as a chain of four stages: the trait must show in the judge's scores, favour strings that move the student, accumulate and reach the answers. Judges with the same preference built from other data, or on another model, place it on other strings, and transfer is not detected. Carried unchanged to AI-safety dispositions, the chain holds: undesirable dispositions reach the student's answers, and stages measured before training anticipate where, helping to prevent part of this risk.
Abstract:Subliminal learning, the transfer of a model's trait through data with no semantic trace of it, has been shown for students trained on a teacher's text. Post-training relies on group relative policy optimization with a language-model judge, where the student learns only from its own outputs and one judge's scores. We show this suffices: a student writes invented syllables or numbers, a judge made to prefer one animal scores them, and the student's preference moves toward that animal relative to a run whose judge prefers another. Of 48 design cells, two fail the validity gate and 17 establish the trait at 5% false-discovery rate, numbers-only cells included; elsewhere, our account gives the reason. Removing only the reward difference between judges removes transfer. We establish the principle as a chain of four stages: the trait must show in the judge's scores, favour strings that move the student, accumulate and reach the answers. Judges with the same preference built from other data, or on another model, place it on other strings, and transfer is not detected. Carried unchanged to AI-safety dispositions, the chain holds: undesirable dispositions reach the student's answers, and stages measured before training anticipate where, helping to prevent part of this risk.
37 pages, 4 figures, 2 tables, 2 Extended Data figures, 6 Extended Data tables; Supplementary Information included. Code and data: this https URL
Artificial Intelligence (cs.AI)
arXiv:2610.41301 [cs.AI]
(or arXiv:2610.41301v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.41301 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 319 cuvinte
Subiecte conexe
Thousand-dimensional structure
AI Alignment Forum · 30 iul. 2026
Accelerated Skill Learning via Dream Engineering and Biofeedback
LessWrong · 8 iun. 2026
Some Interesting Papers on RLVR
LessWrong · 9 iun. 2026
How independent researchers could investigate AI propensities after misalignment incidents
METR · 28 iul. 2026