analiza aiqbench
This arXiv preprint proposes a training change for fine-tuning language models on chain-of-thought math data. Soft-TF mixes teacher-forced embeddings with soft tokens computed in one parallel pass, adding no parameters or curriculum. The reported gains of +2.1 to +4.1 points on GSM8K are across several model families, but they come from a single unreviewed preprint, so readers should check whether they hold under independent replication and broader benchmarks.
Soft Teacher Forcing
arXiv:2610.17189v1 Announce Type: new Abstract: Soft tokens have so far been used to replace discrete chain-of-thought in latent reasoning methods. These approaches typically require multi-stage training and, in some cases, auxiliary modules, yet they still underperform supervised fine-tuning (SFT) on triples. In this paper, we ask whether soft tokens can be used to improve SFT itself, instead of as an avenue for latent reasoning. We introduce Soft Teacher Forcing (Soft-TF), a simple training algorithm that combines teacher-forced embeddings with soft tokens through random convex combinations. The soft tokens are computed in a single parallel forward pass over the ground-truth sequence. Soft-TF adds no parameters, no external model and no curriculum, and leaves inference unchanged. Experiments show that Soft-TF improves over SFT on GSM8K, MATH, MathQA and AQuA, by +2.1 to +4.1 points on GSM8K across seven Gemma, Llama and Mistral models.
Abstract:Soft tokens have so far been used to replace discrete chain-of-thought in latent reasoning methods. These approaches typically require multi-stage training and, in some cases, auxiliary modules, yet they still underperform supervised fine-tuning (SFT) on <input, chain-of-thought, output> triples. In this paper, we ask whether soft tokens can be used to improve SFT itself, instead of as an avenue for latent reasoning. We introduce Soft Teacher Forcing (Soft-TF), a simple training algorithm that combines teacher-forced embeddings with soft tokens through random convex combinations. The soft tokens are computed in a single parallel forward pass over the ground-truth sequence. Soft-TF adds no parameters, no external model and no curriculum, and leaves inference unchanged. Experiments show that Soft-TF improves over SFT on GSM8K, MATH, MathQA and AQuA, by +2.1 to +4.1 points on GSM8K across seven Gemma, Llama and Mistral models.
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2610.17189 [cs.CL]
(or arXiv:2610.17189v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.17189 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 237 cuvinte
Subiecte conexe
Beyond Distribution Matching: Selective Margin Preservation for Supervised Fine-Tuning
arXiv · 15 oct. 2026
CRAD: Class-wise Reliability-Aware Distillation for Decentralized Heterogeneous Federated Learning
arXiv · 2 sept. 2026
CALM: Class-wise Agreement and Label-gated Disagreement Modulation for Decentralized Federated Learning
arXiv · 9 sept. 2026
Oscar Winner Brings Monsters to Life With His Simulation Software
IEEE Spectrum · 28 aug. 2026