aiqbench analysis
This research demonstrates that Group Relative Policy Optimization (GRPO) enables robust Chain-of-Thought learning in Natural Language Inference without requiring human-labeled rationales, effectively mitigating dataset biases. By leveraging parameter-efficient fine-tuning and quantization, the study proves that 32B models can achieve superior adversarial generalization while fitting within constrained memory limits. The findings offer a practical pathway for deploying more reliable NLI systems in real-world applications like fact-checking.
Evaluating the Scalability and Adversarial Generalization of GRPO-Trained NLI Models
arXiv:2504.18376v3 Announce Type: replace Abstract: Natural Language Inference (NLI) is a central task in natural language understanding with applications in fact-checking, question answering, and information retrieval. Despite its importance, current NLI systems heavily rely on supervised learning with datasets that often contain annotation artifacts and biases, limiting generalization and real-world applicability. In this work, we apply a reinforcement learning-based approach using Group Relative Policy Optimization (GRPO) for Chain-of-Thought (CoT) learning in NLI, eliminating the need for human-labeled rationales and enabling this type of training on challenging datasets such as ANLI. We fine-tune 7B, 14B, and 32B language models using parameter-efficient techniques (LoRA and QLoRA), demonstrating strong performance across standard and adversarial NLI benchmarks. At the 32B scale, GRPO-trained models generalize better than other supervised baselines in adversarial sets. With AWQ quantization, the 32B model fits within 22GB of CUDA memory. This work provides a scalable and practical framework for building robust NLI systems without sacrificing inference quality.
Abstract:Natural Language Inference (NLI) is a central task in natural language understanding with applications in fact-checking, question answering, and information retrieval. Despite its importance, current NLI systems heavily rely on supervised learning with datasets that often contain annotation artifacts and biases, limiting generalization and real-world applicability. In this work, we apply a reinforcement learning-based approach using Group Relative Policy Optimization (GRPO) for Chain-of-Thought (CoT) learning in NLI, eliminating the need for human-labeled rationales and enabling this type of training on challenging datasets such as ANLI. We fine-tune 7B, 14B, and 32B language models using parameter-efficient techniques (LoRA and QLoRA), demonstrating strong performance across standard and adversarial NLI benchmarks. At the 32B scale, GRPO-trained models generalize better than other supervised baselines in adversarial sets. With AWQ quantization, the 32B model fits within 22GB of CUDA memory. This work provides a scalable and practical framework for building robust NLI systems without sacrificing inference quality.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2504.18376 [cs.CL]
(or arXiv:2504.18376v3 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2504.18376 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 248 words
Related coverage
[AINews] not much happened today
Latent Space · Oct 3, 2026
Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
LessWrong · Oct 5, 2026
Against Export Controls (and China Threat Models)
LessWrong · Sep 24, 2026
Generating Adversarial Texts for Machine Translation via GRPO
arXiv · Sep 9, 2026