aiqbench analysis
SLMFix addresses the critical bottleneck of syntactic errors in LLM-generated code by employing a cost-effective small language model fine-tuned with reinforcement learning. This approach significantly improves validation pass rates for low-resource programming languages and reduces error rates in high-resource domains without requiring expensive large-scale training. The method offers a practical pathway to enhance code quality in specialized environments where computational resources are constrained.
SLMFix: Leveraging Small Language Models for Domain Specific Language Error Fixing with Reinforcement Learning
arXiv:2511.19422v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown impressive capabilities in code generation across many programming languages but even state-of-the-art LLMs generate programs that contain syntactic errors and fail to complete the given tasks, especially for low-resource programming languages (LRPLs). In addition, the high cost of training makes finetuning LLMs unaffordable for those with constrained computational resources, further weakening the effectiveness of LLMs for code generation. In this work, we propose SLMFix, a novel code generation pipeline that leverages a small language model (SLM) finetuned using reinforcement learning (RL) techniques to fix syntactic errors in LLM-generated programs for domain-specific languages (DSLs) based on interpreter feedback. Our experimental results demonstrate the effectiveness and generalizability of our approach across multiple DSLs, improving the validator pass rates by 40% on LRPLs and eliminating more than 50% of syntactic errors for high-resource DSLs. Notably, SLMFix brings substantial performance improvement to the base model and outperforms supervised finetuning approach even for 7B models on LRPLs including Ansible and Lean, showing the potential of our approach in improving the quality of LLM-generated programs.
Abstract:Large language models (LLMs) have shown impressive capabilities in code generation across many programming languages but even state-of-the-art LLMs generate programs that contain syntactic errors and fail to complete the given tasks, especially for low-resource programming languages (LRPLs). In addition, the high cost of training makes finetuning LLMs unaffordable for those with constrained computational resources, further weakening the effectiveness of LLMs for code generation. In this work, we propose SLMFix, a novel code generation pipeline that leverages a small language model (SLM) finetuned using reinforcement learning (RL) techniques to fix syntactic errors in LLM-generated programs for domain-specific languages (DSLs) based on interpreter feedback. Our experimental results demonstrate the effectiveness and generalizability of our approach across multiple DSLs, improving the validator pass rates by 40% on LRPLs and eliminating more than 50% of syntactic errors for high-resource DSLs. Notably, SLMFix brings substantial performance improvement to the base model and outperforms supervised finetuning approach even for 7B models on LRPLs including Ansible and Lean, showing the potential of our approach in improving the quality of LLM-generated programs.
Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Programming Languages (cs.PL)
arXiv:2511.19422 [cs.SE]
(or arXiv:2511.19422v2 [cs.SE] for this version)
https://doi.org/10.48550/arXiv.2511.19422 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 273 words
Related coverage
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
Learn2Zinc: Fine-tuning Small Language Models for Text-to-Model Translation in MiniZinc
arXiv · Jul 25, 2026
Presentation: Beyond Prompting: Context Engineering and Memory Management for AI Systems at Scale
InfoQ · Jun 10, 2026
The basic argument for making AI better at moral philosophy
LessWrong · Oct 14, 2026