aiqbench analysis
This survey consolidates fragmented research on using LLMs and agentic systems to automate GPU kernel generation, addressing the critical bottleneck of translating high-level algorithms into efficient low-level hardware operations. By systematically organizing existing approaches, datasets, and benchmarks, the work provides a necessary reference for scaling kernel optimization beyond manual expert intervention. The analysis highlights the potential of compressing tacit hardware knowledge into automated workflows while outlining key challenges for the next generation of AI infrastructure.
Towards Automated Kernel Generation in the Era of LLMs
arXiv:2601.15727v3 Announce Type: replace Abstract: The performance of modern AI systems is fundamentally constrained by the quality of their underlying GPU kernels, which translate high-level algorithmic semantics into low-level hardware operations. Achieving near-optimal kernels requires expert-level understanding of hardware architectures and programming models, making kernel engineering a critical but notoriously time-consuming and non-scalable process. Recent advances in large language models and LLM-based agents have opened new possibilities for automating kernel generation and optimization. LLMs are well-suited to compress expert-level kernel knowledge that is difficult to formalize, while agentic systems further enable scalable optimization by casting kernel development as an iterative, feedback-driven loop. Rapid progress has been made in this area. However, the field remains fragmented and lacks a systematic perspective for LLM-driven kernel generation. This survey addresses this gap by providing a structured overview of existing approaches, spanning LLM-based approaches and agentic optimization workflows, and systematically organizing the datasets and benchmarks that underpin learning and evaluation in this domain. Moreover, key open challenges and future research directions are further outlined, aiming to establish a comprehensive reference for the next generation of automated kernel optimization. To keep track of this field, we maintain an open-source GitHub repository at https://github.com/flagos-ai/awesome-LLM-driven-kernel-generation.
Abstract:The performance of modern AI systems is fundamentally constrained by the quality of their underlying GPU kernels, which translate high-level algorithmic semantics into low-level hardware operations. Achieving near-optimal kernels requires expert-level understanding of hardware architectures and programming models, making kernel engineering a critical but notoriously time-consuming and non-scalable process. Recent advances in large language models and LLM-based agents have opened new possibilities for automating kernel generation and optimization. LLMs are well-suited to compress expert-level kernel knowledge that is difficult to formalize, while agentic systems further enable scalable optimization by casting kernel development as an iterative, feedback-driven loop. Rapid progress has been made in this area. However, the field remains fragmented and lacks a systematic perspective for LLM-driven kernel generation. This survey addresses this gap by providing a structured overview of existing approaches, spanning LLM-based approaches and agentic optimization workflows, and systematically organizing the datasets and benchmarks that underpin learning and evaluation in this domain. Moreover, key open challenges and future research directions are further outlined, aiming to establish a comprehensive reference for the next generation of automated kernel optimization. To keep track of this field, we maintain an open-source GitHub repository at this https URL.
In IJCAI 2026. 9 pages, 1 figure
Machine Learning (cs.LG); Computation and Language (cs.CL)
arXiv:2601.15727 [cs.LG]
(or arXiv:2601.15727v3 [cs.LG] for this version)
Full text under the publisher's open licence · 217 words
Related coverage
2026.10: You are here 📍
Home Assistant · Oct 7, 2026
Automated Hardware Validation Test Plan Generation for Large Scale AI Datacenter Platforms Using a Generative AI Multi-Agents Architecture
arXiv · Jul 21, 2026
LLMs as MDP Designers: A Dependency-Aware Agentic Framework for Automated Robot Policy Generation
arXiv · Nov 4, 2026
Automated IEP Generation from Traditional Chinese Parent-Teacher Interviews via Corpus-Grounded Feature Diffusion
arXiv · Jun 9, 2026