aiqbench analysis
DataKernelBench demonstrates that LLMs can effectively generate optimized CUDA and Triton kernels for database queries, achieving up to 2.54x speedup over standard compilation methods on large-scale TPC-H workloads. The study highlights that full-query specialization and kernel fusion are critical strategies, with model performance driven more by workload context than hardware specifics. This research bridges the gap between machine learning operators and irregular data-movement patterns in high-performance computing environments.
DataKernelBench: Can LLMs Optimize Database Queries on GPUs?
arXiv:2608.25061v1 Announce Type: new Abstract: GPUs increasingly accelerate database systems, but query-specific peak performance still often relies on hand-written kernels. Existing LLM kernel benchmarks focus on machine learning operators, leaving irregular, heterogeneous, data-movement-heavy database-style operators untested. We introduce DataKernelBench, which translates SQL into validated PyTorch TorchPlan programs and evaluates LLMs that optimize either the core tensor-bounded snippet or the full query in CUDA or Triton through execution-guided repair. Across ten proprietary and open-weight models on TPC-H SF10 with an H100 GPU, the strongest full-query CUDA configuration achieves $2.11\times$ speedup over torch.compile at full pass rate. We find that higher-performing implementations commonly use kernel fusion and execution-strategy changes, stronger models benefit most from full-query specialization, and workload context matters more than hardware context. To handle data larger than GPU memory, we extend TorchPlan with Dask-cuDF for on-demand partition loading on TPC-H SF100 with four H100 GPUs, achieving $2.54\times$ speedup
Title:DataKernelBench: Can LLMs Optimize Database Queries on GPUs?
Abstract:GPUs increasingly accelerate database systems, but query-specific peak performance still often relies on hand-written kernels. Existing LLM kernel benchmarks focus on machine learning operators, leaving irregular, heterogeneous, data-movement-heavy database-style operators untested. We introduce DataKernelBench, which translates SQL into validated PyTorch TorchPlan programs and evaluates LLMs that optimize either the core tensor-bounded snippet or the full query in CUDA or Triton through execution-guided repair. Across ten proprietary and open-weight models on TPC-H SF10 with an H100 GPU, the strongest full-query CUDA configuration achieves $2.11\times$ speedup over this http URL at full pass rate. We find that higher-performing implementations commonly use kernel fusion and execution-strategy changes, stronger models benefit most from full-query specialization, and workload context matters more than hardware context. To handle data larger than GPU memory, we extend TorchPlan with Dask-cuDF for on-demand partition loading on TPC-H SF100 with four H100 GPUs, achieving $2.54\times$ speedup
Accepted at EMNLP 2026. Homepage: this https URL
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Databases (cs.DB); Machine Learning (cs.LG); Programming Languages (cs.PL)
arXiv:2608.25061 [cs.CL]
(or arXiv:2608.25061v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.25061 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 267 words
Related coverage
The hard physics and complex economics of AI’s insatiable hunger for power
SiliconANGLE · Sep 11, 2026
Presentation: Running AI at the Edge: Running Real Workloads Directly in the Browser
InfoQ · Aug 31, 2026
Opaque recurrence, and other AI terms that you should probably know
TechCrunch · Sep 7, 2026
Presentation: Designing AI Platforms for Reliability: Tools for Certainty, Agents for Discovery
InfoQ · May 27, 2026