aiqbench analysis
This paper introduces DP-Bench, the first comprehensive benchmark for automatically generating data products from high-level business requests using LLMs. By bridging natural language intent with structured data representation, it addresses a critical gap in end-to-end automated data pipeline evaluation beyond isolated text-to-SQL tasks. The release of this dataset and baseline methods provides a foundational resource for advancing research in autonomous data engineering.
Bridging Business Intent and Data: A Benchmark for Automatic Relational Data Product Generation
arXiv:2512.15798v2 Announce Type: replace-cross Abstract: A data product is designed to address a specific business need by transforming raw data into a curated, usable asset that delivers actionable insights. Despite practical advances in related areas like text-to-SQL and ELT pipelines, there is no comprehensive benchmark for evaluating the end-to-end process of automatically generating such data products from high-level business requests. To fill this gap, we introduce DP-Bench, a first-of-its-kind benchmark for automatic data product creation, built by combining elements from existing ELT and text-to-SQL datasets. We also propose baseline methods using LLMs to generate data products, providing a foundation for future research in bridging natural language business intent and structured data representation. We make the DP-Bench dataset and code available at: https://huggingface.co/datasets/ibm-research/dp-bench.
Abstract:A data product is designed to address a specific business need by transforming raw data into a curated, usable asset that delivers actionable insights. Despite practical advances in related areas like text-to-SQL and ELT pipelines, there is no comprehensive benchmark for evaluating the end-to-end process of automatically generating such data products from high-level business requests. To fill this gap, we introduce DP-Bench, a first-of-its-kind benchmark for automatic data product creation, built by combining elements from existing ELT and text-to-SQL datasets. We also propose baseline methods using LLMs to generate data products, providing a foundation for future research in bridging natural language business intent and structured data representation. We make the DP-Bench dataset and code available at: this https URL.
Databases (cs.DB); Computation and Language (cs.CL)
arXiv:2512.15798 [cs.DB]
(or arXiv:2512.15798v2 [cs.DB] for this version)
https://doi.org/10.48550/arXiv.2512.15798 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 214 words
Related coverage

Digital States 2026: AI Moves From Experiment to Infrastructure
govtech.com · Sep 24, 2026

News
Quantum Computing Report · Jul 25, 2026

Netskope (NTSK) Q2 2027 Earnings Call Transcript
fool.com · Sep 10, 2026

Presentation: From AI Agent Demo to Production: Automated Testing and Evaluation
InfoQ · Sep 7, 2026