Skip to content
aiqbench takeAI Research

aiqbench analysis

This paper introduces DP-Bench, the first comprehensive benchmark for automatically generating data products from high-level business requests using LLMs. By bridging natural language intent with structured data representation, it addresses a critical gap in end-to-end automated data pipeline evaluation beyond isolated text-to-SQL tasks. The release of this dataset and baseline methods provides a foundational resource for advancing research in autonomous data engineering.
Sources coveringarXiv, LessWrong, Latent Space
What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

Bridging Business Intent and Data: A Benchmark for Automatic Relational Data Product Generation

arXiv2 min read

arXiv:2512.15798v2 Announce Type: replace-cross Abstract: A data product is designed to address a specific business need by transforming raw data into a curated, usable asset that delivers actionable insights. Despite practical advances in related areas like text-to-SQL and ELT pipelines, there is no comprehensive benchmark for evaluating the end-to-end process of automatically generating such data products from high-level business requests. To fill this gap, we introduce DP-Bench, a first-of-its-kind benchmark for automatic data product creation, built by combining elements from existing ELT and text-to-SQL datasets. We also propose baseline methods using LLMs to generate data products, providing a foundation for future research in bridging natural language business intent and structured data representation. We make the DP-Bench dataset and code available at: https://huggingface.co/datasets/ibm-research/dp-bench.

Abstract:A data product is designed to address a specific business need by transforming raw data into a curated, usable asset that delivers actionable insights. Despite practical advances in related areas like text-to-SQL and ELT pipelines, there is no comprehensive benchmark for evaluating the end-to-end process of automatically generating such data products from high-level business requests. To fill this gap, we introduce DP-Bench, a first-of-its-kind benchmark for automatic data product creation, built by combining elements from existing ELT and text-to-SQL datasets. We also propose baseline methods using LLMs to generate data products, providing a foundation for future research in bridging natural language business intent and structured data representation. We make the DP-Bench dataset and code available at: this https URL.

Databases (cs.DB); Computation and Language (cs.CL)

arXiv:2512.15798 [cs.DB]

(or arXiv:2512.15798v2 [cs.DB] for this version)

https://doi.org/10.48550/arXiv.2512.15798 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Full text under the publisher's open licence · 214 words

Related coverage