aiqbench analysis
TabQueryBench introduces a critical evaluation framework for synthetic tabular data by prioritizing analytical query fidelity over standard statistical metrics. The benchmark exposes significant gaps in current generative models, particularly regarding high-cardinality attributes and tail distributions where synthetic data fails to mimic real-world analytical behavior. This work establishes query-centric fidelity as an essential metric alongside privacy and correlation checks for advancing reliable synthetic data generation.
TabQueryBench: A Query-Centric Benchmark for Synthetic Tabular Data
arXiv:2607.03926v2 Announce Type: replace-cross Abstract: Synthetic tabular data support use cases like data sharing, model development under access restrictions, and rapid prototyping of analytical workflows. Modern generative models are evaluated by their statistical similarity, correlation structure, privacy, and downstream machine-learning utility. However, such evaluations leave a gap: they rarely test the structure that matters for analytical queries. We present TabQueryBench, a query-centric benchmark for evaluating synthetic tabular data with SQL-shaped analytical queries. TabQueryBench distills 12 public query sources into 44 reusable query templates and grounds them to each dataset through a policy-guided template-to-SQL pipeline. Across 49 datasets and 11 generative models, it produces more than 100 executable queries per dataset while preserving cross-model comparability. Our evaluation reveals a substantial gap between conventional and query-centric fidelity. Even the strongest model, RealTabFormer, reaches only 0.76 +/- 0.23 query-centric fidelity compared with 1.00 for real data. Failures are most pronounced for high-cardinality attributes and local or tail queries, while model quality also trades off against generation cost. These results show that synthetic data can appear statistically realistic yet still fail to preserve the analytical behavior of real data, motivating query-centric fidelity as a necessary complement to existing evaluation metrics. We open source TabQueryBench to facilitate research in this direction.
Abstract:Synthetic tabular data support use cases like data sharing, model development under access restrictions, and rapid prototyping of analytical workflows. Modern generative models are evaluated by their statistical similarity, correlation structure, privacy, and downstream machine-learning utility. However, such evaluations leave a gap: they rarely test the structure that matters for analytical queries. We present TabQueryBench, a query-centric benchmark for evaluating synthetic tabular data with SQL-shaped analytical queries. TabQueryBench distills 12 public query sources into 44 reusable query templates and grounds them to each dataset through a policy-guided template-to-SQL pipeline. Across 49 datasets and 11 generative models, it produces more than 100 executable queries per dataset while preserving cross-model comparability. Our evaluation reveals a substantial gap between conventional and query-centric fidelity. Even the strongest model, RealTabFormer, reaches only 0.76 +/- 0.23 query-centric fidelity compared with 1.00 for real data. Failures are most pronounced for high-cardinality attributes and local or tail queries, while model quality also trades off against generation cost. These results show that synthetic data can appear statistically realistic yet still fail to preserve the analytical behavior of real data, motivating query-centric fidelity as a necessary complement to existing evaluation metrics. We open source TabQueryBench to facilitate research in this direction.
Databases (cs.DB); Artificial Intelligence (cs.AI)
arXiv:2607.03926 [cs.DB]
(or arXiv:2607.03926v2 [cs.DB] for this version)
https://doi.org/10.48550/arXiv.2607.03926 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 295 words
Related coverage
Fuzzy Distribution Modeling for Synthetic Tabular Data Generation with Causality Preservation
arXiv · Sep 29, 2026
Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspective
arXiv · Oct 2, 2026
Introducing TabFM: A zero-shot foundation model for tabular data
Google Research · Jun 30, 2026
TabKD: Tabular Knowledge Distillation through Interaction Diversity of Learned Feature Bins
arXiv · Jun 15, 2026