analiza aiqbench
This research introduces Polaris, a system that leverages existing retrieval benchmark data to fine-tune LLMs via Direct Preference Optimization (DPO) specifically for generating metadata useful in NL2SQL tasks. By prioritizing retrieval effectiveness over mere fluency and addressing vocabulary mismatches through abbreviation expansion, the method significantly outperforms current state-of-the-art solutions like AutoDDG. The work demonstrates a practical pathway to repurpose standard evaluation datasets into direct supervision signals for optimizing LLMs toward specific downstream utility metrics.
Polaris: Learning to Generate Table Descriptions from Retrieval Feedback
arXiv:2608.17171v1 Announce Type: new Abstract: Many table-centric NLP tasks such as NL2SQL first retrieve relevant tables from large collections using keyword search. Recent work uses LLMs to generate natural-language table descriptions to improve retrieval, but they are typically optimized for fluency rather than retrieval effectiveness. We present Polaris, a system that trains an LLM to generate table descriptions directly from retrieval feedback. Our key insight is that existing table retrieval benchmarks already contain the supervision needed for this task: given query-table relevance judgments, we generate multiple candidate descriptions for each table, rank them by their BM25 retrieval effectiveness, and use the resulting preference pairs to fine-tune the LLM with Direct Preference Optimization (DPO). Polaris further expands abbreviated table and column names before generation to reduce vocabulary mismatch. Extensive experiments show that Polaris outperforms the state-of-the-art AutoDDG solution, often by a significant margin. More broadly, our results demonstrate that retrieval benchmarks can be repurposed as supervision for training LLMs to generate retrieval-oriented metadata.
Abstract:Many table-centric NLP tasks such as NL2SQL first retrieve relevant tables from large collections using keyword search. Recent work uses LLMs to generate natural-language table descriptions to improve retrieval, but they are typically optimized for fluency rather than retrieval effectiveness. We present Polaris, a system that trains an LLM to generate table descriptions directly from retrieval feedback. Our key insight is that existing table retrieval benchmarks already contain the supervision needed for this task: given query-table relevance judgments, we generate multiple candidate descriptions for each table, rank them by their BM25 retrieval effectiveness, and use the resulting preference pairs to fine-tune the LLM with Direct Preference Optimization (DPO). Polaris further expands abbreviated table and column names before generation to reduce vocabulary mismatch. Extensive experiments show that Polaris outperforms the state-of-the-art AutoDDG solution, often by a significant margin. More broadly, our results demonstrate that retrieval benchmarks can be repurposed as supervision for training LLMs to generate retrieval-oriented metadata.
Computation and Language (cs.CL); Databases (cs.DB)
arXiv:2608.17171 [cs.CL]
(or arXiv:2608.17171v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.17171 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 254 cuvinte
Subiecte conexe
Operationalizing Genie Ontology in Your Data Stack
Databricks · 1 sept. 2026
New in Amazon Bedrock AgentCore: Build agents with broader knowledge and continuous learning
AWS · 17 iun. 2026
Flow generation through natural language: An agentic modeling approach
Shopify Engineering Blog · 22 apr. 2026
Modernizing and scaling support operations with generative AI on AWS
AWS · 2 sept. 2026