analiza aiqbench
DeepTable addresses a critical limitation in current LLM table understanding by explicitly modeling hierarchical row and column structures rather than treating data as linear sequences. By integrating Structural Attention Biases and Tree Path Encoding with parameter-efficient fine-tuning, the method delivers consistent performance gains across major benchmarks without requiring full model retraining. This structural awareness offers a practical pathway to improve multi-level header comprehension in existing large language models.
DeepTable: Structural Attention Biases and Tree Path Encoding for Hierarchical Table Understanding
arXiv:2609.07707v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated strong performance in table understanding. However, they typically process table content and headers as linearized token sequences. This representation weakens the two-dimensional and hierarchical structural relationships encoded by multi-level row and column headers. Existing parameter-efficient fine-tuning methods incorporate basic row and column information but do not explicitly capture the rich structural dependencies induced by hierarchical table headers. We propose DeepTable, a structure-aware approach for table understanding with LLMs. DeepTable comprises two complementary components. Structural Attention Bias (SAB) introduces learnable biases into the attention logits to explicitly represent whether pairs of table tokens share the same row or column. Tree Path Encoding (TPE) represents each table token using the ancestor paths of its row and column headers, preserving its position within the multi-level table structure. We integrate DeepTable with TableLoRA (He et al., 2025) to inject structural information into parameter-efficient adaptation. Across three LLM backbones, DeepTable consistently improves the corresponding TableLoRA baselines on three table question answering benchmarks, achieving average gains of 7.42 points on HiTab, 3.23 points on WikiTQ, and 2.01 BLEU points on FeTaQA. These results demonstrate the effectiveness of the proposed structural biases across different LLM backbones.
Abstract:Large language models (LLMs) have demonstrated strong performance in table understanding. However, they typically process table content and headers as linearized token sequences. This representation weakens the two-dimensional and hierarchical structural relationships encoded by multi-level row and column headers. Existing parameter-efficient fine-tuning methods incorporate basic row and column information but do not explicitly capture the rich structural dependencies induced by hierarchical table headers. We propose DeepTable, a structure-aware approach for table understanding with LLMs. DeepTable comprises two complementary components. Structural Attention Bias (SAB) introduces learnable biases into the attention logits to explicitly represent whether pairs of table tokens share the same row or column. Tree Path Encoding (TPE) represents each table token using the ancestor paths of its row and column headers, preserving its position within the multi-level table structure. We integrate DeepTable with TableLoRA (He et al., 2025) to inject structural information into parameter-efficient adaptation. Across three LLM backbones, DeepTable consistently improves the corresponding TableLoRA baselines on three table question answering benchmarks, achieving average gains of 7.42 points on HiTab, 3.23 points on WikiTQ, and 2.01 BLEU points on FeTaQA. These results demonstrate the effectiveness of the proposed structural biases across different LLM backbones.
14 pages, 2 figures. Submitted to EACL 2027
Computation and Language (cs.CL)
arXiv:2609.07707 [cs.CL]
(or arXiv:2609.07707v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.07707 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 298 cuvinte
Subiecte conexe
PIRAMID: Progress and Plans
LessWrong · 27 iul. 2026
Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization
Meta Engineering · 15 iul. 2026
Restricting Trainable Lie-Algebra Growth in Equivariant Quantum Networks via Hierarchical Ancilla-Controlled Subspace Projections
arXiv · 28 sept. 2026
Adaptive Parallel Reasoning: The Next Paradigm in Efficient Inference Scaling
BAIR Blog · 8 mai 2026