aiqbench analysis
PulseBench-Tab introduces a robust, multilingual benchmark for table extraction that addresses complex real-world challenges like merged cells across nine languages. The release of the T-LAG metric, which evaluates structural and content fidelity using graph-based logic, provides a more rigorous standard for assessing both open-source and commercial systems. This resource offers a critical testing ground for advancing document intelligence models beyond simple content matching.
PulseBench-Tab: A Multilingual Benchmark for Table Extraction with Graph-Based Evaluation
arXiv:2606.07534v1 Announce Type: cross Abstract: We introduce PulseBench-Tab, an open multilingual benchmark for evaluating table extraction from document images. The benchmark comprises 1,820 human-annotated tables spanning 9 languages and 4 scripts (Latin, CJK, Arabic, Cyrillic), drawn from 380 real-world source documents including financial filings, government reports, and regulatory disclosures. Tables range from 2 to 1,183 cells, with 48.1% containing merged or spanning cells. Alongside the dataset, we propose T-LAG (Table Logical Adjacency Graph), a novel evaluation metric that models tables as directed graphs over cell adjacencies and computes structural and content fidelity in a single score via optimal bipartite matching. We evaluate 9 commercial and open-source table extraction systems across the benchmark and report per-language breakdowns. The full dataset, scoring code, and all provider outputs are publicly available.
Abstract:We introduce PulseBench-Tab, an open multilingual benchmark for evaluating table extraction from document images. The benchmark comprises 1,820 human-annotated tables spanning 9 languages and 4 scripts (Latin, CJK, Arabic, Cyrillic), drawn from 380 real-world source documents including financial filings, government reports, and regulatory disclosures. Tables range from 2 to 1,183 cells, with 48.1% containing merged or spanning cells. Alongside the dataset, we propose T-LAG (Table Logical Adjacency Graph), a novel evaluation metric that models tables as directed graphs over cell adjacencies and computes structural and content fidelity in a single score via optimal bipartite matching. We evaluate 9 commercial and open-source table extraction systems across the benchmark and report per-language breakdowns. The full dataset, scoring code, and all provider outputs are publicly available.
14 pages, 5 figures, 8 tables. Dataset: this https URL Code: this https URL
Information Retrieval (cs.IR); Computation and Language (cs.CL)
arXiv:2606.07534 [cs.IR]
(or arXiv:2606.07534v1 [cs.IR] for this version)
https://doi.org/10.48550/arXiv.2606.07534 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 233 words