aiqbench analysis
TechPulse take: This research exposes a critical "Footprint Bias" in Document Layout Analysis, demonstrating that current robustness evaluations relying on area-centric stress testing fail to capture the true structural vulnerabilities of document intelligence systems. The authors introduce ProSA, a lightweight auditing framework that shifts the focus to structure-aware diagnosis, revealing that structural identity loss correlates far more strongly with performance degradation than simple occlusion metrics. For practitioners building retrieval-augmented generation pipelines, this underscores the necessity of moving beyond footprint-based metrics to accurately assess failure propagation in long-document QA tasks.
How Do Document Parsers Break? Auditing Structural Vulnerability in Document Intelligence
arXiv:2605.19309v3 Announce Type: replace Abstract: Document Layout Analysis (DLA) pipelines provide structured page representations for retrieval-augmented generation, long-document question answering, and other document intelligence systems, yet their robustness evaluation remains largely area-centric. We identify this Footprint Bias and propose ProSA, a lightweight output-level auditing framework that decouples controlled probing, policy-driven targeting, and structure-aware diagnosis. ProSA combines Block-level Structural Loss Rate (B-SLR), granularity-aware exposure descriptors, and pathway attribution to analyze where structural identity is lost, at what exposure granularity failures emerge, and how failures propagate. Across MinerU and PP-StructureV3 on 1,000 pages, affected area weakly tracks perturbation-induced OCR instability (R^2=0.384/0.110), whereas B-SLR aligns much more closely with it (R^2=0.727/0.916). Exposure descriptors further separate occlusion- and topology-dominant pathways, while matched-footprint structural probes cause much larger downstream QA/retrieval degradation compared to area-matched erasure. These results shift DLA robustness evaluation from footprint-based stress testing toward structure-aware vulnerability auditing.
Abstract:Document Layout Analysis (DLA) pipelines provide structured page representations for retrieval-augmented generation, long-document question answering, and other document intelligence systems, yet their robustness evaluation remains largely area-centric. We identify this Footprint Bias and propose ProSA, a lightweight output-level auditing framework that decouples controlled probing, policy-driven targeting, and structure-aware diagnosis. ProSA combines Block-level Structural Loss Rate (B-SLR), granularity-aware exposure descriptors, and pathway attribution to analyze where structural identity is lost, at what exposure granularity failures emerge, and how failures propagate. Across MinerU and PP-StructureV3 on 1,000 pages, affected area weakly tracks perturbation-induced OCR instability (R^2=0.384/0.110), whereas B-SLR aligns much more closely with it (R^2=0.727/0.916). Exposure descriptors further separate occlusion- and topology-dominant pathways, while matched-footprint structural probes cause much larger downstream QA/retrieval degradation compared to area-matched erasure. These results shift DLA robustness evaluation from footprint-based stress testing toward structure-aware vulnerability auditing.
Computation and Language (cs.CL)
arXiv:2605.19309 [cs.CL]
(or arXiv:2605.19309v3 [cs.CL] for this version)
Full text under the publisher's open licence · 151 words
Related coverage
How Do Document Parsers Break? Auditing Structural Vulnerability in Document Intelligence
arXiv · Sep 17, 2026
5 Real-World Applications of Agentic AI in Enterprise Automation
KDnuggets · Sep 2, 2026
Article: Governing AI in the Cloud: A Practical Guide for Architects
InfoQ · Jun 15, 2026
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · Sep 18, 2026