analiza aiqbench
This research presents SmolDocling, a compact vision-language model that performs end-to-end key-value extraction from document images without relying on traditional OCR preprocessing. By jointly handling identification, localization, and association in a single pass, the approach significantly reduces error propagation while offering substantial efficiency gains over larger zero-shot baselines. The work advances practical document processing by introducing specialized tagging for complex relationships and a layout-aware evaluation framework that verifies spatial bounding boxes alongside text matching.
Identify, Locate, Link: End-to-End Key-Value Extraction from Document Images
arXiv:2608.20868v1 Announce Type: cross Abstract: Document processing pipelines traditionally cascade optical character recognition (OCR) engines with downstream models for structured information extraction, leading to multi-stage error propagation. We fine-tune SmolDocling, a compact 256M-parameter vision-language model (VLM), to perform end-to-end key-value extraction directly from document images, jointly solving identification, localization, and association in a single pass without OCR preprocessing. We extend DocTags with specialized key, value, region, and link tags, enabling many-to-many relationships in a unified output sequence. To address data limitations, we design an augmentation pipeline combining synthetic form filling and graph-based crops that preserve complete key-value subgraphs. We further introduce a layout-aware evaluation framework extending text matching with spatial bounding box verification. On FUNSD, XFUND, and a large-scale private dataset, our model outperforms larger zero-shot VLM baselines under layout-aware evaluation, while being 27 times smaller than Qwen2.5-VL (7B) and over 5 times faster at inference. The model weights will be released publicly after publication.
Authors:A. Said Gurbuz (1 and 2), Ahmed Nassar (1), Christoph Auer (1), Maksym Lysak (1), Lucas Morin (1), Matteo Omenetti (1), Tim Strohmeyer (1), Panagiotis Vagenas (1), Nikolaos Livathinos (1), Michele Dolfi (1), Peter Staar (1) ((1) IBM Research Zurich, (2) ETH Zurich)
Abstract:Document processing pipelines traditionally cascade optical character recognition (OCR) engines with downstream models for structured information extraction, leading to multi-stage error propagation. We fine-tune SmolDocling, a compact 256M-parameter vision-language model (VLM), to perform end-to-end key-value extraction directly from document images, jointly solving identification, localization, and association in a single pass without OCR preprocessing. We extend DocTags with specialized key, value, region, and link tags, enabling many-to-many relationships in a unified output sequence. To address data limitations, we design an augmentation pipeline combining synthetic form filling and graph-based crops that preserve complete key-value subgraphs. We further introduce a layout-aware evaluation framework extending text matching with spatial bounding box verification. On FUNSD, XFUND, and a large-scale private dataset, our model outperforms larger zero-shot VLM baselines under layout-aware evaluation, while being 27 times smaller than Qwen2.5-VL (7B) and over 5 times faster at inference. The model weights will be released publicly after publication.
Accepted at ICDAR 2026. 17 pages, 6 figures, 7 tables
Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
arXiv:2608.20868 [cs.CV]
(or arXiv:2608.20868v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.20868 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 228 cuvinte
Subiecte conexe
Built Technologies builds an AI-powered document intelligence solution on AWS to power agents across real estate finance
AWS · 15 iul. 2026
From PDFs to insights: Architecting an intelligent document processing pipeline with AWS generative AI services
AWS · 12 iun. 2026
[AINews] not much happened today
Latent Space · 18 sept. 2026
Modernizing and scaling support operations with generative AI on AWS
AWS · 2 sept. 2026