Skip to content
aiqbench takeAI Products

aiqbench analysis

The guide's main value is its warnings about silent failures. Two of the routes return normal-looking responses with no error: a scanned PDF sent through the free parser can yield zero rows, and a long table can be truncated at the output token limit. Readers building LLM extraction pipelines should add checks for empty results and `finish_reason` "length" rather than trusting a successful response. The reported accuracy comes from five self-generated test PDFs, so results will vary with scan quality and text layers.
Sources coveringOpenRouter, Databricks, Akka Blog, Hugging Face

How to Extract Tables from a PDF with an LLM

OpenRouter13 min read

Send a PDF to a model that reads files, with a strict JSON schema for one row, and you get typed rows back, even from scans and tables that run across pages. We tested five routes on five PDFs and found two failures that return a normal response with no error, and the checks that catch them.

To extract tables from a PDF with an LLM, send the PDF to a model that can read files, together with a strict JSON schema that describes one row of the table. The model returns typed rows. Quantities come back as integers, a loss written as (3,878) comes back as -3878, and a table that runs across two pages comes back as one list. We tested this on five PDFs, two of them scans. Every model that read the PDF directly got all 644 cells right. The failures in our tests came from the route the PDF took to reach the model, and two of them returned a normal response with no error. A strict JSON schema for one row turns a PDF table into typed rows, and field descriptions handle cleanup such as negatives in parentheses and footnote markers. Models that read PDFs handled both digital files and scans in our tests. A model that can’t read PDFs needs the mistral-ocr parser for scans. A scan sent through the free cloudflare-ai parser returns zero rows with no error, so reject empty results. Long tables can stop at the output token limit, so reject any response whose finish_reason is “length”.…

Excerpt shown under the publisher's RSS terms · 200 words · full text and figures are on the publisher's site

Related coverage