aiqbench analysis
This research introduces D2Snap, an algorithm that compresses DOM snapshots for web agents while preserving critical actionability features. By reducing token usage from excessive levels to under 17%, the method enables LLMs like GPT-4o to process HTML structures more efficiently than traditional GUI screenshot baselines. The study demonstrates that text-based grounding alone can achieve competitive success rates, offering a viable path forward for scalable autonomous browsing systems.
Beyond Pixels: Exploring DOM Downsampling for LLM-Based Web Agents
arXiv:2508.04412v4 Announce Type: replace Abstract: The advent of large language models (LLMs) has sparked an evolution of autonomous web browsing agents: given a web browsing task and serialised user interface (UI) state, an LLM is expected to suggest input actions that incrementally solve the given task. The central challenge lies in serialising UI state for LLMs. Web agents have increasingly relied on grounded graphical UI (GUI) snapshots - screenshots augmented with visual cues - favoured for their modest input token footprint. Document object model (DOM) snapshots, serialised as HTML, represent a compelling alternative that leverages previously demonstrated HTML interpretation capabilities of LLMs. Their excessive token footprint, however, has precluded reliable deployment with web agents to date. We propose D2Snap, an algorithm to downsample the DOM, premised on preserving actionability and actionability-discriminating features. We evaluate D2Snap-downsampled DOM snapshots using a snapshot-variant web agent (GPT-4o) on a dataset sampled from Online-Mind2Web. Whilst 42% of raw DOM snapshots exceed the model context window (128 x 10^3 tokens), all D2Snap-downsampled DOM snapshots of our reference configuration fit, at a mean context utilisation of 16.5%. Against the 67% success rate of a grounded GUI snapshot baseline, our configuration attains 73% (+5.8%pt; 95% CI -13.6 to +26.0%pt; McNemar, p = 0.47), excluding a deficit (one-sided 95%) beyond 11%pt. Image input moreover appears to add little to snapshot utility; grounding text alone attains 62% (-5.8%pt; McNemar, p = 0.37).
Abstract:The advent of large language models (LLMs) has sparked an evolution of autonomous web browsing agents: given a web browsing task and serialised user interface (UI) state, an LLM is expected to suggest input actions that incrementally solve the given task. The central challenge lies in serialising UI state for LLMs. Web agents have increasingly relied on grounded graphical UI (GUI) snapshots - screenshots augmented with visual cues - favoured for their modest input token footprint. Document object model (DOM) snapshots, serialised as HTML, represent a compelling alternative that leverages previously demonstrated HTML interpretation capabilities of LLMs. Their excessive token footprint, however, has precluded reliable deployment with web agents to date. We propose D2Snap, an algorithm to downsample the DOM, premised on preserving actionability and actionability-discriminating features. We evaluate D2Snap-downsampled DOM snapshots using a snapshot-variant web agent (GPT-4o) on a dataset sampled from Online-Mind2Web. Whilst 42% of raw DOM snapshots exceed the model context window (128 x 10^3 tokens), all D2Snap-downsampled DOM snapshots of our reference configuration fit, at a mean context utilisation of 16.5%. Against the 67% success rate of a grounded GUI snapshot baseline, our configuration attains 73% (+5.8%pt; 95% CI -13.6 to +26.0%pt; McNemar, p = 0.47), excluding a deficit (one-sided 95%) beyond 11%pt. Image input moreover appears to add little to snapshot utility; grounding text alone attains 62% (-5.8%pt; McNemar, p = 0.37).
Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC)
arXiv:2508.04412 [cs.AI]
(or arXiv:2508.04412v4 [cs.AI] for this version)
From: Thassilo M. Schiepanski [view email] [v1] Wed, 6 Aug 2025 12:56:54 UTC (2,708 KB) [v2] Fri, 31 Oct 2025 13:21:04 UTC (2,708 KB) [v3] Mon, 10 Aug 2026 13:40:08 UTC (2,571 KB) [v4] Mon, 31 Aug 2026 14:39:02 UTC (2,572 KB)
Full text under the publisher's open licence · 287 words
Related coverage
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
[AINews] Claude Opus 5.5, the new default model for AINews — and everybody cuts prices 40-50%
Latent Space · Sep 23, 2026
Why do models task game?
AI Alignment Forum · Aug 6, 2026
World Models and Yann LeCun’s Bet Against Large Language Models
Quantum Zeitgeist · Jul 17, 2026