analiza aiqbench
AnnoBench introduces a specialized evaluation framework for automating visualization annotations using Vision-Language Models (VLMs) as judges. By rigorously testing models against visual, semantic, and stylistic constraints found in professional data journalism, this benchmark addresses a critical gap where prior studies failed to adequately assess annotation quality across diverse formats and prompt specifications. This work provides an essential foundation for improving the reliability of AI-driven tools that generate chart descriptions and annotations within human-computer interaction pipelines.
AnnoBench: A Benchmark for Visualization Annotation Generation
arXiv:2607.25911v1 Announce Type: cross Abstract: Annotation is among the most demanding visualization tasks to automate, as it simultaneously requires correctly navigating visual, semantic, and stylistic constraints. Failure to meet any of these conditions severely undermines the utility of an annotation, rendering it challenging to read, inaccurate, or visually discordant. Despite a growing body of annotation tools and automations, no existing benchmark or evaluation framework tests whether these conditions are met because of their scope and annotation not being the focus of their studies. We introduce AnnoBench, a benchmark for visualization annotation that materializes the inherent challenges of this domain in a structured and testable manner. AnnoBench pairs visualizations from professional data journalism and visualization galleries with annotation tasks, spanning four representation formats, five chart description conditions, and two prompt specification levels. The benchmark is executed via VLM-as-a-judge, using models aligned with manual human assessment. We evaluate the benchmark via four one-factor-at-a-time experiments, exploring the effects of input representation, semantic context, and prompt specificity, and model selection on annotation quality. This work provides a foundation for advancing annotation automation, tooling, and visualization-generation pipelines.
Abstract:Annotation is among the most demanding visualization tasks to automate, as it simultaneously requires correctly navigating visual, semantic, and stylistic constraints. Failure to meet any of these conditions severely undermines the utility of an annotation, rendering it challenging to read, inaccurate, or visually discordant. Despite a growing body of annotation tools and automations, no existing benchmark or evaluation framework tests whether these conditions are met because of their scope and annotation not being the focus of their studies. We introduce AnnoBench, a benchmark for visualization annotation that materializes the inherent challenges of this domain in a structured and testable manner. AnnoBench pairs visualizations from professional data journalism and visualization galleries with annotation tasks, spanning four representation formats, five chart description conditions, and two prompt specification levels. The benchmark is executed via VLM-as-a-judge, using models aligned with manual human assessment. We evaluate the benchmark via four one-factor-at-a-time experiments, exploring the effects of input representation, semantic context, and prompt specificity, and model selection on annotation quality. This work provides a foundation for advancing annotation automation, tooling, and visualization-generation pipelines.
Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI)
arXiv:2607.25911 [cs.HC]
(or arXiv:2607.25911v1 [cs.HC] for this version)
https://doi.org/10.48550/arXiv.2607.25911 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 274 cuvinte
Subiecte conexe
AI-generated Images Challenge Visual Trust in High-risk Scenarios
arXiv · 28 iul. 2026
VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing
arXiv · 2 sept. 2026
VISTA: An End-to-End Benchmark for Visual Spec-to-Web-App Coding Agents
arXiv · 12 iun. 2026
TaoD2C-Bench: Benchmarking MLLMs for Industrial UI Code Generation Beyond Visual Fidelity
arXiv · 8 oct. 2026