aiqbench takeAI Research

Perplexity's new WANDR benchmark addresses a critical gap in evaluating research agents by shifting focus from single-answer accuracy to the ability to discover and verify large collections of evidence. This open harness challenges models to perform wide, deep searches across complex hierarchies, which is essential for real-world knowledge work like competitive mapping and due diligence. By validating independent paths through data trees rather than polished narratives built on incomplete research, WANDR sets a new standard for measuring the robustness of autonomous agents in handling extensive information gathering tasks.

Sources coveringMarkTechPost, Latent Space

What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

Perplexity AI Releases WANDR: An Open Benchmark Evaluating Research Agents That Must Search Wide And Deep

MarkTechPostJuly 19, 20265 min read

Perplexity's WANDR is an open benchmark and evaluation harness with 500 evidence-heavy tasks. It tests whether research agents can discover many qualifying entities and back each one with cited, re-verifiable evidence. Perplexity Search as Code leads at 0.363 soft F1 and 0.133 hard F1. The post Perplexity AI Releases WANDR: An Open Benchmark Evaluating Research Agents That Must Search Wide And Deep appeared first on MarkTechPost.

Research agents already handle real knowledge work today. Teams delegate competitive mapping, due diligence, and literature review to them. However, most benchmarks test a single answer, not large evidence-backed collections. Perplexity targets that gap with a new open benchmark. Perplexity released WANDR (Wide ANd Deep Research). It is an open benchmark and evaluation harness. It is built around 500 realistic, challenging data-collection tasks for knowledge work. WANDR is the wide sibling of Perplexity’s DRACO benchmark for deep research. DRACO asks whether an agent produces an accurate, complete, objective long-form report. WANDR…

Excerpt shown under the publisher's RSS terms · 91 words · full text and figures are on the publisher's site

Read at MarkTechPost ↗Ask about this article