Skip to content
aiqbench

Anthropic · model passport

Claude 3 Haiku

claude-3-haiku
  • Cloud only
  • Proprietary licence
  • Released 2024-03-14(curated)
11/27axes covered
Overall · LMArena
1,262#297 of 396
Best percentile
30.2IFBench (Artificial Analysis)
Cohort standing
27.5median 31.9

What the data says

Benchmarked on 11 of 27 tracked axes. Every figure below is the model's own published result on that board.

Weakest axes

  • 7 pct — AIME (math) (1.8%)
  • 7.3 pct — Vision Arena (1,000)

Uneven: 23.2 percentile points between its best and worst axis — 30.2 pct on IFBench (Artificial Analysis) versus 7 pct on AIME (math).

  • IFBench (Artificial Analysis)30.2 pct36.1%Verify →
  • SWE-bench Verified (live board)29.2 pct40.6%Verify →
  • Coding26.1 pct1,301Verify →
  • Instruction following26 pct1,245Verify →
  • Overall25.3 pct1,262Verify →
  • Hard prompts25.3 pct1,266Verify →
  • LiveBench21.3 pct35.4%Verify →
  • LiveCodeBench (Artificial Analysis)19.5 pct15.4%Verify →
  • GPQA Diamond17.2 pct36.3%Verify →
  • Vision Arena7.3 pct1,000Verify →
  • AIME (math)7 pct1.8%Verify →
  • WebDev ArenaNot benchmarked
  • Factuality ArenaNot benchmarked
  • Search ArenaNot benchmarked
  • Document ArenaNot benchmarked
  • Agent ArenaNot benchmarked
  • FrontierMathNot benchmarked
  • SimpleQA VerifiedNot benchmarked
  • ARC-AGI-2Not benchmarked
  • Humanity's Last ExamNot benchmarked
  • SimpleBenchNot benchmarked
  • BFCL v4 (function calling)Not benchmarked
  • Terminal-Bench 4.0Not benchmarked
  • MMLU-Pro (Artificial Analysis)Not benchmarked
  • HHEM Hallucination (Vectara)Not benchmarked
  • SWE-bench VerifiedNot benchmarked
  • Terminal-BenchNot benchmarked

Percentiles are the model's standing within each board's own population. Axes marked "not benchmarked" have no published result for this model — that is an absence of data, not a low score.

  • claude-3-haiku-20240307 stands at 27.5 (raw mean 21.3) — measured on 11 of 27 axes.

    standingtemplateVerify →

  • claude-3-haiku-20240307 stands at 27.5 versus a cohort median of 31.9 across 104 models released within 180 days — below the peer median.

    cohorttemplateVerify →

Every claim is a computed fact from the current board data, phrased by a low-temperature model call and rejected by a validation gate if any number is not in the source row. How the leaderboards work →