Skip to content
aiqbench

Anthropic · model passport

Claude 3 5 Haiku

claude-3-5-haiku
  • Cloud only
  • Proprietary licence
  • Released 2024-11-04(curated)
12/27axes covered
Overall · LMArena
1,325#232 of 396
Best percentile
46.8Coding
Cohort standing
36.3median 40.4

What the data says

Benchmarked on 12 of 27 tracked axes. Every figure below is the model's own published result on that board.

Weakest axes

  • 7.2 pct — FrontierMath (0.3%)
  • 14.5 pct — AIME (math) (4.3%)

Uneven: 39.6 percentile points between its best and worst axis — 46.8 pct on Coding versus 7.2 pct on FrontierMath.

  • Coding46.8 pct1,386Verify →
  • IFBench (Artificial Analysis)46.6 pct42.8%Verify →
  • Instruction following44.4 pct1,316Verify →
  • Hard prompts44.4 pct1,346Verify →
  • Overall41.7 pct1,325Verify →
  • LiveCodeBench (Artificial Analysis)40.8 pct31.4%Verify →
  • LiveBench33.3 pct44.8%Verify →
  • MMLU-Pro (Artificial Analysis)29.6 pct63.4%Verify →
  • Vision Arena25.3 pct1,127Verify →
  • GPQA Diamond19.5 pct38.1%Verify →
  • AIME (math)14.5 pct4.3%Verify →
  • FrontierMath7.2 pct0.3%Verify →
  • WebDev ArenaNot benchmarked
  • Factuality ArenaNot benchmarked
  • Search ArenaNot benchmarked
  • Document ArenaNot benchmarked
  • Agent ArenaNot benchmarked
  • SimpleQA VerifiedNot benchmarked
  • ARC-AGI-2Not benchmarked
  • Humanity's Last ExamNot benchmarked
  • SimpleBenchNot benchmarked
  • BFCL v4 (function calling)Not benchmarked
  • SWE-bench Verified (live board)Not benchmarked
  • Terminal-Bench 4.0Not benchmarked
  • HHEM Hallucination (Vectara)Not benchmarked
  • SWE-bench VerifiedNot benchmarked
  • Terminal-BenchNot benchmarked

Percentiles are the model's standing within each board's own population. Axes marked "not benchmarked" have no published result for this model — that is an absence of data, not a low score.

  • claude-3-5-haiku-20241022 stands at 36.3 (raw mean 32.9) — measured on 12 of 27 axes.

    standingtemplateVerify →

  • claude-3-5-haiku-20241022 stands at 36.3 versus a cohort median of 40.4 across 140 models released within 180 days — below the peer median.

    cohorttemplateVerify →

Every claim is a computed fact from the current board data, phrased by a low-temperature model call and rejected by a validation gate if any number is not in the source row. How the leaderboards work →