Skip to content
aiqbench

Anthropic · model passport

Claude 3 Opus

claude-3-opus
  • Cloud only
  • Proprietary licence
  • Released 2024-03-04(curated)
13/27axes covered
Overall · LMArena
1,322#237 of 396
Best percentile
45.7LiveBench
Cohort standing
32.8median 31.5

What the data says

Benchmarked on 13 of 27 tracked axes. Every figure below is the model's own published result on that board.

Weakest axes

  • 7.3 pct — SimpleQA Verified (12.6%)
  • 9.2 pct — SWE-bench Verified (live board) (7.0%)
  • 14.9 pct — SimpleBench (23.5%)

Uneven: 38.4 percentile points between its best and worst axis — 45.7 pct on LiveBench versus 7.3 pct on SimpleQA Verified.

  • LiveBench45.7 pct50.6%Verify →
  • Instruction following43.2 pct1,314Verify →
  • Hard prompts40.7 pct1,330Verify →
  • Overall40.4 pct1,322Verify →
  • MMLU-Pro (Artificial Analysis)40.1 pct69.6%Verify →
  • Coding37.6 pct1,355Verify →
  • LiveCodeBench (Artificial Analysis)35.1 pct27.9%Verify →
  • GPQA Diamond29 pct47.2%Verify →
  • Vision Arena16 pct1,061Verify →
  • AIME (math)16 pct4.7%Verify →
  • SimpleBench14.9 pct23.5%Verify →
  • SWE-bench Verified (live board)9.2 pct7.0%Verify →
  • SimpleQA Verified7.3 pct12.6%Verify →
  • WebDev ArenaNot benchmarked
  • Factuality ArenaNot benchmarked
  • Search ArenaNot benchmarked
  • Document ArenaNot benchmarked
  • Agent ArenaNot benchmarked
  • FrontierMathNot benchmarked
  • ARC-AGI-2Not benchmarked
  • Humanity's Last ExamNot benchmarked
  • BFCL v4 (function calling)Not benchmarked
  • Terminal-Bench 4.0Not benchmarked
  • IFBench (Artificial Analysis)Not benchmarked
  • HHEM Hallucination (Vectara)Not benchmarked
  • SWE-bench VerifiedNot benchmarked
  • Terminal-BenchNot benchmarked

Percentiles are the model's standing within each board's own population. Axes marked "not benchmarked" have no published result for this model — that is an absence of data, not a low score.

  • claude-3-opus-20240229 stands at 32.8 (raw mean 28.9) — measured on 13 of 27 axes.

    standingtemplateVerify →

  • claude-3-opus-20240229 stands at 32.8 versus a cohort median of 31.5 across 102 models released within 180 days — above the peer median.

    cohorttemplateVerify →

Every claim is a computed fact from the current board data, phrased by a low-temperature model call and rejected by a validation gate if any number is not in the source row. How the leaderboards work →