Skip to content
aiqbench

Anthropic · model passport

Claude Sonnet 4

claude-sonnet-4
  • Cloud only
  • Proprietary licence
  • Released 2025-05-22(curated)
13/27axes covered
Overall · LMArena
1,402#147 of 396
Best percentile
93.8SWE-bench Verified (live board)
Cohort standing
53.5median 50.3

What the data says

Benchmarked on 13 of 27 tracked axes. Every figure below is the model's own published result on that board.

Strongest axes

  • 93.8 pct — SWE-bench Verified (live board) (76.8%)

Uneven: 69.9 percentile points between its best and worst axis — 93.8 pct on SWE-bench Verified (live board) versus 23.9 pct on Humanity's Last Exam.

  • SWE-bench Verified (live board)93.8 pct76.8%Verify →
  • Instruction following73.7 pct1,416Verify →
  • Coding73.1 pct1,474Verify →
  • Hard prompts67.9 pct1,433Verify →
  • Overall63.1 pct1,402Verify →
  • GPQA Diamond58.8 pct78.3%Verify →
  • AIME (math)52.5 pct71.1%Verify →
  • Vision Arena48.7 pct1,206Verify →
  • HHEM Hallucination (Vectara)43.8 pct89.7%Verify →
  • SimpleBench39.1 pct45.5%Verify →
  • ARC-AGI-238.2 pct5.9%Verify →
  • FrontierMath29 pct4.1%Verify →
  • Humanity's Last Exam23.9 pct7.8%Verify →
  • WebDev ArenaNot benchmarked
  • Factuality ArenaNot benchmarked
  • Search ArenaNot benchmarked
  • Document ArenaNot benchmarked
  • Agent ArenaNot benchmarked
  • SimpleQA VerifiedNot benchmarked
  • LiveBenchNot benchmarked
  • BFCL v4 (function calling)Not benchmarked
  • Terminal-Bench 4.0Not benchmarked
  • MMLU-Pro (Artificial Analysis)Not benchmarked
  • LiveCodeBench (Artificial Analysis)Not benchmarked
  • IFBench (Artificial Analysis)Not benchmarked
  • SWE-bench VerifiedNot benchmarked
  • Terminal-BenchNot benchmarked

Percentiles are the model's standing within each board's own population. Axes marked "not benchmarked" have no published result for this model — that is an absence of data, not a low score.

  • claude-sonnet-4-20250514-thinking-32k stands at 53.5 (raw mean 54.3) — measured on 13 of 27 axes.

    standingtemplateVerify →

  • claude-sonnet-4-20250514-thinking-32k stands at 53.5 versus a cohort median of 50.3 across 146 models released within 180 days — above the peer median.

    cohorttemplateVerify →

Every claim is a computed fact from the current board data, phrased by a low-temperature model call and rejected by a validation gate if any number is not in the source row. How the leaderboards work →