Anthropic · model passport
Claude 3 5 Sonnet
claude-3-5-sonnet- Cloud only
- Proprietary licence
- Released 2024-06-20(curated)
- Overall · LMArena
- 1,374#179 of 396
- Best percentile
- 66.2SWE-bench Verified (live board)
- Cohort standing
- 45.5median 36.2
What the data says
Benchmarked on 14 of 27 tracked axes. Every figure below is the model's own published result on that board.
Weakest axes
- 6.5 pct — Humanity's Last Exam (4.1%)
Uneven: 59.7 percentile points between its best and worst axis — 66.2 pct on SWE-bench Verified (live board) versus 6.5 pct on Humanity's Last Exam.
- SWE-bench Verified (live board)66.2 pct62.8%Verify →
- LiveBench60.8 pct59.9%Verify →
- Coding59.6 pct1,436Verify →
- Instruction following58.1 pct1,374Verify →
- MMLU-Pro (Artificial Analysis)56.6 pct77.2%Verify →
- Hard prompts56.6 pct1,399Verify →
- Overall55.1 pct1,374Verify →
- LiveCodeBench (Artificial Analysis)48.1 pct38.1%Verify →
- GPQA Diamond37.6 pct55.3%Verify →
- SimpleBench36.8 pct41.4%Verify →
- Vision Arena36 pct1,160Verify →
- AIME (math)24.5 pct8.5%Verify →
- FrontierMath21.7 pct2.1%Verify →
- Humanity's Last Exam6.5 pct4.1%Verify →
- WebDev ArenaNot benchmarked
- Factuality ArenaNot benchmarked
- Search ArenaNot benchmarked
- Document ArenaNot benchmarked
- Agent ArenaNot benchmarked
- SimpleQA VerifiedNot benchmarked
- ARC-AGI-2Not benchmarked
- BFCL v4 (function calling)Not benchmarked
- Terminal-Bench 4.0Not benchmarked
- IFBench (Artificial Analysis)Not benchmarked
- HHEM Hallucination (Vectara)Not benchmarked
- SWE-bench VerifiedNot benchmarked
- Terminal-BenchNot benchmarked
Percentiles are the model's standing within each board's own population. Axes marked "not benchmarked" have no published result for this model — that is an absence of data, not a low score.
claude-3-5-sonnet-20241022 stands at 45.5 (raw mean 44.6) — measured on 14 of 27 axes.
claude-3-5-sonnet-20241022 stands at 45.5 versus a cohort median of 36.2 across 125 models released within 180 days — above the peer median.
On GPQA Diamond, claude-3-5-sonnet-20241022 raised the running best score by 6.1 percentage points to 55.3% on 2024-06-20.
On OTIS Mock AIME, claude-3-5-sonnet-20241022 raised the running best score by 1.7 percentage points to 8.5% on 2024-06-20.
claude-3-5-sonnet-20241022 scores at percentile 6.5 on Humanity's Last Exam.
claude-3-5-sonnet-20241022 ranks between #126 and #663 across 5 built-in weightings — best under "Coding-focused", worst under "All sources".
Every claim is a computed fact from the current board data, phrased by a low-temperature model call and rejected by a validation gate if any number is not in the source row. How the leaderboards work →