Meta · model passport
Llama 3 1 405b Instruct
llama-3-1-405b-instruct- Open weights
- Llama 3.1 Community licence
- 405.9B params
- Released 2024-07-16(HF repo created)
- Overall · LMArena
- 1,335#213 of 371
- Best percentile
- 42.9Overall
- Cohort standing
- 39.2median 35.9
What the data says
Benchmarked on 7 of 22 tracked axes. Every figure below is the model's own published result on that board.
Weakest axes
- 14 pct — SimpleBench (23.0%)
Uneven: 28.9 percentile points between its best and worst axis — 42.9 pct on Overall versus 14 pct on SimpleBench.
- Overall42.9 pct1,335Verify →
- Coding42.9 pct1,376Verify →
- Hard prompts42.6 pct1,342Verify →
- Instruction following41.8 pct1,315Verify →
- GPQA Diamond33.9 pct50.9%Verify →
- AIME (math)23.9 pct9.7%Verify →
- SimpleBench14 pct23.0%Verify →
- WebDev ArenaNot benchmarked
- Vision ArenaNot benchmarked
- Factuality ArenaNot benchmarked
- Search ArenaNot benchmarked
- Document ArenaNot benchmarked
- Agent ArenaNot benchmarked
- FrontierMathNot benchmarked
- SimpleQA VerifiedNot benchmarked
- ARC-AGI-2Not benchmarked
- Humanity's Last ExamNot benchmarked
- LiveBenchNot benchmarked
- BFCL v4 (function calling)Not benchmarked
- SWE-bench Verified (live board)Not benchmarked
- Terminal-Bench 4.0Not benchmarked
- HHEM Hallucination (Vectara)Not benchmarked
Percentiles are the model's standing within each board's own population. Axes marked "not benchmarked" have no published result for this model — that is an absence of data, not a low score.
How to run it
Run it yourself
Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "meta-llama/Llama-3.1-405B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)Ollama
We don't sync Ollama tags, so we won't print a command we can't verify — a wrong one would pull the wrong model. Search the Ollama library →
Unsloth fine-tuning recipes are model-specific and we don't derive them here. Unsloth docs →
Context, limits and prices come from the last catalogue sync; the API id, the request and the link come from the provider's own documentation. Any field we haven't synced is left out entirely rather than guessed.
Llama 3.1 405B Instruct (BF16) from meta has a composite score of 39.2 on the shrunk benchmark, covering 7 out of 22 axes with a raw composite score of 34.6 and is not thin coverage.
standingVerify →Llama 3.1 405B Instruct (BF16) from meta has a standing of 39.2, which is above the cohort median standing of 35.9 across 111 models within an 180-day window.
cohortVerify →
In SimpleBench, Llama 3.1 405B Instruct (BF16) from meta scores at percentile 14.
weaknessVerify →
For Llama 3.1 405B Instruct (BF16) from meta, the sensitivity spread is 415 between ranks 208 and 623, with best performance under Conversation-first and worst under All sources across 5 templates.
rank swingVerify →
Every claim is a computed fact from the current board data, phrased by a low-temperature model call and rejected by a validation gate if any number is not in the source row. How the leaderboards work →