DeepSeek · model passport
DeepSeek LLM 67b Chat
deepseek-llm-67b-chat- Open weights
- DeepSeek License licence
- Released 2023-11-29(curated)
- Overall · LMArena
- 1,185#333 of 396
- Best percentile
- 16.2Overall
- Cohort standing
- 24.1median 29.3
What the data says
Benchmarked on 6 of 27 tracked axes. Every figure below is the model's own published result on that board.
Weakest axes
- 4 pct — AIME (math) (0.8%)
- 4.5 pct — GPQA Diamond (24.6%)
- 13.1 pct — Hard prompts (1,176)
Uneven: 12.2 percentile points between its best and worst axis — 16.2 pct on Overall versus 4 pct on AIME (math).
- Overall16.2 pct1,185Verify →
- Instruction following15.2 pct1,158Verify →
- Coding14.3 pct1,218Verify →
- Hard prompts13.1 pct1,176Verify →
- GPQA Diamond4.5 pct24.6%Verify →
- AIME (math)4 pct0.8%Verify →
- WebDev ArenaNot benchmarked
- Vision ArenaNot benchmarked
- Factuality ArenaNot benchmarked
- Search ArenaNot benchmarked
- Document ArenaNot benchmarked
- Agent ArenaNot benchmarked
- FrontierMathNot benchmarked
- SimpleQA VerifiedNot benchmarked
- ARC-AGI-2Not benchmarked
- Humanity's Last ExamNot benchmarked
- SimpleBenchNot benchmarked
- LiveBenchNot benchmarked
- BFCL v4 (function calling)Not benchmarked
- SWE-bench Verified (live board)Not benchmarked
- Terminal-Bench 4.0Not benchmarked
- MMLU-Pro (Artificial Analysis)Not benchmarked
- LiveCodeBench (Artificial Analysis)Not benchmarked
- IFBench (Artificial Analysis)Not benchmarked
- HHEM Hallucination (Vectara)Not benchmarked
- SWE-bench VerifiedNot benchmarked
- Terminal-BenchNot benchmarked
Percentiles are the model's standing within each board's own population. Axes marked "not benchmarked" have no published result for this model — that is an absence of data, not a low score.
How to run it
Run it yourself
Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "deepseek-ai/deepseek-llm-67b-chat"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)Ollama
We don't sync Ollama tags, so we won't print a command we can't verify — a wrong one would pull the wrong model. Search the Ollama library →
Unsloth fine-tuning recipes are model-specific and we don't derive them here. Unsloth docs →
Identifiers, limits and prices are synced from the upstream source shown above. Any field we haven't synced is left out entirely rather than guessed.
deepseek-llm-67b-chat stands at 24.1 (raw mean 11.2) — measured on 6 of 27 axes.
deepseek-llm-67b-chat stands at 24.1 versus a cohort median of 29.3 across 81 models released within 180 days — below the peer median.
deepseek-llm-67b-chat scores in the 4th percentile on AIME (math).
deepseek-llm-67b-chat scores at percentile 4.5 on GPQA Diamond.
deepseek-llm-67b-chat ranks between #347 and #1,010 across 5 built-in weightings — best under "Coding-focused", worst under "All sources".
Every claim is a computed fact from the current board data, phrased by a low-temperature model call and rejected by a validation gate if any number is not in the source row. How the leaderboards work →