Skip to content
aiqbench

Meta · model passport

Meta: Llama 4 Scout

llama-4-scout-17b-16e-instruct
  • Open weights
  • Llama licence
  • 108.6B params
  • 1,310,720 context
  • Released 2025-04-05(curated)
11/27axes covered
Overall · LMArena
1,322#240 of 396
Best percentile
64.8HHEM Hallucination (Vectara)
Cohort standing
36.4median 48.1

What the data says

Benchmarked on 11 of 27 tracked axes. Every figure below is the model's own published result on that board.

Weakest axes

  • 2.9 pct — FrontierMath (0.0%)
  • 11.2 pct — ARC-AGI-2 (0.0%)

Uneven: 61.9 percentile points between its best and worst axis — 64.8 pct on HHEM Hallucination (Vectara) versus 2.9 pct on FrontierMath.

  • HHEM Hallucination (Vectara)64.8 pct92.3%Verify →
  • Hard prompts40.4 pct1,329Verify →
  • Coding40.2 pct1,363Verify →
  • Overall39.6 pct1,322Verify →
  • Instruction following38.1 pct1,299Verify →
  • BFCL v4 (function calling)Dormant board36.6 pct28.1%Verify →
  • GPQA Diamond36.2 pct51.8%Verify →
  • Vision Arena26.7 pct1,127Verify →
  • AIME (math)22.5 pct7.8%Verify →
  • ARC-AGI-211.2 pct0.0%Verify →
  • FrontierMath2.9 pct0.0%Verify →
  • WebDev ArenaNot benchmarked
  • Factuality ArenaNot benchmarked
  • Search ArenaNot benchmarked
  • Document ArenaNot benchmarked
  • Agent ArenaNot benchmarked
  • SimpleQA VerifiedNot benchmarked
  • Humanity's Last ExamNot benchmarked
  • SimpleBenchNot benchmarked
  • LiveBenchNot benchmarked
  • SWE-bench Verified (live board)Not benchmarked
  • Terminal-Bench 4.0Not benchmarked
  • MMLU-Pro (Artificial Analysis)Not benchmarked
  • LiveCodeBench (Artificial Analysis)Not benchmarked
  • IFBench (Artificial Analysis)Not benchmarked
  • SWE-bench VerifiedNot benchmarked
  • Terminal-BenchNot benchmarked

Percentiles are the model's standing within each board's own population. Axes marked "not benchmarked" have no published result for this model — that is an absence of data, not a low score.

How to run it

Run it yourself

Repositorymeta-llama/Llama-4-Scout-17B-16E-Instruct
Parameters108.6B
ArchitectureLlama4ForConditionalGeneration
LicenceLlama
AccessGated — you must accept the repo's terms before downloading

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "meta-llama/Llama-4-Scout-17B-16E-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

Ollama

We don't sync Ollama tags, so we won't print a command we can't verify — a wrong one would pull the wrong model. Search the Ollama library →

Unsloth fine-tuning recipes are model-specific and we don't derive them here. Unsloth docs →

Use it hosted

API model IDmeta-llama/llama-4-scout
Context length1,310,720 tokens
Max output16,384 tokens
Acceptstext, image
Input price$0.10 per 1M tokens
Output price$0.30 per 1M tokens

API parameters this endpoint supports

Frequency penaltyLogit biasMax tokensMin-pPresence penaltyRepetition penaltyResponse formatSeedStop sequencesStructured outputsTemperatureTool choiceTool callingTop-kTop-p

Example request

curl https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"meta-llama/llama-4-scout","messages":[{"role":"user","content":"Hello"}]}'

View on OpenRouter →

Identifiers, limits and prices are synced from the upstream source shown above. Any field we haven't synced is left out entirely rather than guessed.

  • llama-4-scout-17b-16e-instruct stands at 36.4 (raw mean 32.7) — measured on 11 of 27 axes.

    standingtemplateVerify →

  • llama-4-scout-17b-16e-instruct stands at 36.4 versus a cohort median of 48.1 across 151 models released within 180 days — below the peer median.

    cohorttemplateVerify →

Every claim is a computed fact from the current board data, phrased by a low-temperature model call and rejected by a validation gate if any number is not in the source row. How the leaderboards work →