Skip to content
aiqbench

OpenAI · model passport

OpenAI: GPT-4o-mini

gpt-4o-mini
  • Cloud only
  • Proprietary licence
  • 128,000 context
  • Released 2024-07-18(curated)
14/27axes covered
Overall · LMArena
1,318#245 of 396
Best percentile
38.4Overall
Cohort standing
28.2median 36.6

What the data says

Benchmarked on 14 of 27 tracked axes. Every figure below is the model's own published result on that board.

Weakest axes

  • 1.1 pct — SimpleBench (10.7%)
  • 2.4 pct — SimpleQA Verified (8.3%)
  • 11.2 pct — ARC-AGI-2 (0.0%)

Uneven: 37.3 percentile points between its best and worst axis — 38.4 pct on Overall versus 1.1 pct on SimpleBench.

  • Overall38.4 pct1,318Verify →
  • Instruction following36.6 pct1,293Verify →
  • Coding35.5 pct1,348Verify →
  • Hard prompts34.1 pct1,311Verify →
  • LiveBench32.6 pct44.3%Verify →
  • MMLU-Pro (Artificial Analysis)31.1 pct64.8%Verify →
  • LiveCodeBench (Artificial Analysis)29 pct23.4%Verify →
  • Vision Arena20.7 pct1,097Verify →
  • AIME (math)20.5 pct6.9%Verify →
  • GPQA Diamond18.6 pct37.7%Verify →
  • IFBench (Artificial Analysis)17.6 pct31.0%Verify →
  • ARC-AGI-211.2 pct0.0%Verify →
  • SimpleQA Verified2.4 pct8.3%Verify →
  • SimpleBench1.1 pct10.7%Verify →
  • WebDev ArenaNot benchmarked
  • Factuality ArenaNot benchmarked
  • Search ArenaNot benchmarked
  • Document ArenaNot benchmarked
  • Agent ArenaNot benchmarked
  • FrontierMathNot benchmarked
  • Humanity's Last ExamNot benchmarked
  • BFCL v4 (function calling)Not benchmarked
  • SWE-bench Verified (live board)Not benchmarked
  • Terminal-Bench 4.0Not benchmarked
  • HHEM Hallucination (Vectara)Not benchmarked
  • SWE-bench VerifiedNot benchmarked
  • Terminal-BenchNot benchmarked

Percentiles are the model's standing within each board's own population. Axes marked "not benchmarked" have no published result for this model — that is an absence of data, not a low score.

How to run it

Use it hosted

API model IDopenai/gpt-4o-mini
Context length128,000 tokens
Max output16,384 tokens
Acceptstext, image, file
Input price$0.15 per 1M tokens
Cached input price$0.07 per 1M tokens
Output price$0.60 per 1M tokens

API parameters this endpoint supports

Frequency penaltyLogit biasLogprobsMax completion tokensMax tokensPredictionPresence penaltyResponse formatSeedStop sequencesStructured outputsTemperatureTool choiceTool callingTop logprobsTop-pWeb search

Example request

curl https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"Hello"}]}'

View on OpenRouter →

Identifiers, limits and prices are synced from the upstream source shown above. Any field we haven't synced is left out entirely rather than guessed.

  • gpt-4o-mini-2024-07-18 stands at 28.2 (raw mean 23.5) — measured on 14 of 27 axes.

    standingtemplateVerify →

  • gpt-4o-mini-2024-07-18 stands at 28.2 versus a cohort median of 36.6 across 131 models released within 180 days — below the peer median.

    cohorttemplateVerify →

Every claim is a computed fact from the current board data, phrased by a low-temperature model call and rejected by a validation gate if any number is not in the source row. How the leaderboards work →