Skip to content
aiqbench

OpenAI · model passport

OpenAI: GPT-4o (2024-11-20)

gpt-4o
  • Cloud only
  • Proprietary licence
  • 128,000 context
  • Released 2024-05-13(curated)
19/27axes covered
Overall · LMArena
1,346#207 of 396
Best percentile
67.8MMLU-Pro (Artificial Analysis)
Cohort standing
33.8median 34.5

What the data says

Benchmarked on 19 of 27 tracked axes. Every figure below is the model's own published result on that board.

Weakest axes

  • 2.2 pct — Humanity's Last Exam (2.7%)
  • 3.1 pct — SWE-bench Verified (31.0%)
  • 3.4 pct — SimpleBench (17.8%)

Uneven: 65.6 percentile points between its best and worst axis — 67.8 pct on MMLU-Pro (Artificial Analysis) versus 2.2 pct on Humanity's Last Exam.

  • MMLU-Pro (Artificial Analysis)67.8 pct80.3%Verify →
  • LiveBench55.7 pct56.5%Verify →
  • LiveCodeBench (Artificial Analysis)51.9 pct42.5%Verify →
  • HHEM Hallucination (Vectara)48.6 pct90.4%Verify →
  • Overall48 pct1,346Verify →
  • Instruction following46.7 pct1,325Verify →
  • Hard prompts42.9 pct1,338Verify →
  • Coding42.2 pct1,369Verify →
  • Vision Arena37.3 pct1,162Verify →
  • GPQA Diamond34.4 pct49.2%Verify →
  • IFBench (Artificial Analysis)29.9 pct36.0%Verify →
  • SimpleQA Verified22 pct26.0%Verify →
  • SWE-bench Verified (live board)21.5 pct38.8%Verify →
  • AIME (math)18.5 pct6.4%Verify →
  • ARC-AGI-211.2 pct0.0%Verify →
  • FrontierMath7.2 pct0.3%Verify →
  • SimpleBench3.4 pct17.8%Verify →
  • SWE-bench Verified3.1 pct31.0%Verify →
  • Humanity's Last Exam2.2 pct2.7%Verify →
  • WebDev ArenaNot benchmarked
  • Factuality ArenaNot benchmarked
  • Search ArenaNot benchmarked
  • Document ArenaNot benchmarked
  • Agent ArenaNot benchmarked
  • BFCL v4 (function calling)Not benchmarked
  • Terminal-Bench 4.0Not benchmarked
  • Terminal-BenchNot benchmarked

Percentiles are the model's standing within each board's own population. Axes marked "not benchmarked" have no published result for this model — that is an absence of data, not a low score.

How to run it

Use it hosted

API model IDopenai/gpt-4o-2024-11-20
Context length128,000 tokens
Max output16,384 tokens
Acceptstext, image, file
Input price$2.50 per 1M tokens
Cached input price$1.25 per 1M tokens
Output price$10 per 1M tokens

API parameters this endpoint supports

Frequency penaltyLogit biasLogprobsMax tokensPredictionPresence penaltyResponse formatSeedStop sequencesStructured outputsTemperatureTool choiceTool callingTop logprobsTop-pWeb search

Example request

curl https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-4o-2024-11-20","messages":[{"role":"user","content":"Hello"}]}'

View on OpenRouter →

Identifiers, limits and prices are synced from the upstream source shown above. Any field we haven't synced is left out entirely rather than guessed.

  • gpt-4o-2024-05-13 stands at 33.8 (raw mean 31.3) — measured on 19 of 27 axes.

    standingtemplateVerify →

  • gpt-4o-2024-05-13 stands at 33.8 versus a cohort median of 34.5 across 118 models released within 180 days — below the peer median.

    cohorttemplateVerify →

Every claim is a computed fact from the current board data, phrased by a low-temperature model call and rejected by a validation gate if any number is not in the source row. How the leaderboards work →