Skip to content
aiqbench

OpenAI · model passport

o1 Preview

o1-preview
  • Cloud only
  • Proprietary licence
  • Released 2024-09-12(curated)
10/27axes covered
Overall · LMArena
1,389#164 of 396
Best percentile
89.9MMLU-Pro (Artificial Analysis)
Cohort standing
55.4median 37.9

What the data says

Benchmarked on 10 of 27 tracked axes. Every figure below is the model's own published result on that board.

Strongest axes

  • 89.9 pct — MMLU-Pro (Artificial Analysis) (84.8%)

Uneven: 56.4 percentile points between its best and worst axis — 89.9 pct on MMLU-Pro (Artificial Analysis) versus 33.5 pct on AIME (math).

  • MMLU-Pro (Artificial Analysis)89.9 pct84.8%Verify →
  • LiveBench74.9 pct66.0%Verify →
  • SWE-bench Verified (live board)70.8 pct64.6%Verify →
  • Instruction following59.8 pct1,381Verify →
  • Overall58.8 pct1,389Verify →
  • Hard prompts56.1 pct1,396Verify →
  • Coding53.7 pct1,415Verify →
  • SimpleBench37.9 pct41.7%Verify →
  • GPQA Diamond34.8 pct50.3%Verify →
  • AIME (math)33.5 pct31.1%Verify →
  • WebDev ArenaNot benchmarked
  • Vision ArenaNot benchmarked
  • Factuality ArenaNot benchmarked
  • Search ArenaNot benchmarked
  • Document ArenaNot benchmarked
  • Agent ArenaNot benchmarked
  • FrontierMathNot benchmarked
  • SimpleQA VerifiedNot benchmarked
  • ARC-AGI-2Not benchmarked
  • Humanity's Last ExamNot benchmarked
  • BFCL v4 (function calling)Not benchmarked
  • Terminal-Bench 4.0Not benchmarked
  • LiveCodeBench (Artificial Analysis)Not benchmarked
  • IFBench (Artificial Analysis)Not benchmarked
  • HHEM Hallucination (Vectara)Not benchmarked
  • SWE-bench VerifiedNot benchmarked
  • Terminal-BenchNot benchmarked

Percentiles are the model's standing within each board's own population. Axes marked "not benchmarked" have no published result for this model — that is an absence of data, not a low score.

  • o1-preview stands at 55.4 (raw mean 57) — measured on 10 of 27 axes.

    standingtemplateVerify →

  • o1-preview stands at 55.4 versus a cohort median of 37.9 across 129 models released within 180 days — above the peer median.

    cohorttemplateVerify →

Every claim is a computed fact from the current board data, phrased by a low-temperature model call and rejected by a validation gate if any number is not in the source row. How the leaderboards work →