GPT-4o mini vs o1-mini

Change models
o1-miniOpenAI
IndexAll categories, out of 100.
26.2 ±4.3
33.5 ±8.2 best
GapIndex points below the top.
−7.3 · no clear gap
Top
CoverageIndex weight that has results.
95% best
65%
SpeedOutput tokens per second.
56/s
—
EloLMArena rating and rank.
1286
1317 best
ReleasedMonth of release.
2024-07
2024-09
Input / 1MUS dollars per 1M input tokens.
$0.15 best
$1.10
Output / 1MUS dollars per 1M output tokens.
$0.60 best
$4.40
1K chats2K tokens in, 500 out, each.
$0.60 best
$4.40
1K documents20K tokens in, 1K out, each.
$3.60 best
$26.40
1K agent tasks200K tokens in, 20K out, each.
$42.00 best
$308.00
ContextMaximum tokens in one request.
128K
128K
AgenticMulti-step tasks with tools.
34.0
—
CodingCode writing and repair.
19.5
32.7 best
ReasoningLogic problems and puzzles.
41.1 best
36.5
MultimodalTasks with images and text.
19.0
—
KnowledgeFacts and expert knowledge.
25.7
28.2 best
MultilingualTasks in many languages.
—
33.4
InstructionTasks with strict prompt rules.
18.0
—
MathMath problems.
22.0
37.2 best

Benchmarks

10 shared
Only GPT-4o mini: 17Only o1-mini: 9
GPT-4o mini
o1-mini
4.2% best194 / 222
3.6%210 / 222
37.7%179 / 217
62.4% best128 / 217
6.9%163 / 200
46.9% best125 / 200
42.6%185 / 199
60.3% best156 / 199
0.0%120 / 131
0.8% best113 / 131
0.0%102 / 122
0.0%102 / 122
MedQAHealthcare
72.4%81 / 95
90.2% best47 / 95
52.6%46 / 88
89.2% best16 / 88
82.0%23 / 34
85.2% best20 / 34
87.2%12 / 22
92.4% best3 / 22