ARC-AGI-2 (public eval)

Public set of abstract reasoning puzzles.

ModelsModels with a result.122
Top resultBest result on this benchmark.98.8%Claude Fable 5.2
Top-3 spreadPoints from first to third.0.8 pts
YearYear of release.—

Result and cost

122 of 122 results

Results

122 results
#ModelResultThe score from the source.CostUS dollars to run one task.
17GPT-6 Solmax effort90.0%$0.414
48Inkling37.8%$0.713
52Grok 4 (Thinking)21.1%$2.08
53GLM-5.220.8%$0.229
57GPT-5 Pro13.3%$8.01
58Kimi K2.512.1%$0.30
59GPT-5high effort9.6%$0.774
62o4-minihigh effort7.5%$0.882
65GLM-55.4%$0.29
65MiniMax M2.55.4%$0.19
76DeepSeek V3.23.9%$0.13
78Claude Sonnet 4.53.8%$0.131
80o3high effort2.9%$0.90
86Claude Sonnet 42.1%$0.131
91Claude Opus 41.3%$0.663
95Qwen3 235B 25070.8%$0.0045
99DeepSeek-R10.3%$0.08
99DeepSeek-R1-05280.3%$0.05
102Claude Haiku 4.50.0%$0.043
102o3-minihigh effort0.0%$0.607
102Grok 3 [Beta]0.0%$0.14
102GPT-4.50.0%$2.07
102GPT-4.10.0%$0.069
102GPT-4.1 mini0.0%$0.014
102Llama 4 Maverick0.0%$0.013
102Magistral Medium0.0%$0.106
102o1-mini0.0%$0.184
102Magistral Small0.0%$0.05
102GPT-4o0.0%$0.08
102Llama 4 Scout0.0%$0.006
102GPT-4.1 nano0.0%$0.0036
102GPT-4o mini0.0%$0.01
102Codex Mini (Latest)0.0%$0.227
102Gemini 1.5 Pro0.0%$0.04