Vals AI

MMLU Pro

Academic multiple-choice benchmark covering 14 subjects including STEM, humanities, and social sciences.

In index
Knowledge
Best in knowledge
Source ↗

Models134

Top result92.4%Claude Fable 5.1

Top-3 spread0.9 pts

Updated1 Sep 2026

Result and cost

133 of 135 results

Results

135 results

#ModelResultCostTime

01Claude Fable 5.192.4% ±0.3$0.09821 s

02Claude Opus 591.6% ±0.3$0.0289.6 s

03Claude Fable 591.5% ±0.3$0.08225 s

04Gemini 3.1 Pro Preview91.0% ±0.3$0.02824 s

05Gemini 3.8 Flash90.2% ±0.3$0.0227.8 s

06Gemini 3.7 Flash90.1% ±0.3$0.0114.2 s

06Gemini 3 Pro90.1% ±0.3$0.03724 s

08Claude Opus 4.789.9% ±0.3$0.0623 min

09Claude Opus 4.889.6% ±0.3$0.06123 s

10Gemini 3.5 Flash89.5% ±0.3$0.02526 s

11Grok 4.689.4% ±0.3$0.01555 s

12Qwen3.7 Max89.3% ±0.3$0.01729 s

12Gemini 3.6 Flash89.3% ±0.3$0.0137.3 s

14Grok 4.589.2% ±0.3$0.0156 min

15Claude Opus 4.6 (Adaptive)89.1% ±0.5$0.05249 s

15GPT-5.6 Sol89.1% ±0.3$0.03216 s

17Muse Spark 1.188.7% ±0.3$0.01121 s

18Qwen3.8 Max88.6% ±0.3$0.01964 s

18Gemini 3 Flash Preview88.6% ±0.3$0.01936 s

20Muse Spark 1.288.3% ±0.3$0.01144 s

21GPT-5.588.1% ±0.3$0.02842 s

22Kimi K388.0% ±0.3$0.02139 s

23Claude Opus 4.1 Thinking87.9% ±0.3$0.12428 s

24Qwen3.6 Plus87.7% ±0.3$0.008846 s

25Kimi K2.687.6% ±0.3$0.0183 min

26Claude Sonnet 587.5% ±0.4$0.04325 s

26GPT-5.487.5% ±0.4$0.05129 s

28Claude Sonnet 4.5 Thinking87.4% ±0.4$0.0438 s

29Claude Sonnet 4.687.3% ±0.4$0.06655 s

29Muse Spark87.3% ±0.3$0.001746 s

29Claude Opus 4.5 Thinking87.3% ±0.4$0.21539 s

32DeepSeek V4 Pro 042387.2% ±0.3$0.02710 min

32Claude Opus 4.187.2% ±0.3$0.007118 s

32Qwen3.5 Plus Thinking87.2% ±0.3$0.008977 s

35MiniMax M2.187.0% ±0.3$0.003241 s

35DeepSeek V4 Pro 081387.0% ±0.3$0.01360 s

37GLM-5.186.9% ±0.3$0.01269 s

38GLM-5.386.8% ±0.3$0.01762 s

39GLM-5.286.7% ±0.3$0.01452 s

39GPT-5.6 Terra86.7% ±0.3$0.0088.2 s

41GPT-586.5% ±0.3$0.02437 s

42GPT-5.186.4% ±0.3$0.01923 s

43Inkling86.3% ±0.3$0.0473 s

43Grok 4.20 (Reasoning)86.3% ±0.3$0.01312 s

45Gemini 3.1 Flash Lite Preview86.2% ±0.3$0.001115 s

45GPT-5.286.2% ±0.3$0.03735 s

45DeepSeek V4 Flash 073186.2% ±0.3$0.002717 s

45Claude Opus 486.2% ±0.3$0.0510 s

49GLM-5.3-Flash86.1% ±0.3$0.001145 s

50GPT-5.6 Luna86.0% ±0.3$0.002817 s

50GLM-5 Thinking86.0% ±0.3$0.01287 s

52Kimi K2.5 Thinking85.9% ±0.3$0.007939 s

53Grok 4.385.8% ±0.3$0.009511 min

53Gemini 3.5 Flash-Lite85.8% ±0.3$0.00454.8 s

55o385.6% ±0.3$0.01217 s

55Claude Opus 4.585.6% ±0.4$0.0528.2 s

55Inkling-Small85.6% ±0.3$0.005772 s

58Grok 485.3% ±0.3$0.0589 s

59DeepSeek V3.2 (Thinking)84.9% ±0.4$0.00112 min

60MiMo-V2.5-Pro84.6% ±0.5$0.002168 s

60GPT-5.4 mini84.6% ±0.4$0.003720 s

62Qwen3 Maxqwen3-max84.4% ±0.4$0.00640.0 s

63Qwen3.8-27B84.3% ±0.4$0.01170 s

64MiniMax M384.2% ±0.4$0.004841 s

64Grok 4.1 Fast (Reasoning)84.2% ±0.4$0.001125 s

66Qwen3.5 Flash84.1% ±0.4$0.001733 s

66Gemini 2.5 Pro Exp 03.2584.1% ±0.4$0.00918 s

68Claude Sonnet 4 Thinking83.9% ±0.4$0.04647 s

69Gemini 2.5 Flash (Preview)83.7% ±0.4$0.00158.5 s

69Gemini 2.5 Flash Preview Thinking83.7% ±0.4$0.001510 s

71Qwen3 Maxqwen3-max-preview83.5% ±0.4$0.006243 s

71o183.5% ±0.4$0.11627 s

73DeepSeek-R183.2% ±0.4$0.01227 s

74DeepSeek V3.283.1% ±0.4$0.0002841 s

75MiMo-V2.582.9% ±0.4$0.0007730 s

76GLM-4.782.7% ±0.4$0.0082 min

76Claude 3.7 Sonnet Thinking82.7% ±0.4$0.0432 s

78GPT-5 mini82.2% ±0.4$0.005222 s

78GLM-4.682.2% ±0.4$0.006647 s

80Ling 3.0 Flash82.0% ±0.4$0.0008620 s

81Grok 3 Mini Fast High Reasoning81.4% ±0.4$0.001817 s

82Qwen3 235B-A22B81.2% ±0.4$0.002352 s

82GLM-4.581.2% ±0.4$0.00982 min

84Kimi K2 Thinking81.1% ±0.4$0.00212 min

85Claude 3.7 Sonnet80.7% ±0.4$0.00976.3 s

86o4-mini80.6% ±0.4$0.005510 s

87GPT-4.180.5% ±0.4$0.00718.2 s

88MiniMax M2.780.4% ±0.4$0.003950 s

89MiniMax M2.580.1% ±0.4$0.005448 s

90Grok 3 Mini Fast Low Reasoning80.0% ±0.4$0.0017.4 s

91Grok 3 [Beta]79.9% ±0.4$0.01712 s

92Mistral Large 379.8% ±0.5$0.001514 s

93Grok 4 Fast (Reasoning)79.7% ±0.4$0.00177.3 s

94DeepSeek V3 032479.5% ±0.4$0.001525 s

95Claude Sonnet 479.4% ±0.4$0.019.7 s

95Llama4 Maverick Instruct Basic79.4% ±0.4$0.000696.8 s

95Kimi K2 Instruct79.4% ±0.4$0.002820 s

98GPT-OSS 120B79.2% ±0.4$0.001435 s

99Gemini 2.5 Flash Lite Preview Thinking79.1% ±0.4$0.000319.6 s

100Claude Haiku 4.5 Thinking78.7% ±0.5$0.01826 s

100o3-mini78.7% ±0.4$0.01322 s

102Gemini 2.5 Flash Lite Preview78.6% ±0.4$0.000563.2 s

103Claude 3.5 Sonnet78.4% ±0.4$0.00866.3 s

104Gemini 2.0 Flash 00177.4% ±0.4$0.000414.3 s

105GPT-4.1 mini77.2% ±0.4$0.00133.8 s

105GPT-5.4 nano77.2% ±0.4$0.00048.2 s

107GPT-5 nano76.1% ±0.4$0.001926 s

108Grok 275.5% ±0.4$0.0068.7 s

109Mistral Medium 3.575.3% ±0.4$0.04266 s

109Gemini 1.5 Pro 00275.3% ±0.4$0.00263.3 s

109Mistral Medium75.3% ±0.4$0.000218.8 s

112Grok 4.1 Fast Non Reasoning75.2% ±0.4$0.000223.1 s

113DeepSeek V373.8% ±0.4$0.001311 s

114GPT-4o72.6% ±0.4$0.00759.1 s

115GPT-OSS 20B71.6% ±0.4$0.0006230 s

116Llama 3.3 Nemotron Super 49B V1 42e8456170.8% ±0.5—17 s

117Grok 4 Fast Non Reasoning70.3% ±0.4$0.00112.1 s

118Llama 3.3 70B Instruct Turbo69.9% ±0.4$0.00164.2 s

119Mistral Large 241169.7% ±0.4$0.0057.2 s

120Llama 4 Scout 17B 16E Instruct69.6% ±0.4$0.000515.3 s

120Llama 3.3 Nemotron Super 49B V1 42e84561 Thinking69.6% ±0.4—36 s

122Command A69.2% ±0.5$0.00799.9 s

123Laguna XS.269.0% ±0.5$0.0006148 s

124Laguna M.168.8% ±0.5$0.00182 min

125Magistral Medium68.7% ±0.4$0.01532 s

126Mistral Small 3.166.0% ±0.5$0.00023.6 s

127Gemini 1.5 Flash 00265.6% ±0.5$0.000171.7 s

128Mistral Small 240264.4% ±0.5$0.00054.7 s

129Claude 3.5 Haiku64.1% ±0.5$0.00225.8 s

130GPT-4.1 nano63.5% ±0.5$0.000342.4 s

131GPT-4o mini62.7% ±0.5$0.000415.1 s

132Magistral Small62.1% ±0.5$0.003616 s

133Jamba Large 1.649.8% ±0.5$0.00469.5 s

134Command R Plus44.0% ±0.5$0.00574.4 s

135Jamba Mini 1.630.3% ±0.4$0.000392.4 s