Vals AI

MGSM

A multilingual benchmark for mathematical questions.

Not in index
Multilingual
Source ↗

Models73

Top result95.2%Claude Opus 4.5 Thinking

Top-3 spread0.8 pts

Updated9 Jan 2026

Result and cost

74 of 74 results

Results

74 results

#ModelResultCostTime

01Claude Opus 4.5 Thinking95.2% ±1.3$0.06312 s

02Claude Opus 4.594.8% ±1.4$0.0275.6 s

03Claude Opus 4.1 Thinking94.4% ±1.4$0.05514 s

04Claude Sonnet 4.5 Thinking94.3% ±1.4$0.01110 s

05Claude Opus 4.194.2% ±1.5$0.0277.9 s

06GPT-5.294.0% ±1.4$0.01111 s

07Gemini 3 Pro93.9% ±1.4$0.01813 s

08Claude Opus 493.8% ±1.5$0.02710 s

09o4-mini93.4% ±1.5$0.00298.1 s

10Gemini 3 Flash Preview93.3% ±1.5$0.007114 s

11Claude Sonnet 493.0% ±1.6$0.00586.0 s

11GPT-5.193.0% ±1.5$0.00618.3 s

11Claude 3.7 Sonnet Thinking93.0% ±1.6$0.02624 s

14GPT-592.8% ±1.6$0.01420 s

15GPT-5 mini92.6% ±1.6$0.003713 s

15Claude 3.5 Sonnet92.6% ±1.6$0.0044.1 s

17Qwen3 235B-A22B92.5% ±1.6$0.001132 s

18Llama4 Maverick Instruct Basic92.4% ±1.6$0.000222.7 s

18Claude 3.7 Sonnet92.4% ±1.6$0.00574.9 s

20DeepSeek-R192.3% ±1.7$0.004110 s

21Qwen3 Maxqwen3-max-preview92.1% ±1.6$0.002321 s

21Claude Haiku 4.5 Thinking92.1% ±1.6$0.00699.6 s

21DeepSeek V392.1% ±1.7$0.0003616 s

24GPT-OSS 120B92.0% ±1.6$0.000665.8 s

25Qwen3 Maxqwen3-max91.8% ±1.7$0.002911 s

26o391.7% ±1.7$0.00386.8 s

26DeepSeek V3 032491.7% ±1.7$0.000418 s

28o3-mini91.3% ±1.7$0.008216 s

28Grok 3 [Beta]91.3% ±1.7$0.00575.8 s

30Llama 3.3 70B Instruct Turbo91.1% ±1.8$0.000523.0 s

31Kimi K2 Instruct90.9% ±1.8$0.00112 s

31Grok 490.9% ±1.8$0.0262 min

31Claude Sonnet 4 Thinking90.9% ±1.7$0.01514 s

31Grok 4 Fast (Reasoning)90.9% ±1.8$0.000643.9 s

31DeepSeek V3.290.9% ±1.8$0.000159.0 s

31Mistral Medium90.9% ±1.7$0.000495.0 s

37GLM-4.590.8% ±1.8$0.006390 s

38Grok 3 Mini Fast High Reasoning90.4% ±1.8$0.000810 s

38GPT-4o90.4% ±1.8$0.00434.0 s

38Grok 3 Mini Fast Low Reasoning90.4% ±1.8$0.000516.6 s

41Kimi K2 Thinking90.1% ±1.8$0.0008948 s

42Gemini 2.5 Flash (Preview)89.9% ±1.8$0.0013.4 s

43Gemini 2.5 Flash Preview Thinking89.8% ±1.8$0.0015.7 s

44GLM-4.689.7% ±1.8$0.004334 s

45Grok 4.1 Fast (Reasoning)89.5% ±1.9$0.0005214 s

45Gemini 2.5 Flash Lite Preview89.5% ±1.9$0.000211.5 s

47o189.3% ±1.8$0.05711 s

47GPT-5 nano89.3% ±1.9$0.001923 s

49Gemini 1.5 Pro 00289.2% ±1.9$0.0012.8 s

50GPT-OSS 20B89.0% ±1.9$0.000455.8 s

50Gemini 2.0 Flash 00189.0% ±1.9$0.00011.5 s

52Gemini 2.5 Flash Lite Preview Thinking88.4% ±1.9$0.000154.6 s

53GLM-4.788.2% ±1.8$0.004363 s

54Grok 4 Fast Non Reasoning88.0% ±2.0$0.000341.8 s

54Llama 4 Scout 17B 16E Instruct88.0% ±2.0$0.000153.5 s

56MiniMax M2.187.9% ±1.8$0.001314 s

57GPT-4.1 mini87.8% ±1.9$0.000392.7 s

58GPT-4.187.7% ±1.9$0.00172.2 s

59Grok 4.1 Fast Non Reasoning87.6% ±2.0$0.000153.7 s

60Mistral Large 241187.2% ±2.0$0.0038.0 s

61Gemini 1.5 Flash 00286.6% ±2.1$0.0000641.4 s

62Magistral Small86.3% ±2.1$0.00158.0 s

63GPT-4o mini86.2% ±2.1$0.000194.0 s

64Grok 286.1% ±2.0$0.00386.3 s

65DeepSeek V3.2 (Thinking)86.0% ±2.0$0.000428 s

66Command A85.7% ±2.0$0.00378.4 s

67Mistral Large 385.4% ±1.8$0.000477.6 s

68Claude 3.5 Haiku84.6% ±2.2$0.00113.7 s

69Mistral Small 3.184.2% ±2.2$0.0000953.9 s

70Mistral Small 240284.0% ±2.2$0.000213.0 s

71Magistral Medium74.6% ±2.5$0.005712 s

72Jamba Large 1.671.2% ±2.6$0.00389.9 s

73GPT-4.1 nano69.3% ±2.8$0.000131.5 s

74Jamba Mini 1.641.7% ±2.9$0.000283.9 s