EuroEval Spanish

How well a model handles Spanish, as the unweighted mean of EuroEval's primary metric over its 8 Spanish datasets (sentiment_headlines_es, conll_es, scala_es, mlqa_es, mlsum_es, include_es, multiloko_es, winogrande_es). The primary metric differs by task — MCC for classification, knowledge and common-sense reasoning, micro-F1 without MISC for entity recognition, F1 for reading comprehension, chrF++ for summarization, METEOR for simplification — and is always the first of the two the board prints. Only models measured on every dataset are included, so each mean covers the same ground. Most rows are the validation split; some older open-weight rows are the test split, which EuroEval ranks in the same table.

ModelsModels with a result.168
Top resultBest result on this benchmark.70.0%GPT-6 Astra
Top-3 spreadPoints from first to third.1.8 pts
YearYear of release.—

Result and price

38 of 176 results

Results

176 results
#ModelResultThe score from the source.
01GPT-6 Astra70.0% ±0.4
02Gemini 3.7 Flash68.3% ±0.4
03GPT-6.1 Sol68.2% ±0.5
04Gemini 3.6 Flash66.8% ±0.4
06GPT-5.6 Sol66.3% ±0.4
07GPT-6 Sol65.6% ±0.4
11GLM-5.3-Flash62.2% ±0.5
12GPT-5.6 Terra62.1% ±0.4
13Qwen3.6 27B FP861.4% ±0.6
17Qwen3.8-27B59.0% ±0.6
18Claude Sonnet 4.658.5% ±0.5
19GPT-5.6 Luna57.9% ±0.4
21Qwen3.5 35B A3B FP857.6% ±0.7
28GPT-5 mini54.2% ±0.6
31GPT-5.253.5% ±0.4
31GPT-OSS 120B53.5% ±0.4
36GPT-4.152.8% ±0.5
38GPT-6 Luna52.1% ±0.5
39Qwen3 4B Thinking51.5% ±0.4
44Claude Haiku 4.549.7% ±0.4
45GLM-4.7 Flash NVFP449.4% ±0.6
47Gemma 3 27B49.2% ±0.4
48GPT-5 nano49.0% ±0.5
52Exaone 4.0 32B48.3% ±0.7
54Apertus V1.5 70B47.3% ±0.6
57Qwen3.5 9B46.1% ±0.7
60Gemma 3 12B45.1% ±0.4
61Gemma 4 12B IT45.0% ±0.6
65Gemma 3 27B Pt43.8% ±0.4
66Nesso 4B43.7% ±0.3
73Ministral 3 8B Base40.2% ±0.6
75Qwen3.5 9B Base40.1% ±0.5
77AMALIA 9B DPO39.6% ±0.5
79Aya Expanse 8B39.2% ±0.5
80Gemma 3 12B Pt39.1% ±0.4
84AMALIA 9B SFT37.8% ±0.6
85Gemma 4 E4B IT37.5% ±0.6
86Gemma 4 E2B IT37.4% ±0.7
88Qwen3.5-4B36.8% ±0.7
89Domyn Small V1.036.4% ±0.7
89DFM Mimir V1.536.4% ±0.5
93Qwen3.5 4B Base35.4% ±0.6
96Olmo 3 32B35.1% ±0.3
97EuroLLM 9B34.8% ±0.4
100LFM2.5 8B A1B Base34.0% ±0.6
103EuroLLM 22B 251232.7% ±0.4
104Gemma 4 12B32.3% ±0.9
106Gemma 3 4B31.9% ±0.4
107Qwen3.5 2B31.5% ±0.5
110Llama 3.1 8B31.3% ±0.5
111Olmo 3 7B30.9% ±0.4
112SmolLM3 3B Base30.8% ±0.4
114Gemma 3 4B Pt30.1% ±0.4
115Llama 3.2 3B Instruct29.6% ±0.4
116Olmo 3 7B Instruct29.1% ±0.4
117Occiglot 7B Eu528.8% ±0.4
119DFM Mimir27.9% ±0.6
122Apertus V1.1 4B27.5% ±0.6
124Qwen3.5 2B Base27.1% ±0.6
125Luciole 23B Base26.8% ±0.8
126Qwen3.5 0.8B26.5% ±0.7
130LFM2.5-8B-A1B24.1% ±1.1
131Gemma 4 E4B23.6% ±0.8
132Luciole 8B Base22.4% ±0.9
134LFM2 1.2B22.3% ±0.4
135TildeOpen 30B22.0% ±0.7
136Qwen3.5 0.8B Base21.8% ±0.6
138YuLan Mini Instruct20.5% ±0.6
140Apertus V1.1 1.5B19.0% ±0.7
142Aya 23 8B18.1% ±0.7
143LFM2 700M17.8% ±0.4
145Salamandra 7B17.2% ±0.3
146Gemma 3 1B Pt17.1% ±0.4
147Gemma 3 1B IT16.2% ±0.4
148Apertus V1.1 0.5B15.9% ±0.7
149Luciole 1B Base15.3% ±0.9
150Llama 3.2 1B15.1% ±0.6
151Pleias 3B Preview14.9% ±0.5
152Gemma 4 E2B14.6% ±0.9
152Pleias 1.2B Preview14.6% ±0.6
154LFM2 350M14.0% ±0.3
158Nesso 0.4B Agentic12.4% ±0.6
159Llama 3.2 1B Instruct12.3% ±0.4
160SmolLM2 360M12.2% ±0.5
161Ling Mini 2.011.3% ±0.6
162Gemma 3 270M10.8% ±0.6
164Pleias 350M Preview10.3% ±0.4
165MiniCPM5-1B9.7% ±0.8
166SmolLM2 135M9.1% ±0.4
169Gemma 3 270M IT7.6% ±0.5
171Zagreus 0.4B Ita4.9% ±0.4
174Monad1.5% ±0.3
175Muse 3B0.8% ±0.4