Inference
EN
EuroEval

EuroEval Swedish

How well a model handles Swedish, as the unweighted mean of EuroEval's primary metric over its 8 Swedish datasets (swerec, suc3, scala_sv, multi_wiki_qa_sv, swedn, skolprov, swedish_facts, winogrande_sv). The primary metric differs by task — MCC for classification, knowledge and common-sense reasoning, micro-F1 without MISC for entity recognition, F1 for reading comprehension, chrF++ for summarization, METEOR for simplification — and is always the first of the two the board prints. Only models measured on every dataset are included, so each mean covers the same ground. Most rows are the validation split; some older open-weight rows are the test split, which EuroEval ranks in the same table.

ModelsModels with a result.187
Top resultBest result on this benchmark.77.6%GPT-6 Astra
Top-3 spreadPoints from first to third.0.9 pts
YearYear of release.—

Result and price

38 of 195 results

Results

195 results
#ModelResultThe score from the source.
01GPT-6 Astra77.6% ±0.5
02GPT-6.1 Sol77.5% ±0.4
03Gemini 3.7 Flash76.6% ±0.5
04Gemini 3.6 Flash75.9% ±0.4
05GPT-5.6 Sol74.9% ±0.5
06GPT-6 Sol71.3% ±0.5
07Gemini 3 Flash Preview Thinkingthinking effort70.9% ±0.6
08DeepSeek Flash Lowlow effort70.8% ±0.6
09GPT-5.6 Terra70.6% ±0.5
10GLM-5.3-Flash69.3% ±0.7
11DeepSeek Flash No Thinkingno-thinking effort69.1% ±0.8
12GPT-5.4 minihigh effort68.9% ±0.6
13Grok 4.1 Fast (Reasoning)Above fitFar above this model's index.68.3% ±0.6
14GPT-5.4 minimedium effort67.3% ±0.6
15GPT-5.6 Luna66.9% ±0.5
16Qwen3.6 27B FP866.6% ±0.6
17GPT-5 mini66.0% ±0.7
17Claude Sonnet 4.5 Thinkingthinking effort66.0% ±0.6
19Gemini 3.1 Flash Lite Preview65.5% ±0.6
20GPT-5.4 minilow effort65.0% ±0.7
21Qwen3.8-27B64.9% ±0.7
22Claude Sonnet 4.664.5% ±0.7
23GPT-4.1Above fitFar above this model's index.63.8% ±0.7
24GPT-5.263.5% ±0.6
25Qwen3.5 35B A3B FP863.4% ±0.7
26Gemma 4 31Bgemma-4-31B-it63.1% ±0.8
27Gemini 3.5 Flash-Lite63.0% ±0.6
28GPT-5.4 Nonenone effort62.4% ±0.7
29GPT-5 nano62.3% ±0.5
30GPT-OSS 120B61.3% ±0.9
31GPT-6 Luna61.2% ±0.7
32Claude Sonnet 4.5 No Thinkingno-thinking effort60.4% ±0.6
33Mistral Small 3.1 24B60.0% ±0.3
34Cogito V1 Preview Qwen 32B59.9% ±0.2
35Mistral Small 3.2 24B59.4% ±0.3
36GPT-5.4 nanohigh effort59.1% ±0.6
37Gemma 4 26B A4Bgemma-4-26B-A4B-it58.8% ±0.6
38Gemini 2.5 Flash Lite Thinkingthinking effort58.6% ±0.7
39Gemma 3 27B Pt58.4% ±0.4
40NVIDIA Nemotron 3 Nano 30B A3B BF1658.2% ±0.8
41Gemma 3 27BAbove fitFar above this model's index.58.0% ±0.2
42GPT-5.4 nanomedium effort57.8% ±0.5
43NVIDIA Nemotron 3 Nano 30B A3B FP857.3% ±0.8
43Gemma 4 12B IT57.3% ±0.8
45Mistral Small 3.1 24B Base56.6% ±0.4
45Qwen3 32B No Thinkingno-thinking effort56.6% ±0.3
47Qwen3.5 9B Base56.5% ±0.8
48GPT-5.4 nanolow effort56.4% ±0.7
49GPT-5.4 Mini Nonenone effort55.8% ±0.6
50GPT-OSS 20Bmedium effort55.5% ±0.2
51Gemma 3 12B Pt55.0% ±0.3
52Qwen3.5 9B54.6% ±0.9
52Qwen3 4B Thinking54.6% ±0.2
54Bielik 11B V3.0 Instruct54.0%
54Cogito V1 Preview Qwen 14B54.0% ±0.3
56Ministral 3 14B Base53.8% ±0.7
56Borealis 27B53.8% ±0.7
56Gemma 3 12B53.8% ±0.3
56Borealis Open 27B53.8% ±0.7
60Claude Haiku 4.553.7% ±0.5
61Qwen3 14B No Thinkingno-thinking effort53.2% ±0.3
62Gemini 2.5 Flash Lite No Thinkingno-thinking effort53.0% ±0.6
63GPT-OSS 20Blow effort52.0% ±0.4
64Ministral 3 8B (Reasoning)50.4% ±0.3
64Ministral 3 14B Instruct50.4% ±0.9
66Gemma 4 31Bgemma-4-31B50.3% ±1.2
67Borealis Open 12B50.2% ±0.8
68Borealis 12B50.0% ±0.8
69Ministral 3 8B Instruct49.9% ±0.8
70Ministral 3 14B (Reasoning)49.8% ±0.9
70Apertus V1.5 70B49.8% ±0.8
72Olmo 3.1 32B Think49.3% ±1.0
73Qwen3.5-4B48.9% ±1.1
74Qwen3 8B No Thinkingno-thinking effort48.4% ±0.3
74GPT-OSS 20Bhigh effort48.4% ±0.4
76Bielik Minitron 7B V3.0 Instruct48.2%
76EuroLLM 22B Instruct 251248.2% ±0.3
78Ministral 3 8B Base48.1% ±0.8
78Gemma 4 E4B IT48.1% ±0.8
80Qwen3.5 4B Base47.0% ±0.8
80NVIDIA Nemotron 3 Nano 30B A3B Base BF1647.0% ±1.0
82Domyn Small V1.045.6% ±0.9
83Nesso 4B44.8% ±0.3
84Olmo 3.1 32B Instruct44.5% ±0.3
85Gemma 4 26B A4Bgemma-4-26B-A4B44.4% ±0.9
86Qwen3 4B No Thinkingno-thinking effort44.3% ±0.3
87Gemini 3 Flash Preview No Thinkingno-thinking effort44.1% ±0.7
88Olmo 3 32B44.0% ±0.4
88Lynx Instruct 30B Thinkingthinking effort44.0% ±0.3
90EuroLLM 9B Instruct43.5% ±0.3
91EuroLLM 22B 251243.3% ±0.5
92GPT-5.4 Nano Nonenone effort43.2% ±0.7
93Llama 3.1 8B Instruct43.0% ±0.3
94Lynx Instruct 30B No Thinkingno-thinking effort42.7% ±0.3
94Lynx Instruct 30B42.7% ±1.0
96EuroLLM 9B Instruct 251242.2% ±0.9
97Gemma 4 12B42.0% ±1.0
98EuroLLM 9B41.4% ±0.4
98Llama 3.1 Tulu 3 8B SFT41.4% ±0.3
100Gemma 4 E2B IT41.1% ±0.8
101Exaone 4.0 32B41.0% ±1.0
102Llama 3.1 8B40.6% ±0.4
103Qwen3.5 2B Da Task40.0% ±1.0
104DFM Mimir V1.539.8% ±0.6
105Ministral 3 3B (Reasoning)39.2% ±0.3
106Cogito V1 Preview Llama 8B38.8% ±0.4
107Qwen3.5 2B38.0% ±0.9
108Gemma 3 4B37.8% ±0.5
109AMALIA 9B DPO37.4% ±0.7
110Gemma 3 4B Pt37.1% ±0.4
111Borealis 4B36.8% ±0.7
112NorwAI Mixtral 8x7B36.5% ±0.9
113Aya Expanse 8B36.1% ±0.8
114DFM Mimir35.8% ±0.7
115Olmo 3 7B35.7% ±0.3
116NorwAI Mixtral 8x7B One Third Epoch35.2% ±0.8
117Borealis Open 4B33.9% ±0.8
118Qwen3 1.7B No Thinkingno-thinking effort33.5% ±0.3
119AMALIA 9B SFT33.3% ±0.6
120Llama 3.2 3B Instruct33.0% ±0.3
120Apertus V1.1 4B Instruct33.0% ±1.0
122Apertus V1.1 4B32.3% ±1.1
123SmolLM3 3B Base32.2% ±0.4
124Oellm 9B 128K Theta32m V332.1% ±1.0
125Occiglot 7B Eu5 Instruct32.0% ±0.3
126NVIDIA Nemotron 3 Nano 4B BF1631.2% ±0.4
127Occiglot 7B Eu531.0% ±0.4
128Qwen3.5 2B Base30.7% ±0.8
129Olmo 3 7B Instruct30.1% ±0.3
130LFM2.5 8B A1B Base30.0% ±0.9
131NVIDIA Nemotron 3 Nano 4B FP829.9% ±1.1
132Olmo 3 7B Instruct DPO29.6% ±0.4
133NorOLMo 13B29.3% ±1.0
134Gemma 4 E4B29.2% ±1.2
135Cogito V1 Preview Llama 3B28.7% ±0.3
135Qwen3.5 0.8B28.7% ±0.7
137Luciole 8B Instruct 1.128.6% ±1.1
137Teuken 7B Instruct V0.628.6% ±0.7
139Olmo 3 7B Instruct SFT27.3% ±0.4
140Bielik 1.5B V3.0 Instruct27.2% ±0.3
141Salamandra 7B Instruct25.7% ±0.3
142Apertus V1.1 1.5B25.4% ±0.9
143YuLan Mini Instruct24.3% ±0.4
144Trustllm V2 8B Midtrain24.1% ±0.6
145Luciole 23B Base23.7% ±0.9
146Apertus V1.1 1.5B Instruct23.4% ±0.9
147Qwen3 0.6B No Thinkingno-thinking effort23.1% ±0.4
148Gemma 3 1B Pt22.8% ±0.4
149Llama 3.2 1B21.8% ±0.4
150Luciole 8B Base21.7% ±1.1
151Trustllm V2 8B21.6% ±0.7
151Aya 23 8B21.6% ±1.0
153Gemma 3 1B IT21.4% ±0.4
153EuroLLM 1.7B Instruct21.4% ±0.8
155Trustllm V2 8B SFT 8K21.2% ±0.2
156Trustllm V2 8B SFT21.0% ±0.2
157Trustllm V2 8B SFT Mixb2 Exp20.4% ±0.2
158Qwen3.5 0.8B Base20.0% ±0.7
159Norbloom 7B Scratch19.9% ±1.2
160Apertus V1.1 0.5B19.6% ±1.0
161Apertus V1.1 0.5B Instruct19.5% ±0.7
162Gemma 4 E2B19.3% ±1.0
163Borealis 1B18.8% ±0.7
164Normistral 7B Scratch18.3% ±0.9
165Borealis Open 1B18.2% ±0.7
166Luciole 1B Instruct 1.116.3% ±0.9
167Salamandra 7B15.4% ±0.3
167EuroMoE 2.6B A0.6B Instruct Preview15.4% ±0.4
169Pleias 1.2B Preview15.2% ±0.6
170Pleias 3B Preview15.1% ±0.8
171LFM2.5-8B-A1B14.3% ±1.2
172Gemma 3 270M13.6% ±1.2
173LFM2 1.2B12.4% ±0.6
174Llama 3.2 1B Instruct12.3% ±0.6
175LFM2 700M10.7% ±0.5
176Luciole 1B Base10.4% ±0.9
177Pleias 350M Preview9.7% ±0.7
178Gemma 3 270M IT9.6% ±0.7
179Falcon H1 Tiny Multilingual 100M Base9.4% ±0.7
180Ling Mini 2.08.9% ±0.8
181Falcon H1 Tiny Multilingual 100M Instruct7.9% ±0.3
182SmolLM2 360M7.1% ±0.6
183LFM2 350M6.9% ±0.5
183Borealis Open 270M6.9% ±1.0
185SmolLM2 360M Instruct6.5% ±0.6
186Normistral 11B Translate6.3% ±0.7
187SmolLM2 135M Instruct6.1% ±0.6
188SmolLM2 135M5.8% ±0.4
189Borealis 270M5.7% ±1.0
190GPT Sw3 1.3B Instruct4.9% ±1.1
191Baguettotron No Thinkingno-thinking effort3.3% ±0.3
192Monad2.8% ±0.6
193Marco Mini Global Base1.6% ±1.4
194NorwAI Magistral 24B Reasoning1.2% ±0.5
195Teuken 7B Base V0.60.0%