Llama 3.1 70B vs Llama 3.1 8B

Change models
IndexAll categories, out of 100.
24.8 ±5.3 best
17.9 ±9.6
GapIndex points below the top.
Top
−6.9 · no clear gap
CoverageIndex weight that has results.
90% best
55%
SpeedOutput tokens per second.
—
—
EloLMArena rating and rank.
—
—
ReleasedMonth of release.
2024-07
2024-07
Input / 1MUS dollars per 1M input tokens.
—
—
Output / 1MUS dollars per 1M output tokens.
—
—
1K chats2K tokens in, 500 out, each.
—
—
1K documents20K tokens in, 1K out, each.
—
—
1K agent tasks200K tokens in, 20K out, each.
—
—
ContextMaximum tokens in one request.
128K
128K
AgenticMulti-step tasks with tools.
17.1
—
CodingCode writing and repair.
21.4 best
13.9
ReasoningLogic problems and puzzles.
23.7
—
MultimodalTasks with images and text.
—
—
KnowledgeFacts and expert knowledge.
27.6 best
14.4
MultilingualTasks in many languages.
41.0 best
24.8
InstructionTasks with strict prompt rules.
27.8 best
25.1
MathMath problems.
13.9 best
10.0

Benchmarks

16 shared
Only Llama 3.1 70B: 4Only Llama 3.1 8B: 4
Llama 3.1 70B
Llama 3.1 8B
3.6%210 / 222
4.3% best190 / 222
44.2% best166 / 217
27.0%205 / 217
EuroEval PolishMultilingual
56.5% best32 / 209
41.4%105 / 209
3.6% best176 / 200
1.7%188 / 200
49.1% best176 / 199
27.0%196 / 199
EuroEval GermanMultilingual
52.1% best30 / 191
36.6%95 / 191
59.3% best32 / 182
41.1%103 / 182
EuroEval FrenchMultilingual
58.4% best45 / 178
45.9%95 / 178
46.7% best126 / 141
30.4%134 / 141
36.7% best62 / 88
22.9%69 / 88
66.4% best62 / 75
48.3%67 / 75
87.5% best33 / 70
80.4%54 / 70
86.0% best16 / 34
73.0%29 / 34
80.5% best15 / 22
72.6%19 / 22
86.9% best4 / 13
68.9%8 / 13
95.1%
84.5%