Inference
EN
Fixie AI

Ultravox v0.6 Llama 3.3 70B

Ultravox v0.6 Llama 3.3 70B is a standard model from Fixie AI. It has 7 scored results in 4 categories. It has no index.

  • Open weights
  • Standard
  • Input: text
UnrankedEnough results for an index.
IndexAll categories, out of 100.UnrankedMedian 51.0
CoverageIndex weight that has results.55%
SpeedOutput tokens per second.—Median 67/s
First tokenSeconds to the first output token.—Median 1.3 s
Input / 1MUS dollars per 1M input tokens.—Median $0.60
Output / 1MUS dollars per 1M output tokens.—Median $2.55
ContextMaximum tokens in one request.—Median 262K
EloLMArena rating and rank.—Median 1422

Capabilities

Index per category, out of 100.
Out of 100
AgenticMulti-step tasks with tools.2 counted27.2—
CodingCode writing and repair.None counted——
ReasoningLogic problems and puzzles.2 counted27.5—
MultimodalTasks with images and text.None counted——
KnowledgeFacts and expert knowledge.2 counted25.4—
MultilingualTasks in many languages.None counted——
InstructionTasks with strict prompt rules.1 counted33.0—
MathMath problems.None counted——

Results

7 counted
BenchmarkThe benchmark name.ResultThe score from the source.PlacePlace among results on this benchmark.
Artificial Analysis IFBenchInstruction47.1%84 / 146
Critical Physics TasksReasoning0.0%150 / 214
GDPval-AA normalizedAgentic0.0%101 / 133
τ²-Bench Tool-Agent-User EvaluationAgentic26.6%116 / 149
Artificial Analysis GPQA DiamondKnowledge49.8%174 / 199
Artificial Analysis Long Context ReasoningReasoning15.7%196 / 217
Artificial Analysis Humanity's Last ExamKnowledge3.6%210 / 222

Sources

BenchLMCC BY-NC 4.0

Questions

Is Ultravox v0.6 Llama 3.3 70B open source?
Yes. Its weights are published.

More from Fixie AI

All models
Ultravox GLM-4P6UnrankedUltravox GLM-4P7UnrankedUltravox v0.4.1 Llama 3.1 8BUnrankedUltravox v0.5 Llama 3.1 8BUnrankedUltravox v0.5 Llama 3.2 1BUnranked