Inference
EN

Smaug Flash

Smaug Flash is a model. It has 6 scored results in 6 categories. It has no index.

  • Input: text
UnrankedEnough results for an index.
IndexAll categories, out of 100.UnrankedMedian 51.0
CoverageIndex weight that has results.85%
SpeedOutput tokens per second.—Median 67/s
First tokenSeconds to the first output token.—Median 1.3 s
Input / 1MUS dollars per 1M input tokens.—Median $0.60
Output / 1MUS dollars per 1M output tokens.—Median $2.55
ContextMaximum tokens in one request.—Median 262K
EloLMArena rating and rank.—Median 1422

Capabilities

Index per category, out of 100.
Out of 100
AgenticMulti-step tasks with tools.1 counted72.1—
CodingCode writing and repair.1 counted63.3—
ReasoningLogic problems and puzzles.1 counted68.7—
MultimodalTasks with images and text.None counted——
KnowledgeFacts and expert knowledge.1 counted64.1—
MultilingualTasks in many languages.None counted——
InstructionTasks with strict prompt rules.1 counted66.7—
MathMath problems.1 counted65.3—

Results

6 counted
BenchmarkThe benchmark name.ResultThe score from the source.PlacePlace among results on this benchmark.
LiveBench Agentic CodingAgentic61.1%14 / 66
LiveBench Data AnalysisReasoning79.1%20 / 80
LiveBench ReasoningReasoning86.2%34 / 80
LiveBench Instruction FollowingInstruction69.3%31 / 66
LiveBench MathematicsMath90.1%33 / 66
LiveBench LanguageKnowledge79.2%34 / 66
LiveBench CodingCoding77.1%41 / 66

Sources

BenchLMCC BY-NC 4.0LiveBenchNo licence stated