Sierra Research

τ²-bench Airline

Share of tasks an agent completes on one attempt, against a simulated user who holds information it must ask for.

In index
Agentic
Best in agentic
Source ↗

Models22

Top result84.0%Claude Opus 4.5

Top-3 spread1.5 pts

Updated2 Mar 2026

Result and price

18 of 22 results

Results

22 results

#ModelResultCost

01Claude Opus 4.5high effort · Sierra84.0%$0.399

02GPT-5.2high effort · Sierra83.0%$0.114

03Gemini 3 Flashhigh effort · Sierra82.5%—

03GLM-5enabled effort · Sierra82.5%—

05Qwen3.5 397B A17Benabled effort · Sierra81.5%—

06Gemini 3 Prohigh effort · Sierra80.5%—

07Gemini 3.0 ProSierra73.0%—

08Claude Sonnet 4.5enabled effort · Sierra71.0%$0.296

09Qwen3 Max ThinkingQwen69.0%—

10Claude 3.7 SonnetSierra64.2%—

11DeepSeek V3.2DeepSeek63.8%—

12GPT-5Sierra62.5%$0.134

13Claude Sonnet 4Anthropic60.0%—

14Claude Opus 4Anthropic59.6%—

15Qwen3 MaxAlibaba59.5%—

16Kimi K2Moonshot AI56.5%—

17Claude Opus 4.1Anthropic56.0%—

17GPT-4.1Sierra56.0%—

17Nemotron Orchestrator 8BNVIDIA56.0%—

20o4-miniSierra52.1%—

21o3OpenAI52.0%—

22GPT-4.1 miniSierra48.7%—