Claude Sonnet 4.6 · Community source · Editorial analysis
On the standard Ubuntu desktop benchmark suite OSWorld-Verified, Claude Sonnet 4.6 achieved a 72.5% task success rate, nearly matching flagship Opus 4.6 (72.7%), but still shows some visual localization jitter in high-frequency complex dynamic popups and multi-level right-click menu scenarios.
Unverified: the original source could not be rechecked. Historical figures below are not current verified results.
On the standard Ubuntu desktop benchmark suite OSWorld-Verified, Claude Sonnet 4.6 achieved a 72.5% task success rate, nearly matching flagship Opus 4.6 (72.7%), but still shows some visual localization jitter in high-frequency complex dynamic popups and multi-level right-click menu scenarios.
Model: Claude Sonnet 4.6 (API version claude-sonnet-4-6).
Pricing: Input $3.00 per million tokens, output $15.00 per million tokens.
Benchmark/workflow: OSWorld-Verified (369 cross-application real desktop tasks covering Chrome, LibreOffice Calc/Writer, VSCode, Thunderbird, GIMP, and native OS file management).
Version boundary: Evaluations uniformly run in real virtual desktop containers at 1024×768 resolution, without assistive accessibility label trees (pure pixel screenshot + coordinate click mode).
Scaffold: Standard OSWorld agent loop.
Parameter configuration: effort=medium, max action steps limited to 25 steps/task, mandatory full-screen capture before each action with action trajectory recorded.
Task forms: e.g. "Aggregate sales data by condition in LibreOffice Calc and export as a CSV with a specific name", "Collect product information across multiple Chrome tabs and fill into a purchase form".
| Task category | Sonnet 4.6 success rate | Opus 4.6 success rate | Sonnet 4.5 baseline | Human reference level |
|---|---|---|---|---|
| Full suite aggregate (OSWorld-Verified) | 72.5% | 72.7% | 61.4% | 72.36% |
| Web multi-step forms (Chrome) | 84.2% | 85.0% | 71.3% | 88.5% |
| Office documents and spreadsheets (LibreOffice) | 71.8% | 72.4% | 58.9% | 76.0% |
| File system and system configuration (OS/Terminal) | 79.5% | 78.9% | 69.2% | 82.0% |
| Complex professional software (GIMP/CAD) | 54.5% | 54.8% | 46.2% | 63.0% |
| Average tokens per task | 34.2k | 48.6k | 38.1k | - |
| Average time per task (seconds) | 38.5s | 62.1s | 45.2s | 22.0s |
Cost-performance leap: Sonnet 4.6's overall computer use score differs from Opus 4.6 by less than 0.2%, but average completion time is 38% shorter and token cost is nearly 40% lower, making it highly practical for large-scale RPA and desktop agent deployment.
Exceeding human norm: In standardized, well-structured cross-application data transcription tasks, Sonnet 4.6's success rate already slightly exceeds the normal human tester benchmark (72.5% vs 72.36%).
Failure modes concentrated: Main failures concentrate on: 1) minor pixel-level dropdown arrow click offset (about 35% of failures); 2) action racing ahead due to slow asynchronous network loading; 3) software shortcut conflicts not triggered.
Advanced graphic editing weaker: In GIMP image cropping, layer blending, and other continuous spatial judgment tasks, success rate is only slightly above 50%.
Clone the official evaluation repository https://github.com/xlang-ai/OSWorld.
Configure environment variable ANTHROPIC_API_KEY, and in the config file specify model="claude-sonnet-4-6" and anthropic-beta="computer-use-2026-01-24".
Run the evaluation command: python run.py --benchmark verified --model claude-sonnet-4-6 --max_steps 25.
Export aggregated results and verify artifact file hashes and status using the evaluation script.
The figures, task set, reasoning tier, and client conditions apply only to the listed source and collection snapshot. Different versions, harnesses, or providers must not be compared directly; undisclosed parameters remain unknown.
For a reproduction, fix the model version, provider or client, reasoning tier, tools, task-set version, sample count, and collection date, and record failures, retries, and human corrections. Full steps are in the source notes below.
OSWorld Benchmark Leaderboard / Evaluation Suite · OSWorld Evaluation Team & Independent Researchers · Original publication date 2026-02-28 · Site edit date 2026-09-20
Open original sourceClaude Sonnet 4.6
Download the Tabbit client to check model access
