Claude Sonnet 4.6 · Community source · Editorial analysis
Harvey recorded Claude Sonnet 4.6 at a 4.2% full-pass rate on the legal agent benchmark LAB, below Opus 4.6 at 6.6%; on the same leaderboard, post-trained NVIDIA Nemotron 3 Ultra reached 5.8%, and claimed operating costs are 1/8 to 1/50 of Sonnet/Opus.
Unverified: the original source could not be rechecked. Historical figures below are not current verified results.
Harvey recorded Claude Sonnet 4.6 at a 4.2% full-pass rate on the legal agent benchmark LAB, below Opus 4.6 at 6.6%; on the same leaderboard, post-trained NVIDIA Nemotron 3 Ultra reached 5.8%, and claimed operating costs are 1/8 to 1/50 of Sonnet/Opus.
Benchmark: Harvey Legal Agent Bench (LAB).
Metric: full-pass / full-pass rate.
Comparison: Nemotron 3 Ultra baseline 0% → post-training 5.8%; Sonnet 4.6 4.2%; Opus 4.6 6.6%.
Additional observation: held-out tasks were ~70% pass before training (because enough scoring dimensions were missed), ~95% after training. The 70%/95% refers to Nemotron before/after post-training, not Sonnet.
The X post did not disclose LAB questions, scoring dimensions, Sonnet's prompt, toolset, effort, or whether legal retrieval plugins were enabled. The Trajectory post stated the entire post-training was completed in under 24 hours after Nemotron 3 Ultra's release.
| Model | LAB full-pass rate |
|---|---|
| Nemotron 3 Ultra (no post-training) | 0% |
| Claude Sonnet 4.6 | 4.2% |
| Nemotron 3 Ultra (legal post-training) | 5.8% |
| Claude Opus 4.6 | 6.6% |
Harvey also wrote: the post-trained open-weight model reached quality close to leading closed-source models, with operating costs at 1/8 to 1/50 of Sonnet 4.6 and Opus 4.6 per-token prices.
Legal agent "full pass" is very strict: Sonnet 4.6's 4.2% does not mean it cannot do legal assistance, but means that under Harvey's all-dimensions-pass standard, it is clearly weaker than Opus 4.6 and also weaker than a specially post-trained Nemotron. Sonnet 4.6 is suitable for legal drafting/retrieval assistance, not as an automatic pass agent in LAB terms. High-risk legal deliverables should still go through Opus or domain post-trained models, with lawyer review retained.
Full-pass rate is not partial correctness rate; 4.2% is not the same metric as everyday "can write a usable memo."
Per-question results and Sonnet configuration not disclosed.
Cost 1/8–1/50 is the author's statement on Nemotron vs Claude unit prices, not LAB scores.
Cannot conflate LAB with IDP document extraction or SWE coding.
If Harvey/Trajectory later publish a LAB subset, fix the same scoring dimensions and run claude-sonnet-4-6 vs claude-opus-4-6.
Report full pass, partial pass, citation errors, and missed scoring dimensions separately.
Record retrieval tools, jurisdictions, and whether internet access was allowed.
Legal conclusions must be reviewed by qualified personnel; model scores cannot substitute.
The figures, task set, reasoning tier, and client conditions apply only to the listed source and collection snapshot. Different versions, harnesses, or providers must not be compared directly; undisclosed parameters remain unknown.
For a reproduction, fix the model version, provider or client, reasoning tier, tools, task-set version, sample count, and collection date, and record failures, retries, and human corrections. Full steps are in the source notes below.
X · Harvey (@harvey), citing Trajectory (@trajectorylabs) · Original publication date 2026-06-11 · Site edit date 2026-09-20
Open original sourceClaude Sonnet 4.6
Download the Tabbit client to check model access
