Reddit Community: Task Steps and High-Effort Cost Pitfall Analysis for Sonnet 5 Based on DeepSWE Benchmark

One-sentence takeaway

Community discussions based on the Datacurve DeepSWE complex coding benchmark point out that while Sonnet 5 has a lower per-token rate, it often requires more steps and trial-and-error loops in difficult, long-horizon tasks; blindly enabling high effort tiers can make its actual cost per task worse than Opus 4.8.

Test environment

• Basis of discussion: Datacurve DeepSWE benchmark evaluation results ( deepswe.datacurve.ai ) and actual billing and step comparisons from heavy community Agent users.

• Task types: Complex multi-turn autonomous software engineering (SWE) tasks, repository refactoring, and single-step document extraction.

• Compared models: Claude Sonnet 5 (different effort tiers) vs Claude Opus 4.8.

Input/configuration

• Long-context coding issues of identical difficulty.

• Sonnet 5 configured to medium/high effort and Opus 4.8 configured to corresponding tiers respectively.

Results data

• Community user comparisons found that on highly complex tasks such as DeepSWE, Sonnet 5's average cost per task under medium effort was close to that of Opus 4.8 high effort, while scoring more than 10 percentile points lower.

• Step count disparity: When tackling difficult problems, Sonnet 5 is prone to "circling around trying things," requiring significantly more Agent interaction turns and generated tokens to complete a single complex task compared to Opus 4.8.

• Scenario divergence: In single-pass structured extraction (such as batch parsing multi-format documents with a single prompt), Sonnet 5 consumes minimal thinking and its per-task cost is far lower than Opus; however, in open-ended, long-horizon autonomous debugging, the cost advantage of high-effort Sonnet 5 is offset by the additional steps required.

Conclusion

Do not blindly enable high/xhigh effort tiers on Sonnet 5 for all complex tasks. A reasonable engineering model selection approach is:

1. Use Sonnet 5 (low/medium effort or disabled thinking) for single-step tasks with well-defined input and output formats.

2. Directly use Opus 4.8 or flagship models for highly complex, multi-module collaborative challenges, where faster convergence and fewer trial-and-error steps often result in a lower overall bill.

Limitations

• The community referenced early public summaries of DeepSWE, where problem-by-problem prompts and Agent frameworks were not fully disclosed.

• The discussion primarily reflects edge cases in complex, long-horizon coding and should not completely negate Sonnet 5's high cost-effectiveness on standardized small-to-medium tasks.

Reproduction steps

1. Select 20 challenging multi-file code defect issues.

2. Run autonomous resolution Agents on Sonnet 5 (high effort) and Opus 4.8 (medium/high effort) respectively.

3. Collect statistics on resolution rate, average interaction steps, total input/output/thinking token consumption, and final dollar cost.

4. Plot a scatter chart of cost per task versus success rate.

Original evidence and data

• Core focus of community discussion: "Sonnet 5 med is the same avg cost as Opus 4.8 high while scoring 10+ pctile points worse on DeepSWE."

• Core mechanism explanation: "Sonnet is actually cheaper per token, but it takes way more steps and tokens to finish the tasks, so that's what makes it lose its price advantage on hard tasks."

Source excerpts or observations (for compliance short quotes only)

• Core discussion viewpoint: "'Cost per task' - That's the difference. Sonnet burns more tokens circling around trying things before it can conclude a task that Opus can finish handily."

• Concluding advice: "You should decompose tasks to the level where they are suitable for smaller models on low/medium reasoning... bigger models orchestrate."