Qwen3.7-Max: BenchLM Public Evidence Coverage and Speed Ledger

One-sentence takeaway

BenchLM rates Qwen3.7 Max at 71.6/100, with a public rank of #16/218 and an evidence-verified rank of #13/104; it ranks #1 in multilingual performance but only #103 in Agentic, while its API price and model ID have not been independently verified.

Test environment

• Dataset: BenchLM's public model directory and 437 tracked benchmark rows.

• Timing: The page is marked “data as of 2026-08-17,” while the model page lists a 2026-05-16 release.

• Evidence method: A Verified row must be tied to a public source; missing fields remain not sourced/not published and are not filled with estimates.

• Runtime fields: The page records 204 tok/s and 14.07s to first token, but marks the direct context source as not stored; this cannot be treated as a complete runtime benchmark.

Inputs/configuration

The BenchLM page provides a summary of 35 source-displayable rows and some provider-exact values, but does not publish a unified original prompt, sampling parameters, or all runtime traces. Its “official exact-value snapshot” cites the Qwen release page dated 2026-05-16.

Results data

Overall and category results

Coding results in the page ledger

• LiveCodeBench 91.6%; listed by the page as best verified.

• SWE-bench Verified 80.4%, 15.6 percentage points below the page's best-verified Claude Opus 5 at 96%.

• SciCode 53.5%, 6.6 percentage points below best-verified Sakana Fugu at 60.1%.

• SWE-bench Pro 60.6%, 19.7 percentage points below best-verified Claude Mythos 5 at 80.3%.

• SWE Multilingual 78.3%, NL2Repo 47.2%, and Terminal-Bench 2.0 69.7%.

Conclusions

• Multilingual performance and instruction following are the most prominent relative strengths in BenchLM's evidence; coding evidence is fairly complete, but not every SWE/Terminal metric is near the top of the leaderboard.

• Agentic #103/131 is in clear tension with the official long-horizon case, showing that a “35-hour demo” cannot substitute for independent Agent metrics across tasks and harnesses.

• The 204 tok/s speed figure is only a directory runtime entry; with 14.07s TTFT and variables such as region, concurrency, and long context, it must be retested with your own API trace.

• BenchLM marks the model as superseded, and Qwen3.8 Max Preview has appeared in the lineage; production selection should first confirm whether the stable 3.7 snapshot is still required.

Limitations

• BenchLM is a third-party aggregator; its composite score includes disclosed weighting and missing-data handling. Do not treat 71.6 as a single benchmark.

• API model ID, price, maximum output, knowledge cutoff, modality, and other fields are marked as unpublished or not independently verified.

• Some benchmarks are provider exact or display only, and their harnesses and evaluators are not fully consistent.

• The direct sources for the speed and context fields are incomplete; they cannot replace the official API pricing page or your own load test.

Reproduction steps

1. Save the BenchLM collection date, model lifecycle status, and evidence label for each ledger entry.

2. Using the official Qwen page, pin a qwen3.7-max dated snapshot and run LiveCodeBench, SWE, and multilingual tasks separately.

3. For Agent tasks, additionally record TTFT, output speed, tool calls, context length, failures/retries, and cost-per-success.

4. Report “public rank,” “verified rank,” category scores, and speed separately; do not recompute them into an unexplained total score.

5. Run an A/B test against Qwen3.8 Max Preview with the same harness to assess the reproducibility and supply risks of continuing with 3.7.

Source excerpt or observation (brief excerpt for compliance only)

BenchLM's decision prompt is “Validate before choosing,” because the 35 visible results still leave some benchmark slots empty.