DeepSeek-V4-Flash: 0731 Benchmark Update and Harness Conditions

One-sentence takeaway

The official 0731 update shows that V4-Flash scores significantly higher than V4-Pro-Preview on agent benchmarks, but the results depend on DeepSeek Harness minimal mode and max effort; the harness and parameters must be reproduced together for a meaningful comparison.

Use cases

• Suitable tasks: Official positioning of the version's capabilities for code agents, terminal operations, tool calling, automation, and full-stack development.

• Unsuitable tasks: Extrapolating scores from a single official harness directly to chat, vision, multimodal use, or another agent framework.

• Applicable model version: The public beta deepseek-v4-flash with the 0731 API update; the update page says the model architecture and size remain unchanged, with only post-training redone.

• Applicable client, agent, or API: DeepSeek API; official DeepSeek Harness minimal mode.

• Recommended reasoning level and parameters: To reproduce the official Code Agent conditions, use max, top_p=0.95, and temperature=1.0; ordinary applications should conduct their own parameter evaluation again.

Test environment, input/configuration

• Model: DeepSeek-V4-Flash 0731 API.

• Harness: DeepSeek Harness minimal mode (the official explanation said it was about to be released at the time).

• Reasoning level: max.

• Sampling configuration: top_p=0.95, temperature=1.0.

• Task sets: Terminal Bench 2.1, NL2Repo, CyberGym, DeepSWE, Toolathlon Verified, Agents' Last Exam, Automation Bench Public, as well as the internal DSBench-FullStack and DSBench-Hard.

• Original inputs: The complete task inputs and harness implementation for each benchmark were not made public on the update page, so an individual test cannot be reconstructed from this page alone.

Result data

Conclusion

This update supports treating V4-Flash as a low-cost code-agent candidate, particularly for terminal work, repository modifications, and tool orchestration. However, it demonstrates the performance of the combination “V4-Flash + official harness + specified parameters,” not a uniform capability of the bare model across all clients.

Limitations

• The official update page does not provide each question's inputs, outputs, failure samples, number of repetitions, confidence intervals, or complete harness code, so this cannot be called an independently reproducible evaluation.

• DSBench-FullStack and DSBench-Hard are internal collections and cannot be rerun directly by external parties.

• The update page says the comparison target is V4-Pro-Preview, but does not provide a complete comparison table in the same paragraph; do not fill in the missing comparison scores yourself.

• The 0731 API upgraded only V4-Flash; the V4-Pro API and the APP/Web models had not changed at the time. Product-side results cannot be treated directly as API results.

Reproduction steps

1. In the DeepSeek API, pin the model version and the available 0731 endpoint.

2. Record the API response protocol, tool descriptions, timeouts, retries, context trimming, and workspace snapshot.

3. Have your own agent replicate the official minimal harness as closely as possible, and fix max, top_p=0.95, and temperature=1.0.

4. Start with multiple repetitions of publicly available tasks such as Terminal Bench 2.1 and NL2Repo, saving each task's patch, test results, number of tool calls, and token usage.

5. Present the results alongside the official table, while also running the target production harness separately; do not compare only a single overall score.

Original evidence and data

The public figures given in the original update page are 82.7 / 54.2 / 76.7 / 54.4 / 70.3 / 25.2 / 25.1, and the two DSBench figures are marked as internal tests; the same page explicitly states DeepSeek Harness minimal mode, max effort, topp=0.95, and temperature=1.0.

Source excerpt or observation (compliance short quote only)

The official announcement calls this update “significantly enhanced agent capabilities,” while also noting that Flash 0731 “keeps the same model architecture and size ... and was only re-post-trained.”