Kimi K3.1 Pro official release: Evaluation boundaries for coding, 3D, and finance benchmarks

One-sentence takeaway

Kimi reports that K3.1 Pro scored 65.7% on Terminal-Bench 4.0, 92.7% on BenchCAD with tools, and 64.4 on OfficeQA Pro v2. These results come from different harnesses, with some based on internal benchmarks or external references, so they are useful for locating capability areas but cannot be treated as independent win rates under uniform conditions.

Use cases

• Tasks these results can help assess: Long-horizon coding agents, tool-assisted 3D/CAD generation, financial document analysis, and financial modeling and research workflows.

• Tasks these results should not be extrapolated to: Treating Kimi's internal benchmarks as daily success rates; combining different harnesses, model reports, or vendor charts directly into a ranking; inferring stable production quality from examples.

• Applicable model version: Kimi K3.1 Pro. Kimi K3.1 Pro Highspeed is a high-speed variant released on the same day; the scores in this article do not independently establish that it is identical to Pro.

• Test environments or clients: Claude-Code, Kimi Code, Kimi API, and the official Kimi Agents/Finance Agent, as specified in the respective footnotes.

• Reasoning setting and parameters: Unless otherwise specified, reasoning effort=max, temperature=1.0, and top_p=0.95. The article says the product and API are available; full model weights are scheduled for release by 2026-11-04, so as of the collection date the weights cannot be treated as publicly downloadable.

Evaluation method

This is Kimi's release note and vendor-run evaluation summary, not an independent rerun. The body says K3.1 Pro is a post-training version of K3, using more post-training compute and joint RL across hundreds of downstream domains; it does not disclose complete prompts, per-question inputs, random seeds, failure logs, or confidence intervals.

• Coding: Terminal-Bench 4.0 uses the Claude-Code harness to test resource-constrained terminal coding agents and scores them strictly using task-specific tests. Kimi Code Bench 3.0 Long-horizon is an internal Kimi benchmark covering multistep execution, tool use, and end-to-end engineering problems.

• 3D: BenchCAD randomly samples a 1,000-item subset from the complete set of 17,900 Vision2Code samples. The with-tools condition uses the Claude Code harness; the without-tools condition uses the official prompt and scoring pipeline for single-turn generation; open models use the maximum reasoning setting.

• Finance: Each OfficeQA Pro v2 question provides a complete PDF corpus; the PDFs are rendered as images and do not provide machine-readable text. GPT-6 Astra uses Codex, while the other models use Claude Code. Frontier Finance uses the official finance agent v2 harness.

• Cross-source data: The GDPval-AA v2.1, AA-Briefcase v1.1, and AutomationBench-AA scores in the body are cited from Artificial Analysis. They are external records from an independent platform and do not represent results Kimi reran on this page.

Key results

Raw data

Public product and weight status

The article says K3.1 Pro and Highspeed were available on Kimi.ai, Kimi Work, Kimi Code, and the Kimi API on the day of release, and gives the API model IDs kimi-k3.1-pro / kimi-k3.1-pro-highspeed. The same paragraph states: “full model weights will be released by Nov 4”. Therefore, “available through the product or API” and “full weights have already been released publicly” are two different things; as of the collection date, only the former can be recorded, and the latter cannot be written as the current status.

Readable data from the 3D and finance charts

• The BenchCAD chart gives results for both with-tools and without-tools conditions. The with-tools group is Claude Opus 5.5 96.2, GPT-6 Astra 95.9, K3.1 Pro 92.7, Claude Fable 5.1 92.6, and DeepSeek-V4.1-Flash-1017 81.6; the without-tools group is Claude Opus 5.5 73.0, K3.1 Pro 72.8, Claude Fable 5.1 60.6, and DeepSeek-V4.1-Flash-1017 41.2.

• The finance chart gives K3.1 Pro 64.4 for OfficeQA Pro v2 and 58.6 for Frontier Finance; the internal charts give Financial Modeling 57.6 vs 50.4, Financial Research 70.4 vs 58.8, and Business Operation 37.0 vs 8.9 (K3.1 Pro vs K3).

• The Kimi Hallu Bench Hard chart lists K3.1 Pro's claim-level hallucination rate as 8.3; the body also states 23.4 for K3 and explicitly says these are deliberately stressful factuality tests.

Parameters and special conventions in the footnotes

• The default unified settings are max, temperature=1.0, and top_p=0.95.

• In PostTrainBench v1.2, K3.1 Pro and DeepSeek-V4.1-Flash-1017 were run internally using H20 rather than H100; the judge model was fixed as DeepSeek-V4.1-Flash-1017 at maximum effort. The remaining scores come from official leaderboards.

• BenchCAD with tools and without tools are two different run configurations and cannot be combined into one score; GPT-6 Astra on OfficeQA Pro v2 also uses a different Codex harness.

• BrowseComp is a special parameter exception: context compression is triggered at 200K tokens, with up to 500 steps, and uses temperature=1.0 and top_p=1.0.

• GDP.pdf with python, ZeroBench with python, and ChartoGraphy with python are averaged over five runs; other multimodal (with python) benchmarks are averaged over four runs. GDPval-AA, AA-Briefcase, and AutomationBench-AA are cited data from Artificial Analysis and cannot be interchanged with one or more runs of Kimi's internal benchmarks.

Conclusions and limitations

K3.1 Pro's official materials provide useful positioning signals in three areas: on coding, Terminal-Bench is close to Claude Opus 5.5; on 3D, BenchCAD with tools is 92.7; and on finance, OfficeQA Pro v2 is 64.4, with internal finance tasks clearly higher than K3. The most usable conclusion is that it is worth controlled trials in long-horizon coding, tool-assisted 3D, and finance-agent workflows, rather than that it leads across the board.

The following limitations must be retained:

• The Terminal-Bench comparison scores mix Kimi's self-evaluation, model release articles, and the highest values from official leaderboards; Kimi Code Bench, the internal finance benchmarks, and Hallu Bench have not been independently verified.

• BenchCAD uses a random subset of 1,000 from a complete dataset of 17,900, and its with-tools and without-tools harnesses differ; the result cannot be extrapolated to all 3D tasks or to tool-free scenarios.

• OfficeQA, Frontier Finance, and other agent results use different clients, tools, and evaluators; when comparing models across benchmarks, harness differences may affect results alongside model differences.

• Internal comparisons such as “7.2 points,” “11.6 points,” and “4.2 times” describe the release framing from K3 to K3.1 Pro; without per-question samples, variance, or failure types, they cannot establish a general improvement rate.

• Official charts, cases, and AA citations cannot replace independent testing; do not invent undisclosed per-question inputs, outputs, sample splits, or raw trajectories.

Reproduction notes

1. Fix the API model ID, client version, harness, tool schema, maximum steps, and timeout; use reasoning effort=max, temperature=1.0, and top_p=0.95 by default, and record top_p=1.0 and the context-compression rules separately for BrowseComp.

2. For coding tasks, distinguish at least Terminal-Bench, long-horizon repository tasks, and pure code generation; save per-question pass/fail status, tool calls, tokens, elapsed time, and the amount of human repair.

3. Rerun 3D tasks separately with and without tools, and record the sampling method for the 1,000-case subset, Voxel IoU, generation scripts, and final assets; do not combine the two sets of scores.

4. For finance tasks, separately record the PDF rendering method, retrieval sources, cross-document verification, numerical answers, and factual errors; report OfficeQA, Frontier Finance, and the internal finance tasks separately.

5. In a rerun, do not describe product/API availability as proof that the weights are public; recheck the weight status against the official Kimi article's plan for 2026-11-04.