Technical report offers many numbers, but cross-harness ranking is unsafe

Test environment

• Kimi K3: 2.8T total parameters, 104B activated, native vision, 1M context.

• Main-table baselines: Claude Fable 5, GPT-5.6 Sol, Claude Opus 4.8, GPT-5.5, and GLM-5.2.

• Default evaluation: K3 reasoning_effort=max, temperature=1.0; top-p=0.95 for single-step knowledge/vision tasks and top-p=1.0 for agent tasks.

• Harnesses: Kimi Code, Claude Code, and Codex; PostTrainBench is averaged over three runs on H20, ZeroBench over five, and most other vision tasks over three runs.

Input/configuration

• BrowseComp uses 300K tokens to trigger context compaction; with the full 1M context and no management, the reported score is 90.4.

• OfficeQA Pro renders all PDFs as images and does not provide machine-readable text.

• MCP-Atlas uses 500 public tasks with a 100-turn limit and Gemini 3.1 Pro as the judge; AutomationBench uses 600 public tasks.

• The report explicitly discloses that Fable 5 has fallback and GPT-5.6 Sol may trigger cyberguard, so the table is not a fully same-condition blind test.

Result data

The report also gives an internal Kimi Webdev Bench: using the same Claude Code harness as Opus 4.8, blind review produced an overall Win of 58.6%, Tie of 13.8%, and Lose of 27.6%, with Win-Lose at +31.0; this is an internal comparison, not a public benchmark.

Conclusion

The technical report supports placing K3 in the frontier-competitive range for long-horizon coding, web/tool agents, and Python-assisted vision tasks, while it still trails top closed models on knowledge-work tasks such as HLE, GDPval/AA-Briefcase, and OfficeQA. The report's greatest reuse value is that it spells out reasoning, top-p, harness, run count, and tool conditions, making a same-configuration local pilot possible.

Limitations

• The main table is published by the model provider; internal benchmarks and cases cannot be treated as independent verification.

• Different models use different harnesses, fallback behavior, guards, and hardware; scores cannot simply be treated as a fair competition.

• Some third-party scores are snapshots cited from other organizations; version dates can change Elo and leaderboard standings.

• A 1M context, tool augmentation, and agent loops can all substantially change token usage, cost, and success rate.

Reproduction steps

1. Fix the Kimi K3 version, max, and temperature 1.0, and set top-p to 0.95/1.0 by task type.

2. Record the Kimi Code/Claude Code/Codex harness, tool permissions, context-compaction threshold, and hardware.

3. Run public benchmarks at least three times according to the official version; for vision tasks record whether Python is enabled, and run ZeroBench five times.

4. Store each benchmark's inputs, stopping rules, failures/refusals, tokens, cost, and results separately; do not merge figures from different harnesses into a single conclusion.

Original evidence and data

The technical report publicly provides the main table, evaluation configuration, third-party citation notes, internal benchmark definitions, tool conditions, and result explanations. These fields are sufficient to verify the “comparability boundaries caused by configuration,” but insufficient to reproduce every private internal task.

Source excerpt or observation (compliance short quote only)

The report's table header explicitly says: “All maxed out on thinking effort: max or xhigh.”