Official cases show long-horizon potential and reproducibility limits

One-sentence takeaway

The official materials position Kimi K3 as a 2.8T/104B-active model suited to long-running coding, visual feedback loops, knowledge work, and research-oriented agents, while explicitly acknowledging that it still trails Claude Fable 5 and GPT-5.6 Sol overall.

Test environment

• Model: Kimi K3, 2.8T parameters, 16/896 expert activation, 1M context, native vision.

• Official products: Kimi.com, Kimi Work, Kimi Code, Kimi API.

• Default settings: max thinking at release; the official footnote says evaluations used max, temperature=1.0, and top-p=1.0.

• API pricing: $0.30/M for cache-hit input, $3/M for cache-miss input, and $15/M for output.

Input/configuration

• Long-running agents must retain the complete thinking history; the official recommendation is to use a compatible harness such as Kimi Code and not switch models mid-session.

• The official recommendation for overly proactive behavior is to add a more explicit system prompt or AGENTS.md boundaries.

• In Kimi Code, select Kimi K3 with /model; the official API model name is kimi-k3.

Result data

• GPU kernel optimization: The official materials say K3 is competitive with Fable 5 with fallback and clearly exceeds Opus 4.8, GPT-5.6 Sol, and GPT-5.5; each model was given up to 24 hours in the same sandbox to optimize, rewrite, and benchmark four tasks.

• MiniTriton: The official case study says the model built a complete compiler from MLIR/IR to PTX codegen, reaching or exceeding Triton/torch.compile on some roofline workloads.

• Chip design: The official materials say one 48-hour autonomous run completed design, optimization, and verification, with simulation reaching 100 MHz and 8,700+ tokens/s; these are case-study figures from the release materials.

• Research workflow: The official materials say it reproduced I–Love–Q in about two hours, a task that normally takes 1–2 weeks, reviewing/cross-checking 20+ papers, evaluating 300+ EOS, and generating 3,000+ lines of Python plus an interactive dashboard.

• Knowledge work: One ASIC research case used 120+ rounds of recursive improvement, 2.8k+ web searches/fetches, 1.1k+ terminal data pulls, 11k+ pages, 87 quarterly reports, and 99 original PDFs.

• Multimodal creation: The official materials show screenshot-loop cases for games, frontend work, and CAD, as well as editing and multi-round revision from 56 source clips.

Conclusion

These cases support treating K3 as a candidate for “long-running, tool-intensive tasks that require vision or large-document context”; they do not support the conclusion that it outperforms closed models on every task. The official materials themselves place overall performance behind Fable 5 and GPT-5.6 Sol and point out that the user experience still has gaps.

Limitations

• The page is primarily official case-study narrative and does not publish complete inputs, random seeds, tool logs, failure rates, or downloadable artifacts for each case.

• Different benchmarks use different harnesses, including Kimi Code, Claude Code, and Codex; the official figures should not be treated as same-harness comparisons.

• Max thinking and high token usage may cause high latency and cost; when the model is overly proactive, it may make decisions on the user's behalf under ambiguous instructions.

• Evaluation cases are not production SLAs; enterprises should run pilots against their own repositories, data, and acceptance criteria.

Reproduction steps

1. Select kimi-k3 in Kimi Code or the official API, and fix the harness, model version, temperature, top-p, tool permissions, and stopping conditions.

2. Choose a real long-running task, such as a repository-level fix, screenshot-driven UI repair, or multi-document research; save the initial inputs and data snapshot.

3. Require the complete assistant reasoning/tool-call history to be retained, and set explicit permission boundaries for overly proactive behavior.

4. Record success rate, number of tool calls, total output tokens, elapsed time, amount of human editing, and deliverable quality; compare the same task with a known baseline model.

Original evidence and data

Verifiable fields on the official page include the model architecture, API pricing, default thinking mode, case-study runtimes/round counts/file counts, and limitations concerning thinking history, overly proactive behavior, and gaps in user experience.

Source excerpt or observation (compliance short quote only)

The official conclusion is: “While its overall performance still trails the most powerful proprietary models”.