Kimi K3 · Media / benchmark · Editorial analysis
Try Friday’s comparison of Grok 4.6 and Kimi K3 pricing, context, and deployment conditions.
Comparison targets: Grok 4.6 and Kimi K3; the article uses both vendors’ official releases and developer documentation.
Kimi K3: 2.8T sparse MoE, 1M context, open weights, $3/M input, $15/M output, and $0.30/M cached input.
Grok 4.6: 500K context; $2/$6 below 200K and $4/$12 above 200K, with $0.30/M cached input.
The article explicitly cautions that there is no independent head-to-head under the same prompt, scaffold, reasoning budget, and model snapshot.
K3’s vendor-reported agent scores: Terminal-Bench 2.1 88.3, FrontierSWE 81.2, Program Bench 77.8, and DeepSWE 67.5, all in the Moonshot harness/max-mode context.
The article’s proposed reusable comparison design: 20 routine, 20 difficult, and 10 known-failure tasks; the same tool permissions and stopping rules; blind review of correctness, unnecessary edits, citations, and maintainability; and measurement of cost and p50/p95 latency for each accepted result.
| Workload | Article’s suggested routing | Rationale/boundary |
|---|---|---|
| Cost-sensitive coding agent below 200K | Grok 4.6 | Lower unit price, but task success rate was not measured by this article |
| High throughput, low latency | Grok 4.6 | xAI positioning and pricing advantage; requires self-testing |
| Long-horizon agent-coded benchmark | Kimi K3 | Stronger public coding-agent scores |
| Self-hosted/open weights | Kimi K3 | K3 weights have been released; Grok is closed-source |
| Video, multimodal, or ultra-long corpus | Kimi K3 | Advantages in 1M context and video input |
| Context above 200K | Test both | Grok moves into a higher pricing tier, narrowing the gap |
This comparison is better treated as a routing-experiment design than as proof of “which is stronger”: K3’s open weights, 1M context, and long-horizon coding evidence correspond to capability and deployment choices, while Grok 4.6’s lower price and speed correspond to cost choices. The final routing decision should use the cost and quality of each task’s accepted result.
“Grok 4.6 matches K3 with roughly half the parameters” is xAI positioning, not an independent benchmark result.
K3’s benchmark table comes from Moonshot; Grok has no scores on the same Terminal-Bench/FrontierSWE/ProgramBench/DeepSWE set.
Prices are token list prices; retries, tool calls, output length, context billing, and success rate will change per-task cost.
The article’s 20/20/10 split is a proposed reproduction design, not an experiment the authors ran.
Prepare 20 routine tasks, 20 difficult tasks, and 10 known-failure tasks, preserving repository/data snapshots.
Run both models under the same harness, tool permissions, stopping rules, and context strategy.
Blind-review correctness, unnecessary edits, citations, and maintainability; also record tokens, retries, p50/p95 latency, and accepted-result cost.
Analyze routing separately for <200K and ≥200K context, then assign models by task type rather than by the overall average.
The original article fully lists the two models’ pricing, context, and deployment differences; K3’s four vendor benchmark figures; and the 20+20+10 reproduction steps. It repeatedly cautions that different harnesses are not directly comparable.
The article states: “Benchmark figures are labeled as vendor-reported where applicable.”
The figures, task set, reasoning tier, and client conditions apply only to the listed source and collection snapshot. Different versions, harnesses, or providers must not be compared directly; undisclosed parameters remain unknown.
For a reproduction, fix the model version, provider or client, reasoning tier, tools, task-set version, sample count, and collection date, and record failures, retries, and human corrections. Full steps are in the source notes below.
Try Friday AI Research · Friday AI Research · Original publication date 2026-08-12 · Site edit date 2026-09-20
Open original sourceKimi K3
Download the Tabbit client to check model access
