BenchLM: Kimi K3’s Publicly Verifiable Benchmark Ledger

Test environment

• BenchLM model profile and benchmark ledger; the page places each row’s score, comparison, weight, cohort, and evidence status together.

• The page’s data is current through 2026-08-17 and covers a ranking view of 218 models.

• Pricing: $3/M input, $15/M output, and $0.30/M cached input.

Input/configuration

• The page shows 44 benchmark rows with displayable sources; evidence statuses include Verified, Mixed sources, Provider exact, Benchmark exact, and Secondary exact.

• Many of K3’s coding/agentic figures come from Kimi’s official published tables. BenchLM is an evidence ledger and cross-model comparison layer, not a rerun using a unified harness.

Results

Key ledger rows include: DeepSWE 67.5 (below GPT-5.6 Sol at 72.7), Terminal-Bench 2.1 88.3 (below Sol at 91.9), BrowseComp 91.2 (below Sol at 92.2), AutomationBench 30.8 (below GLM-5.3 at 48.2), APEX-Agents 37.6 (below Grok 4.6 at 57.5), MathVision 94.3/97.8 (without tools/with Python), and ExploitBench 32 (with a source chain pointing to NIST/UK AISI/CAISI).

Conclusion

BenchLM’s value is not announcing another “first”, but breaking K3’s ranking down into traceable sources: it is in the top five for Agentic, Coding, Knowledge, and Multimodal. However, mixing Provider exact, Benchmark exact, and Verified figures can easily lead readers to misinterpret all of them as independently rerun results.

Limitations

• The page does not provide K3’s unified inputs, complete harness, run logs, or raw outputs; it cannot replace an independent controlled evaluation.

• Many of the 44 rows are provider exact, and some categories are entirely unmeasured; the overall score does not cover all capabilities.

• Rankings and Elo/source records change as new models, sources, and benchmark versions are added or updated; the collection date should be retained.

Reproduction steps

1. Save a page snapshot and each row’s source link, evidence status, and date.

2. Use only Verified or Benchmark exact rows as candidates for review, opening the original leaderboard/report for each one.

3. Build a local held-out set with the same categories for the actual selection task, and report K3 and baseline results under the same harness.

4. Do not write Provider exact figures as independent experimental results; retain the original evidence labels.

Original evidence and data

The page publishes the overall score of 80.5, #5/218, 44 source-displayable rows, category scores/rankings, pricing, and the comparison and evidence type for each benchmark; it links to Kimi’s official blog, NIST/UK AISI/CAISI, and individual benchmark pages.

Source excerpt or observation (short compliance quote only)

The page’s evidence summary is: “Evidence: Supported; this profile shows 44 source-displayable benchmark rows.”