Kimi K2.5 Official Release: Multimodality, Agent Swarm, and Coding Benchmarks

One-sentence takeaway

The official evaluation positions K2.5 as a natively visual model for coding and Agent Swarm workloads: its Thinking/tool configuration, context, and repeat counts are disclosed in considerable detail, but SWE and other results still depend on Moonshot's proprietary harness and should be considered separately from third-party deployments.

Test environment

• Model/mode: Kimi K2.5 Thinking; compared with GPT-5.2 xhigh, Claude Opus 4.5 extended thinking, Gemini 3 Pro high, and DeepSeek V3.2 thinking.

• General configuration: Kimi temperature=1.0, top_p=0.95, context 256k; HLE/AIME/HMMT/GPQA max completion 96k.

• Repeats: AIME/HMMT avg@32, GPQA avg@8, Vision avg@3, Agentic Search Seal-0/WideSearch avg@4, and coding tasks avg@5; the official SWE-Bench appendix notes that different tasks use an internal framework.

• Tools: search, code interpreter, and web browsing; HLE/Agentic Search use tools and context-management strategies.

Input/configuration

The official coding evaluation uses a minimal tool set including bash, createfile, insert, view, strreplace, and submit. The SWE series uses Moonshot's internal evaluation framework and tailored system prompts; Terminal-Bench 2.0 uses Terminus-2 and runs in non-thinking mode in that implementation.

Results

• HLE-Full: Thinking 30.1 without tools; 50.2 with tools.

• SWE-Bench Verified: 76.8; SWE-Pro: 50.7; SWE Multilingual: 73.0; Terminal Bench 2.0: 50.8.

• LiveCodeBench v6: 85.0; PaperBench: 63.5; CyberGym: 41.3; SciCode: 48.7.

• BrowseComp: 60.6; BrowseComp context management: 74.9; Agent Swarm: 78.4.

• WideSearch: 72.7; Agent Swarm: 79.0; DeepSearchQA: 77.1; AA-LCR: 70.0; LongBench v2: 61.0.

• Vision: MMMU-Pro 78.5, OCRBench 92.3, OmniDocBench 1.5 88.8, VideoMMMU 86.6, and LongVideoBench 79.8.

Conclusion

K2.5's core advantage is the combination of “vision + tools + parallel agents.” For coding, research retrieval, long-form video/document work, and large-scale searches that can be parallelized, the official results support considering it as an open-model candidate. Swarm scores should not be applied directly to tasks without tools or to strictly single-agent tasks.

Limitations

• The official results are self-reported; some benchmarks use internal harnesses, tailored system prompts, and non-public evaluations.

• The official K2.5 appendix explicitly says that some coding high scores come from non-thinking runs; the model name alone cannot be used to infer Thinking capability.

• The comparison models use different effort levels, tools, and service stability; the official report says GPT-5.2 xhigh had about 10% no-output failures, counted as errors.

• Agent Swarm results use separate parallelism limits and context strategies and are not equivalent to ordinary Chat Completions.

Reproduction steps

1. Fix K2.5 Thinking, temperature=1, top_p=.95, context 256k, and max tokens; save the system prompt and tool definitions.

2. Re-run the no-tool, tool, single-agent, Swarm, and vision tasks separately according to the official appendix; record repeat strategies such as avg@32/8/5.

3. Use the same tool set and parser for coding, and report Thinking/non-thinking separately.

4. Use the same context management, timeout, and cost accounting for GPT/Claude/DeepSeek, and compare success rate, latency, and tool errors.