LongCat reports that LongCat-2.5 outperformed LongCat-2.0 in its evaluations: 88.4 versus 70.8 on Terminal Bench v2.1, and 91.9 versus 79.9 on BrowseComp. These are vendor-reported results and do not replace independent retesting.
Tasks this is suitable for assessing: A preliminary comparison of coding agents, web information retrieval, automation, and vision-to-web benchmarks.
Tasks this should not be extrapolated to: Success rates across all real-world projects, operating costs, latency, or rankings under different evaluation frameworks.
Applicable model versions: LongCat-2.5; LongCat-2.0 is the comparison model.
Test environment or client: Claude Code, official evaluation code, or a lightweight in-house harness, depending on the benchmark; see below.
Reasoning tier and parameters: Unless otherwise specified, enable_thinking=on, temperature=1.0, top_p=0.95.
Terminal Bench v2.1: Claude Code harness v2.1.280, maximum sequence length 512K, averaged over three independent runs.
BrowseComp: A lightweight in-house harness with Verify and Discard-All; the latter has a 256K threshold. Up to 1,024 turns, with only search and browse_with_goal available.
AutomationBench public v1.0.6: All 600 publicly available questions, using the official evaluation code.
Vision2Web: Claude Code v2.1.7, temperature=1.0, top_p=0.95, top_k=50, sequence length 256K.
LongCat describes LongCat-2.5 as a native multimodal MoE model designed for long-horizon agent tasks, with 1.6 trillion total parameters and approximately 48 billion parameters activated per token. LongCat presented end-to-end demos of software development and professional office work, but did not provide the complete prompts or success rates for these demos.
| Benchmark | LongCat-2.5 | LongCat-2.0 | Notes |
|---|---|---|---|
| Terminal Bench v2.1 | 88.4 | 70.8 | Average of three independent runs |
| BrowseComp | 91.9 | 79.9 | In-house harness, limited to two tools |
| AutomationBench v1.0.6 | 51.8 | 15.8 | 600 publicly available questions |
| Vision2Web | 78.6 | Not listed | Vision-to-web |
The figures above are transcribed from the original table. The source does not fully disclose sample variance, all costs, or specific model snapshots for these entries.
These figures come from the model provider and are official benchmarks, not an independent evaluation. For competing models, the source cites public technical reports, blogs, or official leaderboards for scores marked with an asterisk; scores without an asterisk were measured by LongCat itself. Configurations may differ across sources, so these results should not be used to assert a general ranking. Vision tasks, real development repositories, and actual costs still need to be validated in the target environment.
Following the evaluation notes in the source, use the corresponding harness, versions, tools, and parameters, run the specified benchmarks, and report the sample size, number of runs, and cost. The source does not list every implementation detail, so the release post alone cannot guarantee exact reproduction.
LongCat 2.5
