Inferwerk reports that LongCat 2.5 (max) met 79.6% of detail checks in its EvalMapBench test on 2026-11-02; 51 of 129 trials met every check. The result reflects performance only on this private test set.
Tasks this can help assess: Whether a model can find and follow details in a small codebase that are not individually listed in a ticket during maintenance work.
Tasks this should not be generalized to: Software engineering tasks as a whole, complex problem-solving, or overall ability in other Agent environments.
Applicable model version: The original post labels it LongCat 2.5 (max); it does not specify an exact model ID or whether this is a production release or Preview, so that cannot be determined from the post.
Test environment or client: The linked methodology page identifies Terminus-2 within Harbor 0.24 and says the reasoning from each turn is passed back to subsequent turns.
Reasoning tier and parameters: max; other parameters are not specified.
The original post describes EvalMapBench as a test of attention to detail. Its linked methodology page describes 43 short maintenance tickets across two private codebases. Each ticket was run 3 times, with a 20-minute limit per run, for 129 trials in total; hidden validators checked the results. The page says only aggregate data is public; the tickets, codebases, and checks are not disclosed, so the test cannot be fully reproduced externally.
Detail checks passed: 79.6%, with a reported 95% confidence interval of 75.1%–84.2%.
51 of 129 trials met every check, or about 39.5%; this is not the detail-check pass rate above.
The original post reports $0.10 per trial and a median duration of 18.1 minutes; these should not be treated as general cost or latency figures for other workloads.
The image attached to the original post gives the test date as 2026-11-02 and the number of trials as 129, and shows 79.6% with a 95% confidence interval of 75.1–84.2. The linked methodology page adds that the median output was 48k tokens for that row, with reasoning passed back each turn at runtime. The page lists an output-token price of $1.2 per million and notes that this was a limited-time discounted price on 2026-11-02. The price and the actual cost per trial are different metrics.
This is an independent tester's aggregate report on its own benchmark, not an official model result. The private tasks came from two codebases belonging to one developer and were small; they cannot represent all maintenance work. The confidence interval applies to this set of trials. It cannot establish how the model performs on other tasks or in other environments, and similar scores alone should not be used to rank models. The original post does not disclose whether the model was a production release or Preview, so the version status remains unknown.
The linked methodology page can be used to check the publicly described framework, number of runs, and time limit. Because the tickets, repositories, and validation checks are not public, the score cannot be independently reproduced. The original post does not disclose the full model ID, call parameters, or per-trial results.
LongCat 2.5
