On the independent FrontierCode Extended benchmark built by the Devin team for real-world software engineering tasks, Kimi K2.7 Code achieved a 39.5% pass rate, placing it firmly in the competitive tier alongside top-tier proprietary models. It excels at generating standalone UI components and self-contained features, but remains constrained by its context window and memory span during long-sequence multi-file refactoring.
Evaluation Benchmark: FrontierCode Extended (a comprehensive benchmark suite by the Devin / Cognition team designed to evaluate real-world end-to-end software engineering tasks) .
Execution Environment: Real agent execution environments across Devin Desktop and Devin CLI.
Model Comparison Group: Claude Opus 4.8, GPT-5.5, GLM 5.2, Kimi K2.7 Code.
Client Constraints: During testing, the Devin client capped the allocated context window for K2.7 / GLM 5.2 at approximately 200K tokens.
FrontierCode Extended official published results (Pass Rate) :
| Model | Model Type | FrontierCode Extended Score |
|---|---|---|
| Claude Opus 4.8 | Frontier Proprietary Flagship | 51.8% |
| GPT-5.5 | Frontier Proprietary Flagship | 44.8% |
| GLM 5.2 | Open-Weight MoE | 43.0% |
| Kimi K2.7 Code | Open-Weight MoE | 39.5% |
Community developer feedback distribution in real-world engineering:
Self-Contained Features / UI Tasks: Kimi K2.7 generates code exceptionally fast with snappy responsiveness and high out-of-the-box usability.
**Large-Scale Long-Horizon Refactoring (>15 consecutive files modified) **: Performance degrades when the agent must maintain precise memory of changes in files 1–8 while working through files 9–15.
Open-Source Models Enter the Top Tier: Kimi K2.7 Code (39.5%) and GLM 5.2 (43.0%) demonstrated problem-solving capabilities on real-world complex engineering tasks that approach top-tier proprietary models (GPT-5.5 at 44.8%) .
Task Suitability Divide:
High-Win Scenarios: End-to-end single-feature development, standalone module refactoring, UI/frontend page implementation, and unit test generation.
Challenging Scenarios: Long-horizon cascading refactoring involving more than 15 interrelated files and synchronized dependency updates across multiple repositories.
FrontierCode Extended incorporates Devin platform-specific agent toolsets and execution feedback mechanisms; switching to alternative agent frameworks (such as SWE-agent or Aider) may produce different pass rates.
The client-side limit of a 200K context during testing prevented the model from fully leveraging its native 256K token potential.
Select Kimi K2.7 Code as the underlying execution model in Devin Desktop / CLI.
Configure two sets of engineering tasks: standalone feature creation and multi-file refactoring.
Record the number of execution turns, tool invocation success rate, final build/test pass status, and token consumption.
Devin team engineer theodormarcu published the specific percentage scores above in an official announcement, confirming FrontierCode Extended as the comparative benchmark.
Official Announcement: "GLM 5.2 scores 43.0% and Kimi K2.7 scores 39.5% on FrontierCode Extended — placing them in a competitive tier alongside GPT-5.5 (44.8%) and Claude Opus 4.8 (51.8%) ."
Senior Community Developer Observation: "200k covers most single-feature work but shows up on large refactors — when you're touching 15 files in sequence and need the model to hold what changed in files 1-8 while working on 9-15, the ceiling matters."
Kimi K2.7 Code
