Unsiloed Benchmark: Kimi K2.7 Code vs GLM 5.2 Controlled Benchmark on Real-World Code Generation and Large Repository Analysis

One-Sentence Takeaway

In strictly controlled tests using identical prompts, Kimi K2.7 beat GLM 5.2 (48/60) with a score of 53/60 in scaffolding a runnable greenfield project (FastAPI) thanks to complete components and zero missing dependencies; meanwhile, in deconstructing a massive repository (Saleor) end-to-end, GLM 5.2 came out on top by leveraging its 1M context window to unearth deeper implementation details.

Test Environment, Inputs / Configuration

• Evaluated Models:

  • Kimi K2.7 Code: MoE 1T total parameters / 32B active, 256K context window, official API pricing at $0.95 input ($0.19 cached) / $4.00 output per 1M tokens.

  • GLM 5.2: MoE 744B–753B total parameters / 40B active, 1M context window, official API pricing at $1.40 input ($0.26 cached) / $4.40 output per 1M tokens.

• Task Design:

  • Task 1 (Feature Implementation) : Build a complete FastAPI + PostgreSQL task management API from scratch (JWT, SQLAlchemy, CRUD, layered architecture, input validation) .

  • Task 2 (Repository Analysis) : Analyze the production-grade open-source e-commerce platform Saleor (Python/Django/GraphQL) codebase, delivering architecture breakdown, request trace flows, technical debt, and improvement recommendations.

Results Data

Task 1: FastAPI From-Scratch Implementation Scoring Matrix

Task 2: Saleor Large Repository Analysis Performance

Conclusions

1. Kimi K2.7 is the top pick for code implementation generation: For hands-on coding, scaffolding new projects, and implementing modules, Kimi K2.7 produces significantly higher code completeness. It rarely suffers from fatal omissions where code looks plausible but lacks key definitions, keeping human intervention and fix costs minimal.

2. GLM 5.2 is the top pick for long-context architecture exploration: When surveying monorepo-scale codebases and tracing complex call chains across dozens of files, GLM 5.2 demonstrates deeper cross-file code reasoning powered by its 1M context window and Sparse Attention.

3. Cost-performance advantage: Kimi K2.7's official API unit pricing is approximately 10%–30% lower than GLM 5.2, offering superior cost-efficiency for code generation and iterative Agent loops.

Limitations

• This benchmark relied on single-turn zero-shot / few-shot prompt comparisons and did not evaluate final convergence performance in multi-turn Agent self-correction loops (e.g., self-running pytest to resolve missing components) .

• Pricing comparisons are based solely on official standard API rates and do not account for third-party aggregators or specific subscription plan discounts.

Reproduction Steps

1. Call the Kimi K2.7 Code and GLM 5.2 APIs using identical prompt templates.

2. Write the files generated in Task 1 to a local directory, run pip install -r requirements.txt && uvicorn app.main:app, and verify whether the service starts cleanly without errors and successfully serves requests via Swagger UI.

3. Compare the depth of request flow tracing in Task 2 to verify whether it captures GraphQL resolve and DataLoader execution logic.

Raw Evidence & Data

The original Unsiloed AI article provides full input prompts for both tasks, scoring rubrics, model output defect logs, and a detailed architectural parameter comparison table.

Source Excerpts or Observations (Compliance Short Quote Only)

• The evaluation notes: “Kimi K2.7 seems better suited to implementation-heavy tasks where you want the model to generate working files with fewer missing components. GLM 5.2 seems better suited to codebase exploration, architectural reviews, dependency tracing.”