In its Devin announcement on 2026-11-10, Cognition reported a FrontierCode 2.0 score of 40.0% for Gemini 4 Carbon, above Gemini 4 Argon's 26.3% and GPT-6.1 Sol's 39.0%. The announcement said Carbon at its low reasoning level had already surpassed Argon's best result, and that Carbon's highest-scoring configuration cost less than two-thirds as much per task as Argon. A separate methodology article lists Carbon at 42.6%. The scores differ, so they should be treated as separate result snapshots tied to their respective publication dates.
Tasks this can help assess: Compare models on Cognition FrontierCode 2.0 engineering tasks and its code-merge quality scoring.
Tasks this should not be generalized to: These results do not establish real-world success rates across arbitrary repositories, languages, agents, or clients. Scores from different versions or evaluation snapshots should not be combined directly.
Applicable model version: Gemini 4 Carbon as named in the announcement. The underlying model ID or revision is not specified.
Test environment or client: Cognition FrontierCode 2.0. The announcement says Carbon is available in Devin Desktop and Devin CLI, but does not specify which client version or execution configuration was used for this result.
Reasoning levels and parameters: The announcement only says Carbon at its low reasoning level exceeded Argon's best score. It does not provide Carbon's scores at each level or its parameters. The methodology article says its leaderboard shows each model's best reasoning level.
FrontierCode 2.0 evaluates how well models handle real-world engineering tasks and whether their solutions are mergeable. Scores are weighted aggregates of scoring criteria. A solution that fails a blocking criterion receives a score of 0. The methodology article says the tasks were designed by more than 20 open-source developers, each spending over 40 hours per task. Compared with FrontierCode 1.1, the tasks cover 55% more repositories and include 43% more blocking criteria. The reference solutions modify 67% more files on average and contain 89% more lines of code, while the codebases used are 25% larger.
Cost is defined as the average dollar cost per rollout, calculated using public API list prices and including publicly available discounts. The announcement does not give exact costs for Carbon or Argon. It only reports that Carbon's highest-scoring configuration costs less than two-thirds as much per task as Argon's.
| Source snapshot | Model | FrontierCode 2.0 score | Cost information |
|---|---|---|---|
| Devin announcement, 2026-11-10 | Gemini 4 Carbon | 40.0% | Cost per task for the highest-scoring configuration is less than two-thirds of Argon's. Exact value not specified |
| Devin announcement, 2026-11-10 | Gemini 4 Argon | 26.3% | Carbon's highest-scoring configuration costs less than two-thirds as much. Exact value not specified |
| Devin announcement, 2026-11-10 | GPT-6.1 Sol | 39.0% | Not specified |
| FrontierCode methodology article, 2026-10-27 | Gemini 4 Carbon | 42.6% | The chart includes a cost axis and model points, but does not label Carbon's exact value |
The Devin announcement also says Carbon at its low reasoning level had surpassed Argon's best result, but does not publish the specific score for that level. The two articles report Carbon scores of 40.0% and 42.6%, respectively, and do not explain the difference. They should not be combined into a single score.
Scoring basis: Weighted aggregate of scoring criteria. A solution that fails a blocking criterion receives 0 points.
Cost unit and basis: USD per rollout. Average cost per run is calculated using public API list prices and includes publicly available discounts. The Devin announcement describes this as cost per task.
Reasoning levels: The methodology article's overall leaderboard uses each model's best reasoning level. The Devin announcement only makes a qualitative comparison, saying Carbon at its low level surpassed Argon's best result, and does not disclose scores by level.
Sample size and error: The number of tasks, runs per model, margin of error, and per-task scores are not specified.
Benchmark scope: The methodology article says FrontierCode 2.0 expanded the repositories and task categories and strengthened blocking criteria for correctness, design, completeness, and other qualities. The results measure performance on engineering tasks as defined by this benchmark and do not represent general success rates in production environments.
Cognition's announcement reports Carbon at 40.0%, 13.7 percentage points above Argon at 26.3% in the same announcement and slightly above GPT-6.1 Sol at 39.0%. Cognition also says Carbon's highest-scoring configuration costs less than two-thirds as much as Argon, but does not disclose absolute costs that could be independently checked. The claim that Carbon at its low reasoning level beats Argon's best level is qualitative and does not establish the score gap between levels.
The methodology article and announcement differ by 2.6 percentage points on Carbon's score, with no explanation given. Results should retain their publication dates and sources. The lack of sample sizes, repeat counts, and confidence intervals also limits conclusions about small ranking differences. These conclusions apply only to Cognition's published FrontierCode 2.0 evaluation methodology.
Review Cognition's Devin announcement and the FrontierCode 2.0 methodology article, recording the publication date, score, reasoning-level description, and cost basis for each. The public pages do not provide the task set, model execution configuration, repeat counts, or Carbon's exact cost, so the scores and cost ratio cannot be independently recalculated.
Gemini 4 Carbon