Tabbit
ResourcesBlogModels
Tabbit LogoTabbit

Tabbit — The AI Browser that Works for You

Topics

  • Best Browsers 2026: Rankings and Top Lists
  • AI Browser Guides and Resources
  • Browser Comparisons: Head-to-Head Guides
  • Browser Reviews and Explainers
  • Browser Alternatives
  • Browser Directory: Guides by Browser
  • Agentic Browser Resources
  • Browser Downloads and Install Guides
  • Browser Productivity Resources
  • SillyTavern Model Guides and Presets

Popular Guides

  • Best Browsers 2026: Top 10 Ranked & Compared
  • Best Free AI Browsers 2026: Plans & Prices Compared
  • Rabbit Browser: What It Is & Why Tabbit Is Better
  • Firefox vs Chrome 2026: Privacy, RAM & Speed Compared
  • Best Lightweight Browsers 2026: 7 Tested for RAM & Speed
  • AI Browser List 2026: 10 Browsers Compared
  • Neobrowser: Norton Neo, Download & Alternatives
  • Edge vs Chrome 2026: Speed, RAM & Battery Tested
  • Best Internet Browser for Android 2026
  • Best AI Browser 2026: Top 9 Tested & Ranked

Events

  • Tabbit Skill Competition
  • KPOP SBTI Fandom Personality Test
  • Tabbit Campus Creator Program
  • fifi's Picks: AI Skills for Research Papers
  • User Survey

About

  • Tabbit Blog
  • Press & Media
English
简体中文English
Review
MediaGemini 4 Carbon

FrontierCode 2.0: Gemini 4 Carbon Scores and Cost per Run

Original source

Devin / Cognition official blogs

AuthorCognition. The methodology article is credited to The Cognition Team

Source date2026-11-10

Tabbit curation2026-11-10

Read original

One-sentence takeaway

In its Devin announcement on 2026-11-10, Cognition reported a FrontierCode 2.0 score of 40.0% for Gemini 4 Carbon, above Gemini 4 Argon's 26.3% and GPT-6.1 Sol's 39.0%. The announcement said Carbon at its low reasoning level had already surpassed Argon's best result, and that Carbon's highest-scoring configuration cost less than two-thirds as much per task as Argon. A separate methodology article lists Carbon at 42.6%. The scores differ, so they should be treated as separate result snapshots tied to their respective publication dates.

Use cases

  • Tasks this can help assess: Compare models on Cognition FrontierCode 2.0 engineering tasks and its code-merge quality scoring.

  • Tasks this should not be generalized to: These results do not establish real-world success rates across arbitrary repositories, languages, agents, or clients. Scores from different versions or evaluation snapshots should not be combined directly.

  • Applicable model version: Gemini 4 Carbon as named in the announcement. The underlying model ID or revision is not specified.

  • Test environment or client: Cognition FrontierCode 2.0. The announcement says Carbon is available in Devin Desktop and Devin CLI, but does not specify which client version or execution configuration was used for this result.

  • Reasoning levels and parameters: The announcement only says Carbon at its low reasoning level exceeded Argon's best score. It does not provide Carbon's scores at each level or its parameters. The methodology article says its leaderboard shows each model's best reasoning level.

Evaluation method

FrontierCode 2.0 evaluates how well models handle real-world engineering tasks and whether their solutions are mergeable. Scores are weighted aggregates of scoring criteria. A solution that fails a blocking criterion receives a score of 0. The methodology article says the tasks were designed by more than 20 open-source developers, each spending over 40 hours per task. Compared with FrontierCode 1.1, the tasks cover 55% more repositories and include 43% more blocking criteria. The reference solutions modify 67% more files on average and contain 89% more lines of code, while the codebases used are 25% larger.

Cost is defined as the average dollar cost per rollout, calculated using public API list prices and including publicly available discounts. The announcement does not give exact costs for Carbon or Argon. It only reports that Carbon's highest-scoring configuration costs less than two-thirds as much per task as Argon's.

Key results

Source snapshotModelFrontierCode 2.0 scoreCost information
Devin announcement, 2026-11-10Gemini 4 Carbon40.0%Cost per task for the highest-scoring configuration is less than two-thirds of Argon's. Exact value not specified
Devin announcement, 2026-11-10Gemini 4 Argon26.3%Carbon's highest-scoring configuration costs less than two-thirds as much. Exact value not specified
Devin announcement, 2026-11-10GPT-6.1 Sol39.0%Not specified
FrontierCode methodology article, 2026-10-27Gemini 4 Carbon42.6%The chart includes a cost axis and model points, but does not label Carbon's exact value

The Devin announcement also says Carbon at its low reasoning level had surpassed Argon's best result, but does not publish the specific score for that level. The two articles report Carbon scores of 40.0% and 42.6%, respectively, and do not explain the difference. They should not be combined into a single score.

Raw data

  • Scoring basis: Weighted aggregate of scoring criteria. A solution that fails a blocking criterion receives 0 points.

  • Cost unit and basis: USD per rollout. Average cost per run is calculated using public API list prices and includes publicly available discounts. The Devin announcement describes this as cost per task.

  • Reasoning levels: The methodology article's overall leaderboard uses each model's best reasoning level. The Devin announcement only makes a qualitative comparison, saying Carbon at its low level surpassed Argon's best result, and does not disclose scores by level.

  • Sample size and error: The number of tasks, runs per model, margin of error, and per-task scores are not specified.

  • Benchmark scope: The methodology article says FrontierCode 2.0 expanded the repositories and task categories and strengthened blocking criteria for correctness, design, completeness, and other qualities. The results measure performance on engineering tasks as defined by this benchmark and do not represent general success rates in production environments.

Conclusions and limitations

Cognition's announcement reports Carbon at 40.0%, 13.7 percentage points above Argon at 26.3% in the same announcement and slightly above GPT-6.1 Sol at 39.0%. Cognition also says Carbon's highest-scoring configuration costs less than two-thirds as much as Argon, but does not disclose absolute costs that could be independently checked. The claim that Carbon at its low reasoning level beats Argon's best level is qualitative and does not establish the score gap between levels.

The methodology article and announcement differ by 2.6 percentage points on Carbon's score, with no explanation given. Results should retain their publication dates and sources. The lack of sample sizes, repeat counts, and confidence intervals also limits conclusions about small ranking differences. These conclusions apply only to Cognition's published FrontierCode 2.0 evaluation methodology.

Reproduction notes

Review Cognition's Devin announcement and the FrontierCode 2.0 methodology article, recording the publication date, score, reasoning-level description, and cost basis for each. The public pages do not provide the task set, model execution configuration, repeat counts, or Carbon's exact cost, so the scores and cost ratio cannot be independently recalculated.

Curated by Tabbit

This is a third-party source navigator. Model versions, test environments, and personal experience vary; consult the original source.

Gemini 4 Carbon

Use and compare models in Tabbit

Gemini 4 Carbon

Related reviews

OfficialGoogle DeepMind

Gemini 4 Carbon Model Card: Knowledge Work and Multimodal Benchmarks and Their Limits

MediaArtificial Analysis

Gemini 4 Carbon: Artificial Analysis Reasoning Tiers and Index Costs

MediaZapier

Gemini 4 Carbon on Zapier AutomationBench: Business Workflow Results

MediaAI IQ

Gemini 4 Carbon's Composite Score and Benchmark Coverage on AI IQ

Gemini 4 Carbon

Related showcases & prompts

OfficialGoogle AI Studio / Gemini API

Google AI API Specifications for Gemini 4 Carbon

OfficialGoogle AI Studio / Gemini API

Gemini 4 Carbon Long Outputs: Continuation and Stop Conditions

OfficialGoogle AI Studio2026-11-10

Google AI Studio: Prompting Gemini 4 Carbon for Complex Tasks

OfficialGoogle AI Studio / Gemini API

Google AI Studio: Gemini 4 Carbon Agentic Video Understanding Configuration