Tabbit
ResourcesBlogModels
Tabbit LogoTabbit

Tabbit — The AI Browser that Works for You

Topics

  • Best Browsers 2026: Rankings and Top Lists
  • AI Browser Guides and Resources
  • Browser Comparisons: Head-to-Head Guides
  • Browser Reviews and Explainers
  • Browser Alternatives
  • Browser Directory: Guides by Browser
  • Agentic Browser Resources
  • Browser Downloads and Install Guides
  • Browser Productivity Resources
  • SillyTavern Model Guides and Presets

Popular Guides

  • Best Browsers 2026: Top 10 Ranked & Compared
  • Best Free AI Browsers 2026: Plans & Prices Compared
  • Rabbit Browser: What It Is & Why Tabbit Is Better
  • Firefox vs Chrome 2026: Privacy, RAM & Speed Compared
  • Best Lightweight Browsers 2026: 7 Tested for RAM & Speed
  • AI Browser List 2026: 10 Browsers Compared
  • Neobrowser: Norton Neo, Download & Alternatives
  • Edge vs Chrome 2026: Speed, RAM & Battery Tested
  • Best Internet Browser for Android 2026
  • Best AI Browser 2026: Top 9 Tested & Ranked

Events

  • Tabbit Skill Competition
  • KPOP SBTI Fandom Personality Test
  • Tabbit Campus Creator Program
  • fifi's Picks: AI Skills for Research Papers
  • User Survey

About

  • Tabbit Blog
  • Press & Media
Review
MediaGemini 4 Neon

Gemini 4 Neon pass rates and costs across four reasoning tiers on Zapier AutomationBench

Original source

Zapier

AuthorZapier (no individual author listed)

Tabbit curation2026-11-19

Read original

One-sentence takeaway

Across Zapier's 657 tasks, XHigh passed 13 more in total than High, raising the pass rate from 54.3% to 56.3% and the estimated cost per attempt from $0.88 to $1.69. Results by effort

Use cases

  • What this evaluation can assess: Pass rates and costs across four reasoning tiers for cross-app business workflows that can be judged by their final state.

  • Limits on extrapolation: Workflows that require human clarification, success rates and bills for production automations, and latency in real-time interactions.

  • Model version: Google Gemini 4 Neon, listed in the report as gemini-4-neon.

  • Test environment or client: Zapier AutomationBench 1.0.6. The agent uses APIs to interact with simulated apps.

  • Reasoning tiers and parameters: Low, Medium, High, and XHigh, with a limit of 50 steps per task. The report leaves the exact model snapshot and other generation parameters unspecified. Methodology

Evaluation method

In pre-release testing from 2026-11-17 to 2026-11-18, Zapier used the same 657 tasks for all four tiers. The tasks span sales, marketing, operations, customer support, finance, and human resources across 47 simulated apps. Each task starts with a preset company state. The agent receives a business request and works independently. Under task_completed_correctly, a task passes only if its final state satisfies every assertion. The report gives dates for the overall test run. Per-task runtimes and paired task-level results across tiers are unavailable. Methodology

Key results

Moving from Low to Medium yielded 24 more passes in total and raised the estimated cost per attempt by $0.18. Moving from Medium to High yielded 7 more passes and raised that cost by $0.30. The gains also vary by task type. In operations, Low, Medium, and XHigh each passed 68/100 tasks, while High passed 65/100. Results by effort · Results by domain

Raw data

Reasoning tierTasksPassesStrict pass rateEstimated cost per attempt (USD)Estimated spend per passing task (USD)Share of input tokens cached
Low65732649.6%0.400.8181.3%
Medium65735053.3%0.581.0883.3%
High65735754.3%0.881.6287.2%
XHigh65737056.3%1.693.0188.5%

Estimated spend per passing task = total estimated cost of all attempts ÷ number of passes. The numerator includes failed attempts. The table keeps the precision of the original report. Cost table · Cached input share

Conclusions and limitations

Zapier estimates costs from the token usage recorded in the tests. It uses list prices of $3 per million input tokens and $15 per million output tokens, counting reasoning tokens as output. When Zapier prepared the report, it assumed $0.15 per million tokens for cache reads. The report records no cache writes for Gemini's implicit cache. Usage, reasoning tier, and cache hits affect the estimates per attempt. Actual bills depend on measured usage and applicable rates. Cost methodology

Zapier's AutomationBench 1.0.6 measures the share of tasks that pass every assertion. Artificial Analysis's AutomationBench-AA is an independent implementation. Its primary metric is the share of task objectives completed without violating constraints. Check the benchmark and scoring method when comparing results. Zapier scoring method · AutomationBench-AA overview

Reproduction notes

The report's Results by effort, Cost, Tokens & tools, and Methodology sections give the task counts, pass counts, costs, cached input shares, and test configuration for verification. Zapier says it scores leaderboard entries on a held-out private task set and provides 600 public tasks for experimentation. When rerunning on the public set, use the same task set, starting state, tool configuration, and 50-step limit across tiers. Count a task as a pass only if every assertion passes, and record cache usage for each tier. Fully reproducing the leaderboard results requires the private task set and the original run configuration. Neon results report · Zapier leaderboard

Curated by Tabbit

This is a third-party source navigator. Model versions, test environments, and personal experience vary; consult the original source.

Gemini 4 Neon

Use and compare models in Tabbit

Gemini 4 Neon

Related reviews

OfficialGoogle DeepMind2026-11-18

Science, coding, and multimodal benchmarks in the Gemini 4 Neon model card

MediaArtificial Analysis2026-11-18

Gemini 4 Neon benchmark scores, latency, and cost across four reasoning tiers

CommunityX2026-11-19

Gemini 4 Neon (High) ranks second on Agent Arena's real-world task leaderboard

MediaVals AI2026-11-18

Gemini 4 Neon scores, costs, and runtimes on Vals Index 2.2

Gemini 4 Neon

Related showcases & prompts

OfficialGoogle AI Studio / Gemini API

Gemini 4 Neon model ID and Gemini API specifications

OfficialGoogle AI Studio / Gemini Developer API

Estimating Gemini 4 Neon API costs by inference mode

OfficialOpenRouter

Calling Gemini 4 Neon through OpenRouter and choosing an upstream endpoint

MediaGoogle Cloud Gemini Enterprise Agent Platform2026-11-18

Regions and inference settings for Gemini 4 Neon on Google Cloud