In Plotly's private 43-question data analytics exam, Gemini 4 Hydrogen answered all 43 questions correctly in a single run at a cost of $6.36. That result exceeded Gemini 4 Carbon (42/43, $11.55) and Gemini 4 Argon (41/43, $13.47) in the same test, while GPT-6.1 Sol also scored 43/43 at a cost of $2.61. The result describes only how the models performed on Plotly's analytics-agent exam.
Tasks this can help assess: Data analysis with multiple rounds of tool calls, chart generation, identifying statistical traps, data quality checks, and discovering hidden patterns.
Tasks it should not be generalized to: General question answering, open-ended writing, or analysis of real enterprise data. The result also does not establish that the same score would hold with different reasoning tiers, APIs, or agent configurations.
Applicable model version: Gemini 4 Hydrogen. The Google AI Studio documentation lists the stable model ID as gemini-4-hydrogen; Plotly says it called the model through OpenRouter on launch day, but does not specify which model revision OpenRouter routed to.
Test environment or client: Plotly Studio Embedded data analytics agent, calling the model through OpenRouter.
Reasoning tier and parameters: Not specified. Google labels Hydrogen's default reasoning tier as high, but Plotly does not say whether this run used the default tier.
Plotly used its private PlotlyDataAnalyticsBench, running the same 43 questions for each new model with the same harness and scoring method. The harness is a data analytics agent for Dash apps, supports multiple rounds of tool calls and chart generation, and prioritizes data analysis results over text writing. The exam uses a simulated wind farm operator's data warehouse. The question set is not public; Plotly says it shares it only with researchers to reduce the chance of answers entering training data.
The author ran the complete test once through OpenRouter on Gemini 4 Hydrogen's launch day. The article does not disclose the questions, answers, system prompt, tool schema, per-question call records, or raw outputs. Plotly manually reviewed each of the 43 passing results.
The model scored 43/43, becoming the fourth model at the time to pass every question. The article describes the breakdown as five categories with full marks, then lists: analysis questions 20/20, ambiguous questions 5/5, impossible requests 5/5, data quality 3/3, statistical traps 5/5, and hidden patterns 5/5.
Gemini 4 Carbon scored 42/43 at a cost of $11.55; Gemini 4 Argon scored 41/43 at $13.47. GPT-6.1 Sol also scored 43/43, at a cost of $2.61.
The Hydrogen exam cost $6.36, with a median of 32 seconds per question and a total runtime of about 43 minutes. Among the comparison models, Carbon took a median of 93 seconds per question and about 1 hour 48 minutes overall; Argon took 23 seconds per question; GPT-6.1 Sol took 18 seconds.
Hydrogen found all five planted hidden patterns. The five questions averaged about 54 queries each; together, they involved 95 model calls and 271 tool calls. The article says only 8 of the 48 models on the full leaderboard found all five patterns.
The run used about 17.8 million input tokens, 88% of them cached. Based on 10 questions per person per day, 250 workdays per year, and OpenRouter prices at the time of testing, the article estimates an annual cost of about $370. This is a scenario estimate, not a subscription price.
| Model | Total score | Analysis | Ambiguous | Impossible requests | Data quality | Statistical traps | Hidden patterns | Cost per run | Median time per question |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 4 Hydrogen | 43/43 | 20/20 | 5/5 | 5/5 | 3/3 | 5/5 | 5/5 | $6.36 | 32 seconds |
| Gemini 4 Carbon | 42/43 | 20/20 | 5/5 | 5/5 | 2/3 | 5/5 | 5/5 | $11.55 | 93 seconds |
| Gemini 4 Argon | 41/43 | 19/20 | 5/5 | 5/5 | 2/3 | 5/5 | 5/5 | $13.47 | 23 seconds |
| GPT-6.1 Sol | 43/43 | 20/20 | 5/5 | 5/5 | 3/3 | 5/5 | 5/5 | $2.61 | 18 seconds |
In this test, Hydrogen was more accurate than the other two Gemini 4 models and cost about half as much as Carbon and Argon; it was not the fastest model overall or the least expensive among models with the same score. Plotly says repeated tests of other models have varied by as many as 3 points, so a 1–2 point difference from a single run should be treated as close rather than a stable ranking. The article does not report repeated runs for Hydrogen.
The exam questions and raw run records are not publicly available, so external readers cannot reproduce the scores, costs, or runtimes question by question. The findings apply only to this OpenRouter call and the Plotly Studio Embedded harness; they do not represent performance on production data, with different tool configurations, or at other model tiers. Cost and price comparisons also reflect OpenRouter's rates at the time of testing.
A similar evaluation could be built using the environment and 43 question types described by Plotly. However, the exam cannot be reproduced exactly because its questions, answers, prompts, and tool definitions are not public. For model comparisons, fix the question set, agent harness, tools, model ID, and reasoning settings; run multiple trials and record per-question results, call counts, cached tokens, costs, and runtimes separately. Do not treat this single-run score as a stable measure of capability.
Gemini 4 Hydrogen