Plotly reports that Hy4 answered 42/43 questions (98%) correctly on its data analytics benchmark, tying Grok 4.7 and Kimi K3.1 Pro by overall score; only GPT-6.1 Sol and GPT-6 Astra scored higher at 43/43. Hy4 completed the full benchmark for $2.11, the lowest cost among the models on the leaderboard that scored at least 42, but its median time per question was 117 seconds, bringing the full run to nearly three hours.
Tasks this can help assess: Querying synthetic wind-farm data, creating charts, handling ambiguity and anomalies, auditing data quality, and finding hidden patterns in an analytics agent that makes multiple tool calls.
Tasks this should not be generalized to: Success rates on real enterprise data, non-analytics tasks, performance in other agents or tool environments, and stability. The test reports one run per model; repeated runs of other models varied by as much as 3 points in total score.
Model version covered: The official release of Tencent Hunyuan Hy4; the service build on OpenRouter was not specified.
Test environment or client: Plotly Studio Embedded, an analytics agent running in a Dash app. It used multiple tool calls, focused on querying and charting, and placed less emphasis on long-form writing.
Reasoning tier and parameters: Run with OpenRouter defaults; specific sampling parameters were not specified.
Plotly tested each model on the same 43-question benchmark and used the same scoring method. The questions fell into six categories: 20 analytical questions, 5 ambiguous questions, 5 impossible requests, 3 data quality checks, 5 statistical traps, and 5 hidden-pattern questions. Each question was scored pass or fail; the article did not publish per-question scoring criteria.
The test data was a fictional wind-farm operations warehouse designed to simulate real operator data and keep the answers from appearing elsewhere. The question set remains private to reduce the risk of answers entering training data; Plotly says researchers can contact it to request access. Each model ran once. The article says repeated runs of other models varied by at most 3 points in total score, so a 1-point difference should be treated as a tie.
Score and placement: Hy4 scored 42/43, or 98%. Based on the scores listed in the article, GPT-6.1 Sol and GPT-6 Astra scored 43/43, while Hy4, Grok 4.7, and Kimi K3.1 Pro scored 42/43. This puts Hy4 in a tie for third by overall score; the article did not publish a formal ranking table with tie rules. The leaderboard included 37 models.
Category scores: Analytical questions 20/20, ambiguous questions 5/5, impossible requests 5/5, data quality 3/3, statistical traps 5/5, and hidden patterns 4/5. The missed question concerned whether a type of component failure had become unusually frequent. Hy4 found a significant issue but did not inspect the data section containing the pattern and ruled out causes it had not yet verified.
Cost for the full benchmark: Hy4 $2.11; GPT-6.1 Sol $2.61; Grok 4.7 $7.85; Kimi K3.1 Pro $13.47. Hy4 was the least expensive of the 37 models that scored at least 42. Its position on the cost-accuracy plot was between Particle-1 and GPT-6.1 Sol; Plotly says no other tested model was both cheaper and more accurate than Hy4.
Speed and tool use: Hy4's median time per question was 117 seconds, and the full run took nearly 3 hours; GPT-6.1 Sol took 18 seconds and Grok 4.7 took 38 seconds. Hy4 averaged about 24 query and chart operations per question; on hidden-pattern questions it made about 64, compared with a median of 26 across all models. One Cartesian-product query ran for nearly an hour; the test runner had no query timeout limit.
Tokens and pricing: The full benchmark used about 22.7 million input tokens, 91% of them cached. Hy4's input, output, and cached-input rates were $0.42, $1.25, and $0.021 per million tokens, respectively. The article estimates $123 per person per year, assuming 1,000 people each ask 10 questions a day for 250 working days a year, and using OpenRouter prices at the time of the model run.
| Metric | Hy4 | Comparison or basis |
|---|---|---|
| Overall score | 42/43 (98%) | GPT-6.1 Sol, GPT-6 Astra: 43/43; Grok 4.7, Kimi K3.1 Pro: 42/43 |
| Analytical questions | 20/20 | 20 questions total |
| Ambiguous questions | 5/5 | 5 questions total |
| Impossible requests | 5/5 | 5 questions total |
| Data quality checks | 3/3 | 3 questions total |
| Statistical traps | 5/5 | 5 questions total |
| Hidden patterns | 4/5 | 5 questions total |
| Full benchmark cost | $2.11 | GPT-6.1 Sol $2.61; Grok 4.7 $7.85; Kimi K3.1 Pro $13.47 |
| Median time per question | 117 seconds | GPT-6.1 Sol 18 seconds; Grok 4.7 38 seconds |
| Input tokens | About 22.7 million | 91% cached input |
The result supports conclusions only about the data analytics tool-calling workflow in Plotly Studio Embedded and this synthetic wind-farm dataset. Plotly published the evaluation and used its own product as the test environment; the source did not disclose any sponsorship or partnership with Tencent. A first-party product environment may affect tool fit, so the result cannot be equated directly with that of an independent third-party agent. The question set is private and per-question records were not published, so outside parties cannot fully reproduce the evaluation; a single run is also insufficient to estimate Hy4's own score variability. The score ranking, cost, and speed apply only to the test described in the article and OpenRouter prices at that time.
As described in the article, Hy4 can be run with OpenRouter defaults in Plotly Studio Embedded, recording queries, charts, per-question results, tokens, and elapsed time. Full reproduction depends on Plotly providing the private question set and scoring criteria. This article did not rerun the benchmark; it records the single-run results published by Plotly on 2026-11-04.
Hy4