Plotly reports that Tencent Hy4 Flash passed 39 of 43 questions (91%, A) in a single run of its DataAnalyticsBench, at a total cost of $0.13 and a median of 29 seconds per question. This is a single-sample result from a benchmark whose questions and dataset have not been made public.
Tasks this result can help assess: Comparing how models handle analytical questions, ambiguity, impossible requests, data-quality issues, statistical traps, and hidden patterns in Plotly Studio Embedded and its Dash data-analysis environment.
Tasks this result should not be generalized to: General-purpose question answering, real enterprise data, non-Plotly toolchains, different prompts, or average performance across repeated runs. The result also does not represent every data-analysis task.
Model version covered: The article's Tencent Hy4 flash, released on 2026-11-16; no more specific model snapshot or API version is identified.
Test environment or client: Plotly Studio Embedded (a data-analysis agent running in a Dash app), accessed through OpenRouter.
Reasoning level and parameters: Reasoning level, temperature, and other sampling parameters are not specified.
Plotly says it used the same exam, harness, and scoring rules for the new model. The test ran through multiple tool calls and focused on data-analysis queries and chart generation, with less emphasis on text writing. The exam had 43 questions: 20 standard analysis questions, 5 ambiguity questions, 5 impossible requests, 3 data-quality questions, 5 statistical traps, and 5 hidden-pattern questions.
The exam and answer key are private. The dataset is simulated wind-turbine operations data, used to keep answers from appearing in real data sources. Each model was run once. Plotly says repeated tests of other models varied by up to 3 points in a single run, so differences of 1–2 points should be treated as close to a tie.
Hy4 Flash: 39/43 (91%, A), $0.13 for one run, and a median of 29 seconds per question; the full exam took about 33 minutes.
Scores by category: standard analysis 20/20, ambiguity 5/5, impossible requests 5/5, data quality 3/3, statistical traps 4/5, and hidden patterns 2/5.
Main losses: 3 hidden-pattern questions and 1 statistical-trap question. Plotly checked the transcripts and answer key question by question and says all 4 were genuine misses.
Run overhead: about 8 model calls and 4,800 output tokens per question on average; 88% of input tokens came from provider caching; there were no model errors in this run.
Plotly's comparison in the same table: Hy4 scored 42/43 at $2.11 and 117 seconds per question; GPT-6 Luna scored 38/43 at $0.13 and 18 seconds per question; Particle-1 scored 40/43 at $0.35 and 16 seconds per question. Score differences between models should be interpreted in light of the single-sample limitation.
| Metric | Value reported by Plotly |
|---|---|
| Total score | 39/43 (91%, A) |
| Standard analysis | 20/20 |
| Ambiguity questions | 5/5 |
| Impossible requests | 5/5 |
| Data quality | 3/3 |
| Statistical traps | 4/5 |
| Hidden patterns | 2/5 |
| Cost for one full run | $0.13 |
| Median time per question | 29 seconds |
| Full exam time | About 33 minutes |
| Average model calls per question | About 8 |
| Output tokens per question | About 4,800 |
| Share of input tokens served from cache | 88% |
The article lists OpenRouter rates of about $0.044 per million input tokens and $0.18 per million output tokens. The $0.13 is the cost of this test run and should not be equated directly with per-token rates or with other models tested under different configurations. The article also estimates a cost of about $8 per person per year, assuming 1,000 people, 10 questions per person per day, 250 working days per year, and OpenRouter prices at the time of testing. This is a scenario estimate, not an actual user bill.
The result supports a limited conclusion: in Plotly's private data-analysis exam and tool environment, Hy4 Flash earned an A in one run at low cost, but scored 2/5 on hidden patterns. Compared with Hy4, it answered 3 fewer questions correctly, had a reported cost of about one-sixteenth as much, and a median time per question of about one-quarter as long.
The exam questions, answers, dataset, full run logs, and sampling parameters have not been made public, so outside readers cannot independently reproduce the 39-question score from the article alone. A single run also cannot establish the variance in model performance; a one- or two-point difference between models should not be treated as a stable advantage. OpenRouter rates and cache hits affect the reported cost, and costs and latency may differ with another provider or execution setup.
Reproduction would require Plotly's private DataAnalyticsBench questions and answer key, along with the same version of the Plotly Studio Embedded harness, Dash data environment, OpenRouter access method, and scoring rules. The article says the question bank is shared only with researchers; undisclosed sampling parameters and full execution records remain additional limits on reproducibility. The figures in this note reflect the source article as accessed on 2026-11-17.
Hy4 Flash