Arena said on November 6 that Hy4 achieved a +5.4% net lift across more than 12K real-user Agent sessions in its evaluation, at a median cost of $0.57 per task. These figures reflect the real tasks and runtime environment covered by Agent Arena; they are not a general capability score or the success rate for all tasks.
Tasks this evaluation can help assess: Compare models used as Agent orchestrators on overall performance, user feedback signals, tool reliability, and actual cost in real work sessions in Arena Agent Mode.
Tasks this evaluation should not be extrapolated to: Single-turn Q&A, pure model capability, a specified software environment, tasks that do not use Arena Agent Mode, or the success rate or completion cost of all user tasks.
Applicable model version: Arena's post identifies the model only as Hy4 and does not disclose the specific weights or server-side build; the same post says it was a proprietary model at the time.
Test environment or client: Arena.ai Agent Mode; the model was used as the orchestrator within Arena's Agent tools and runtime environment.
Reasoning tier and parameters: Not specified. The leaderboard version is named Hy4; the reasoning tier, sampling parameters, and associated harness version are not specified.
Agent Arena collects data from work sessions users actually complete in Arena.ai Agent Mode, including software development, file analysis, web research, and multi-step tasks. The leaderboard uses causal tracing: it treats an Agent as a multi-component system and uses randomized component options to conduct randomized controlled evaluations with multiple interventions, estimating the causal effect of a component choice on task outcomes. The current leaderboard evaluates only the orchestrator model.
The overall leaderboard combines five types of signals: explicit user confirmation of success, user praise and complaints, responsiveness to user corrections, Bash error recovery, and tool hallucinations. Arena uses a self-normalized weighted estimator and gives greater weight to newer data to account for changes in the data distribution over time. Cost is not one of the signals in the overall score; Arena separately calculates actual runtime cost for value comparisons.
Disclosed in the 2026-11-06 X post: Arena described Hy4 as the top-ranked Chinese model in Agent Arena and the only model from a Chinese lab in the top five. The sample comprised more than 12K real-world, long-horizon Agent sessions from users worldwide; the net lift was +5.4%; and the median cost was $0.57 per task, which Arena said placed it on the Pareto frontier.
The post did not provide a confidence interval for that +5.4%, an exact session count, scores for individual signals, or raw session data. 12K+ is a lower-bound disclosure, not an exact denominator.
Later snapshot from the live leaderboard: The leaderboard page lists data dated 2026-11-13, with 2,979,271 sessions and 65 models in total. Hy4 ranked 19th, with a net lift of 5.67% ±1.25%, 23,160 sessions, and a median cost of $0.57 per task. This is a live leaderboard result for a later date and should not replace the +5.4% reported in the November 6 post.
Earlier baseline snapshot from the same series: On November 4, Arena said the full release of Hy4 had achieved a +4.7% net lift across more than 8.5K real sessions, at a cost of $0.55 per task; it ranked 14th, and its overall score was 3.6 percentage points higher than Hy4 preview's +1.1%. The post also reported differences between the full release and preview in metrics such as confirmed success, praise and complaints, steerability, and Bash reliability. This is an earlier leaderboard state and is included only as a same-source longitudinal reference.
| Date and source | Leaderboard position | Net lift | Session sample | Median cost per task | Uncertainty |
|---|---|---|---|---|---|
| 2026-11-04 Arena post | 14th overall | +4.7% | 8.5K+ | $0.55 | No interval reported in the post |
| 2026-11-06 Arena post | 1st among Chinese models; the only Chinese lab model in the top five | +5.4% | 12K+ | $0.57 | No interval reported in the post |
| 2026-11-13 leaderboard snapshot | 19th overall | 5.67% ±1.25% | 23,160 | $0.57 | 95% confidence interval listed on the leaderboard |
The methodology says the 95% confidence interval shown next to a leaderboard estimate is calculated as “estimate ± 1.96 × standard error.” Scores on the live leaderboard change as new data arrives and time weights are applied, so scores and rankings should be cited with the snapshot date. The post's “top-ranked Chinese model” does not mean first overall.
This evaluation was published by Arena, which is outside the model vendor, and is based on data from its own platform and real users. Compared with a fixed benchmark, it is closer to actual multi-step work in Arena Agent Mode; however, Arena determines the session distribution, tool environment, and user population. Public materials do not provide Hy4's exact weights version, the complete sampling plan, task-level raw data, or a breakdown of failure and retry costs, so the +5.4% cannot be independently recalculated, nor can it be confirmed whether $0.57 is the cost per successfully completed task. Before directly comparing these results with other leaderboards or vendor-internal evaluations, check the model version, Agent harness, sample, and statistical definitions.
Open and verify Arena's 2026-11-06 X post and its cited November 4 results post in Tabbit. Check the Agent Arena methodology for the definitions of causal tracing, composite signals, cost calculation, and confidence intervals, and record the 2026-11-13 snapshot shown on the Agent Arena leaderboard. Arena has not published the underlying session records or materials needed to reproduce the calculations.
Hy4