Tencent presents its internal WorkIndex benchmark, a WorkBuddy user blind test, and several product-team case studies separately. Its official data indicates that Hy4 Flash was less expensive and faster on these specific sets of everyday work tests. They are not independent replications and do not represent all tasks.
Tasks these results can help assess: The office-work benchmarks, internal user blind test, and published agent and multimodal benchmarks disclosed by Tencent.
Tasks these results should not be generalized to: Tasks with undisclosed environments or toolchains, all user and business scenarios, long-horizon software engineering as a whole, or direct comparisons across different harnesses or pricing periods.
Model version covered: Hy4 Flash; comparison models are listed under their appendix names.
Test environment or client: WorkBuddy CLI was used for each internal WorkIndex test. The user blind test involved 85 internal WorkBuddy users. Environments for public benchmarks varied by project; see the methodology notes.
Reasoning level and parameters: The appendix says that each model was evaluated at its highest available reasoning setting. Specific API parameters, sampling parameters, and model revisions were not stated.
Tencent describes WorkIndex as an internal evaluation set built around typical WorkBuddy tasks. It contains eight task types: creating presentations, editing presentations, co-editing presentations, co-editing Markdown, editing documents, fixing spreadsheets, audio/video processing, and visual understanding. The appendix maps these to PPTGenBench, PPTCoEditBench, PPTFixBench, MdCoEditBench, WordEditBench, ExcelFixBench, AudioVideoBench, and BuddyVQA Atlas.
The appendix table lists scores and per-task costs (in CNY) for seven models. The original chart does not explain the score scale; the table below transcribes the scores without interpreting them as percentages. Costs were calculated using each model's official API prices and the test usage.
The main text lists seven Open Benchmarks: DeepSWE, ProgramBench, Terminal-Bench 4.0, APEX-Agents, AutomationBench, HLE, and OSWorld 2.1. The appendix lists two additional Multimodal items. These are separate sets from WorkIndex and should not be combined and described as one office-work benchmark.
The main text says Tencent invited 85 internal WorkBuddy users to conduct a blind test across 259 everyday work tasks in 27 categories. The results chart reports user satisfaction (on a four-point scale), median end-to-end time, and cost per task for four models. The page does not disclose task assignment, randomization or blinding details, API parameters for each model, the number of raters, score dispersion, or significance testing.
| Model | User satisfaction (4-point scale) | Median end-to-end time | Cost per task |
|---|---|---|---|
| DeepSeek-V4.1-Flash-1017 | 3.09 | 10.3 minutes | ¥1.20 |
| Hy4 Flash | 2.79 | 6.7 minutes | ¥0.26 |
| Claude Haiku 5.5 | 2.81 | 6.8 minutes | ¥4.73 |
| GLM-5.3-Flash | 2.53 | 7.6 minutes | ¥0.81 |
The chart caption says the external models DeepSeek, Claude Haiku, and GLM all used the high setting. Hy4 Flash's satisfaction score was 0.02 points below Claude Haiku 5.5, while its time was 0.1 minute shorter. The chart shows its cost at about 1/18 of Claude's, 1/4.6 of DeepSeek's, and 1/3.1 of GLM's. The chart provides no error ranges, so it cannot establish whether the satisfaction difference is significant.
Each cell is formatted as “score / ¥ per task”; “—” means the original appendix provided no data.
| Benchmark | Hy4 Flash | Hy4 | Hy4 preview | DeepSeek-V4.1-Flash-1017 | Qwen3.8-Flash-Next | GLM-5.3-Flash | GPT-6 Luna |
|---|---|---|---|---|---|---|---|
| PPTGenBench | 54.2 / 0.169 | 61.5 / 3.969 | 52.3 / 3.484 | 64.4 / 1.672 | 56.0 / 0.830 | 53.6 / 1.460 | 56.8 / 0.753 |
| PPTCoEditBench | 74.0 / 0.105 | 78.9 / 1.714 | 71.5 / 1.711 | 76.8 / 0.459 | 62.9 / 0.213 | 67.3 / 0.489 | 43.5 / 0.303 |
| PPTFixBench | 23.7 / 0.253 | 45.6 / 15.400 | 23.8 / 4.160 | 45.3 / 1.672 | 23.9 / 0.875 | 28.6 / 2.517 | 8.6 / 1.360 |
| MdCoEditBench | 56.9 / 0.057 | 57.7 / 0.899 | 49.2 / 0.681 | 29.2 / 0.194 | 53.8 / 0.080 | 32.3 / 0.133 | 53.1 / 0.120 |
| WordEditBench | 46.2 / 0.629 | 57.2 / 10.304 | 44.2 / 5.725 | 53.5 / 2.237 | 46.3 / 1.430 | 47.6 / 2.840 | 42.7 / 1.360 |
| ExcelFixBench | 15.6 / 0.310 | 27.8 / 6.120 | 11.7 / 4.110 | 20.0 / 1.518 | 12.8 / 1.155 | 9.9 / 2.879 | 6.7 / 0.800 |
| AudioVideoBench | 17.8 / 0.344 | 27.4 / 7.290 | 16.3 / 6.030 | 22.2 / 5.330 | 17.8 / 1.290 | 11.9 / 3.370 | 10.0 / 2.304 |
| BuddyVQA Atlas (without tools) | 48.5 / 0.007 | 55.5 / 0.180 | — | 43.7 / 0.059 | 54.5 / 0.012 | 50.1 / 0.019 | 51.6 / 0.016 |
| Benchmark | Hy4 Flash | Hy4 | Hy4 preview | DeepSeek-V4.1-Flash-1017 | Qwen3.8-Flash-Next | GLM-5.3-Flash | GPT-6 Luna |
|---|---|---|---|---|---|---|---|
| DeepSWE v1.1 | 64.6 / 0.290 | 74.9 / 5.409 | 64.3 / 8.951 | 73.8 / 1.304 | 65.1 / 3.251 | 67.8 / 3.230 | 67.1 / 1.535 |
| ProgramBench (almost solved) | 16.0 / 0.561 | 30.0 / 30.281 | 17.5 / 107.063 | 29.0 / 9.214 | 22.0 / 18.432 | 13.0 / 17.959 | 6.0 / 1.024 |
| Terminal-Bench 4.0 | 19.2 / 0.558 | 44.4 / 12.014 | 26.3 / 18.015 | 40.0 / 3.882 | 29.5 / 5.100 | 35.4 / 5.597 | 13.3 / 2.199 |
| APEX-Agents v1.1 (pass@1) | 63.1 / 0.087 | 75.2 / 3.104 | 53.9 / 2.123 | 72.9 / 1.465 | 57.6 / 0.349 | 51.5 / 0.485 | 44.7 / 0.188 |
| AutomationBench v1.0.6 | 52.2 / 0.038 | 56.2 / 0.908 | 36.3 / 0.424 | 54.3 / 0.620 | 49.3 / 0.075 | 48.4 / 0.104 | 38.1 / 0.137 |
| HLE-Full (with tools) | 55.2 / 0.060 | 59.1 / 2.300 | — | 57.1 / 0.657 | 54.1 / 0.275 | 52.2 / 0.616 | 49.7 / 0.142 |
| OSWorld 2.1 (Full, partial score) | 68.1 / 0.451 | 78.9 / 5.992 | — | 68.4 / 2.073 | 58.8 / 2.644 | 64.0 / 3.803 | 55.7 / 4.960 |
| Multimodal: MMMU-Pro | 75.8 / 0.010 | 78.6 / 0.197 | — | 76.7 / 0.076 | 77.2 / 0.015 | 77.0 / 0.013 | 77.5 / 0.017 |
| Multimodal: ZeroBench-main (with tools, pass@5) | 45.8 / 0.077 | 55.4 / 1.389 | — | 16.5 / 0.095 | 41.7 / 0.159 | 35.3 / 1.861 | 33.6 / 0.327 |
Unified settings: Each model was evaluated at its highest available reasoning setting. All internal Buddy Index tests used WorkBuddy CLI.
DeepSWE v1.1: mini-SWE-agent; each task ran in a sandbox with 8 CPU cores and 16 GB of memory.
ProgramBench: The Almost Solved score was reported using mini-SWE-agent.
Terminal-Bench 4.0: Claude Code harness; up to 500 turns and 13 hours per trial; 16 CPUs and 32 GB RAM; each of the three GPU-dependent tasks used one H100.
APEX-Agents v1.1: Official reference agent harness (Loop), with up to 250 steps per run. The official dual-service main + world environment was limited to 8 CPUs and 24 GB RAM; the separate verifier container was limited to 2 CPUs and 8 GB RAM.
AutomationBench v1.0.6: 600 public tasks released by Zapier, evaluated using the official method.
PPTGenBench: Generate presentations from text; combine an overall aesthetic evaluation with penalties for critical errors; render presentations as PDFs using LibreOffice.
PPTCoEditBench: 100 real-world presentation-editing tasks. Checks whether requested changes were made without collateral damage such as overlapping shapes or dangling animations. Includes tasks that select content in the WorkBuddy editor. It is currently single-turn; multi-turn support is still in development.
PPTFixBench: 50 tasks. Each provides a presentation containing more than 25 formatting or content issues for the model to identify and fix.
MdCoEditBench: 65 Markdown co-editing tasks covering structural changes, local edits, continuation, intent understanding for selections, awareness of user edits, feedback generalization, editing conflicts, and version rollback; evaluated in the WorkBuddy editor.
WordEditBench: 100 real-world Word (.docx) editing tasks in four categories: error correction, cross-document formatting, targeted editing, and academic paper formatting. Scored by a rule-based checker on document XML, rewarding required edits and penalizing unrelated edits.
ExcelFixBench: 60 real-world spreadsheet repair tasks. Checks fixes to formulas, data, and formatting while preserving unrelated content; evaluated in a sandbox, with each task scored against a task rubric and averaged over three trials.
AudioVideoBench: Containerized end-to-end audio/video production tasks covering editing, compositing, restoration, aspect-ratio conversion, mixing, transcription, and content understanding.
BuddyVQA Atlas: Image question answering covering recognition, counting, comparison, OCR, spatial orientation, and localization, with an emphasis on small text, thin lines, and local targets that are easy to miss when answering directly from the full image.
ZeroBench-main: Network-isolated Jupyter environment using image output as the tool interface; up to 80 tool calls per round; each question was run independently 10 times to estimate pass@5.
Additional note on visual benchmarks: The original appendix states that, for benchmarks requiring visual capabilities, Hy4 Flash results were used to represent Hy4 preview when plotting.
The appendix says costs were calculated using each model's official API pricing and a fixed exchange rate of 1 USD = 7.2 CNY. For models with discounts or time-of-day pricing, fixed prices were used (per million tokens, uncached input / cached input / output): DeepSeek-V4.1-Flash-1017 at ¥2 / ¥0.04 / ¥8 (peak-hour pricing), and GLM-5.3-Flash at ¥0.8 / ¥0.23 / ¥2.8 (standard API pricing). The Hy4 Flash launch page separately lists API prices of ¥0.02 for cached input, ¥0.5 for input, and ¥2 for output per million tokens; the corresponding Hy4 prices are ¥0.3, ¥6, and ¥18.
These costs are not an independent billing audit. The public materials do not provide complete input/output token counts for each model, task weights for each benchmark, or price snapshots for all models, so the results cannot be fully recalculated from the cost table alone.
The main text says the per-task WorkIndex cost was about 1/13 that of Hy4 preview, 1/20 that of Hy4, and 1/8 that of DeepSeek-V4.1-Flash-1017 (max setting, busy-hour pricing). The cost chart labels Hy4 Flash at about ¥0.2 per task, Hy4 preview at about ¥3.2 per task, and Hy4 at about ¥5.7 per task, and says the first was respectively 13.8× and 24.5× cheaper. The ratios in the body text do not fully match the chart's multipliers. The page gives neither aggregate weights nor unrounded values, so both original statements are retained without reconciling them or deriving a single ratio.
The main text also includes a product-side WorkBuddy evaluation of Hy4: average task time fell by 70%, average cost was about 1/15 that of Hy4, everyday task performance was broadly on par, and long-horizon Coding tasks declined slightly. The page does not explain task overlap with WorkIndex or the 259-task blind test, sample size, or calculation method; this should not be treated as the same experiment.
In Tencent's disclosed set of 259 internal everyday tasks, the blind test shows Hy4 Flash had the lowest median completion time and per-task cost. Its satisfaction score was 2.79/4, close to Claude Haiku 5.5 at 2.81/4. Variance and blinding details were not disclosed, so the statistical significance of the small satisfaction gap cannot be assessed.
The appendix's per-benchmark results are not uniformly favorable: Hy4 Flash scored below Hy4 or other models on several WorkIndex tasks and public benchmarks. The value-for-money claim therefore should not be restated as an overall capability lead.
WorkIndex is Tencent's internal set, the WorkBuddy blind test is a separate set of real-use tasks, and product-team case studies cover their own business scenarios. These three should not be conflated as one evaluation.
Public benchmarks in the tables use different harnesses, resource limits, and scoring methods, while costs also depend on the pricing assumptions. Cross-benchmark comparisons represent only this snapshot reported by Tencent and are not third-party replications under uniform conditions.
This article records the vendor's original disclosure; it has not been independently reproduced. The page does not publish the task sets, raw outputs, complete scoring rubrics, token usage by task, or reproduction scripts.
A reproduction could use the benchmark names, harnesses, resource limits, and highest reasoning settings listed on the original page. Exact reproduction is not possible from that page alone: the WorkIndex task set and complete scoring rules are not public; blind-test task distribution and randomization details are not public; specific API versions and request parameters for most models are not stated; and task-level token usage and complete price snapshots needed to calculate costs are not public. A reproduction should record these differences separately and should not directly replace the original chart with new results.
Hy4 Flash