The author reports 18 same-task tests: Hy4 led on small projects, some Chinese writing, and close-reading tasks; DeepSeek V4.1 Flash led on design and promotional copy scores; and Qwen4 led in correct answers on mazes, grid route planning, and research-level math problems, while also having the lowest reported total output token count. These results apply only to the tasks and calling environments disclosed by the author and do not establish a general ranking.
Tasks this can help assess: A preliminary comparison of the three models on this set of coding, web, visual reasoning, Chinese writing, design, and 3D creation tasks, as well as their usage.
Tasks this should not be generalized to: Other model versions, clients, or tool configurations; actual pricing; success rates across all users' tasks; or general capability rankings on unpublished tasks.
Model versions covered: Hunyuan Hy4, DeepSeek V4.1 Flash 1017 (abbreviated as DS in the article), and Qwen4; specific server-side builds or weight versions were not given.
Test environment or client: Hy4 was accessed through the Tencent Hunyuan API, DS through WorkBuddy, and Qwen4 through a direct API. The author says the prompts were identical and that internet access and tools were prohibited.
Reasoning level and parameters: Not specified.
The author says the tests were run from November 7 to 9, 2026, and comprised 18 tasks covering coding, web development, image-based tasks, long-form writing, reasoning, design, Chinese writing, and 3D creation. Most tasks were run three times per model. Total usage was about 120 million tokens, including evaluation by Claude and Codex.
Scoring methods included hidden tests for code, script checks for web pages, and blind evaluations of writing and design with model labels shuffled. Codex alone scored 3D creations from 0 to 10 based on the task requirements and screenshots; the evaluator could see which model produced each work. Usually, only the result after completion was counted. Models that did not finish in the first round could continue writing, and creations that would not open could be repaired once, with those cases tracked separately. Math problems had a 20-minute limit; timeouts and incomplete answers counted as incorrect. The author says token usage included reasoning, was reported as the median for each task, and combined continuation runs; the overall total included reruns, continuations, and repairs.
Reasoning: On 18 maze problems, Hy4 scored 6, DS 1, and Qwen4 15. On 18 dual-robot grid route-planning tasks, they scored 15, 17, and 18, respectively. Each model attempted 34 research-level math problems three times, for 102 attempts total, with a 20-minute limit per problem; the numbers correct were 10, 8, and 16, respectively.
Design and copywriting: In combined blind-review scores for posters, landing pages, and app homepages, Hy4 scored 5.70, DS 8.00, and Qwen4 7.07. For Xiaohongshu promotional posts, the scores were 4.00, 8.33, and 6.00, respectively. The article does not specify the score scale or per-task variance. Each model attempted each design task three times, and three evaluators scored nine works.
Code and web: The three small projects scored 310, 306, and 305 out of 312. All three models received full scores on all four web projects. For a 248-line code refactoring task with 401 hidden tests, Hy4 passed all three runs; Qwen4 and Hy4 were tied with all tests passing, while DS broke the free-shipping logic in one run.
Output usage: The author's aggregate output token counts across all tests were 17.73 million for Hy4, 33.20 million for DS, and 8.57 million for Qwen4. For web tasks, the counts were about 42,000, 64,000, and 15,000, respectively. The author says these figures include reasoning and continuations, but they are not a price comparison.
Math completion: Across 102 attempts, Hy4 completed 42 within 20 minutes and answered 10 correctly; another 59 timed out, and one was blocked by content filtering. DS completed 65 and answered 8 correctly; Qwen4 completed 94 and answered 16 correctly. The problems came from new arXiv papers published in September 2026, curated by MathArena. They required exact numerical or set answers and awarded no partial credit for reasoning.
3D creations: There were two major tasks and six creations per model. On the first attempt, Hy4 produced two working creations and one partially usable one; DS produced 5/6 working creations; Qwen4 produced 2/6. The average quality scores for working creations were 6.0, 6.4, and 7.1, respectively. All four of Qwen4's non-working creations became runnable after one repair; one of Hy4's creations remained partially usable after repair.
| Task | Denominator or evaluation method | Hy4 | DS | Qwen4 |
|---|---|---|---|---|
| Maze shortest path | 18 problems; count shortest-path steps | 6 | 1 | 15 |
| Dual-robot grid planning | 18 runs; checked by program simulation | 15 | 17 | 18 |
| Paper math problems | 34 problems × 3 runs; 20-minute limit | 10/102 | 8/102 | 16/102 |
| Small projects | 3 projects; hidden tests; 312 points maximum | 310 | 306 | 305 |
| Web pages | 4 web projects; script checks for each | Full score on all | Full score on all | Full score on all |
| 3D creations runnable on first attempt | 6 creations; 3 runs for each of 2 tasks | 2 working, 1 partially usable | 5/6 | 2/6 |
| Output tokens across all tests | Author's aggregate; includes continuations, etc. | 17.73 million | 33.20 million | 8.57 million |
The raw figures in the table come from the author's article. Although the text describes the tests and scoring rules, it does not provide the prompts for each task, all model outputs, hidden tests, complete raw score sheets, or materials for recalculation. Some total-score tables appear as images, so not every cell can be checked against the readable text.
This is an individually organized, cross-platform comparison using the same tasks. It includes clearly described tasks, sample denominators for some results, and several scoring methods, so it can offer clues for choosing a model by task. The platforms and APIs differ, and the specific model versions, reasoning parameters, and full run configurations were not disclosed; the author explicitly says timing was affected by the platforms. The author argues that token counts are "not affected by the platform," but provides no per-API accounting rules for reasoning tokens or verifiable logs. The total token figures should therefore be treated as author-reported values, not direct cross-service cost comparisons. The design sample is small, the blind-review scores lack a defined scale and dispersion data, and only one evaluator—who knew the model identities—scored the 3D creations. Math accuracy is reported over 102 attempts, and the time limit affects completion rates, so these results should not be read as general math ability scores. The report does not compare prices.
Open the author's original X post in Tabbit and check the long-form text embedded in the post, the date, task descriptions, and results. The canonical long-form URL linked from the original post is https://x.com/servasyy_ai/article/2120095290728522127; on this attempt, X rate-limited direct access. No attempt was made to bypass the access restriction. Some score tables mentioned in the public text are images, and the underlying prompts and records were not made public, so only the visible text was checked; this was not an independent replication.
Hy4