Hands-on Test of DeepSeek V4's Agent Capabilities

Background

As an open-source, neutral Agent framework, CowAgent focuses on how models perform in real Agent workflows (tool calling, long contexts, long-term memory, browser automation, and knowledge organization). It tested DeepSeek V4 in six real-world scenarios, with this evaluation focusing primarily on deepseek-v4-flash.

Its price is roughly one-tenth that of Pro, a few dozen times lower than Claude Sonnet/Opus, and one-third that of MiniMax M2.7, while its response speed is faster.

Test environment

• Model: deepseek-v4-flash; extended thinking enabled, with reasoning_effort set to high by default (max can be used)

• Maximum of 50 steps per task; 20 rounds of conversation history retained; context token limit of 1 million

• Tools: 13 built-in tools (bash / edit / read / write / web_search / web_fetch / browser, and others)

• Skills: 30+ Skills (frontend-engineer / image-generation / video-gen / pptx-creator, and others)

Results from six scenarios

Scenario highlights

1. Task planning: 35 tool calls were restrained and precise; decomposition → research → consolidation → PPT → knowledge base completed in one run

2. Complex programming: Proactively wrote in chunks to avoid token truncation and called the browser to review screenshots (a stability fallback added by the model). Shortcoming: despite the requirement of "zero external dependencies," it still referenced the ECharts CDN

3. Long-term memory: In a brand-new session, just 2 tool calls retrieved all brand details (visual colors, suppliers, rent, and salaries) accurately, then produced structured recommendations

4. Browser automation: When it reached a login page, it proactively captured a QR code and paused for the user; before publishing, it stopped at the button and waited for confirmation, showing careful Agent safety details

5. Knowledge base: It split the material into concept pages, server pages, and client pages and linked them to one another. Each page ended with "Related reading," creating a genuine graph

6. Ultra-long context: Instead of stuffing the 3.36 MB full text into the context, it first saved it to disk, used grep to locate line numbers, and read it in segments—the long-context capability an Agent needs is knowing "what to put into context and when to use a tool"

Conclusion

• Zero failures, zero infinite loops: All six scenarios completed successfully in one run, the most significant upgrade from V3 to V4

• Fast responses: Every scenario finished in 2–6 minutes, with steps streamed as they ran and low perceived latency

• Flash is now stable enough to serve as the default Agent model, and its long-term memory and long-context performance exceeded expectations

• Weakness: It occasionally relaxes complex constraints (for example, still pulling in a CDN despite "zero external dependencies"); Pro is probably more reliable in complex scenarios