GLM-5.1: Artificial Analysis Independent Intelligence Index and Inference Throughput Benchmark

One-sentence takeaway

In third-party independent benchmark evaluations, GLM-5.1 (Reasoning) scored 41 on the Intelligence Index with a throughput of 82.7 tokens/s — placing it in the top 20% of its class and demonstrating high intelligence alongside fast generation speeds, though it tends to be more verbose and relatively expensive compared to other open-weight models.

Use cases

• Suitable tasks: Agentic interactions, complex terminal-based coding tasks, and structured reasoning where both deep reasoning and high throughput speed are required.

• Unsuitable tasks: Ultra-high-concurrency, budget-constrained scenarios with extreme sensitivity to per-token output costs or a strong preference for ultra-concise answers.

• Applicable model version: GLM-5.1 (Reasoning).

• Applicable clients, agents, or APIs: First-party Z.ai API and 8 major third-party API providers.

• Recommended reasoning tier and parameters: Official default reasoning tier; enabling Prompt Caching is strongly recommended (delivering up to an 81% cache discount).

Test environment and input/configuration

• Evaluation framework version: Artificial Analysis Intelligence Index v4.1.1.

• Composite evaluation subsets: Integrates 9 independent benchmarks in total:

  1. GDPval-AA v2 (real-world work tasks)

  2. 𝜏³-Banking (tool calling and banking operations)

  3. Terminal-Bench v2.1 (agentic coding and terminal operations)

  4. SciCode (scientific computing coding)

  5. Humanity's Last Exam (HLE) (frontier reasoning and knowledge)

  6. GPQA Diamond (scientific logical reasoning)

  7. CritPt (deep physics reasoning)

  8. AA-Omniscience (knowledge reliability and non-hallucination rate)

  9. AA-LCR (long-context reasoning)

• Model specifications: 744B total parameters, 40B active parameters per token (MoE architecture), and a 200k-token context window.

Results

Conclusion

Independent data from Artificial Analysis confirms that GLM-5.1 ranks among the leading tier of frontier large language models (scoring 41, well above the peer median of 27). Its output speed of 82.7 tokens/s delivers a responsive interactive user experience during long-horizon reasoning. However, it exhibits a tendency toward more verbose outputs (120M tokens evaluated), and its API pricing is on the higher side compared to other open-weight derived models. In production deployments, taking full advantage of Prompt Caching is strongly recommended to optimize inference costs.

Limitations

• This evaluation reflects a composite weighted index for Reasoning mode rather than an isolated stress test for a single domain.

• Pricing metrics are based on official list prices and provider medians at the time of testing; actual expenses may vary across API providers.

• While the 200k context window comfortably accommodates the vast majority of tasks, it still has an upper bound compared to models offering 1M-token windows.

Reproduction steps

1. Standardize input prompts in accordance with the Artificial Analysis benchmark protocol.

2. Connect to individual API provider endpoints and record Time to First Token (TTFT) and token generation speed (tokens/s).

3. Execute the Terminal-Bench v2.1, SciCode, and GPQA Diamond subsets, comparing weighted scores against output token lengths.

4. Compare end-to-end API billing costs with Prompt Caching enabled versus disabled.

Original evidence and data

The official Artificial Analysis leaderboard publicly records GLM-5.1's Intelligence Index score of 41, output speed of 82.7 tokens/s, input pricing of $1.39 / 1M tokens, output pricing of $4.40 / 1M tokens, and its MoE architectural specifications featuring 744B total parameters with 40B activated parameters per token.

Source excerpt or observation (compliant short quote only)

Artificial Analysis notes in its summary: "GLM-5.1 (Reasoning) is amongst the leading models in intelligence, but particularly expensive when comparing to other open weight models of similar size. It's also faster than average, however somewhat verbose."