u/Lopsided_Dot_4557 shared a hands-on test of running GLM 5.2 (744B MoE) on a machine without a GPU, using colibrì, a pure-C engine that streams MoE experts from disk:
A 744B MoE activates only a small number of experts for each token: colibrì keeps the dense portion (about 10GB) resident in RAM and streams the experts selected by the router from disk as needed.
The complete int4 model occupies about 370GB on disk; it does not need to fit entirely in memory.
Test machine: a single machine with 132GB RAM, Ubuntu 22.04, and a local NVMe drive.
| Stage | Speed | Expert hit rate | RSS |
|---|---|---|---|
| First token after cold start | ~0.03 tok/s | ~21% | - |
| After several rounds of short prompts | ~0.15 tok/s | ~65% | - |
| Further warm-up | ~0.22 tok/s | ~71% | ~113GB |
The hit rate rises with use: the engine pins the experts that are actually routed to, so it gets faster the longer it runs.
Full hands-on video: https://youtu.be/jxML3S5C-8Y
Additional comment (Apple M5 Max 128GB): CPU/default, 1.06 tok/s (RSS 21.8GB); after warm-up with Metal cu --ram 96, 1.11→1.83 tok/s; with --ram 110, up to about 2.06 tok/s.
u/Sleepybear2611 (who conducted a three-day hands-on test of 754B): the rise in hit rate depends mainly on having a large amount of RAM (at RSS 113GB, about 30% of expert storage is held, and the LRU converges); on a 31GB low-memory machine, coverage can only settle at ~40–60%, because a single generation touches 38% of all experts and the working set cannot fit in the small memory; the cold-start figures match the prediction from their “bytes/token ÷ disk bandwidth” model (about 11GB/token).
"For the past month, I've been running local GLM 5.2 Q4 non-stop on my workstation rig, on repeat asking the prompt **'R… This is a necessary excerpt; read the original source for full context.
(A local, continuously running audit prompt that can be copied directly; observed result: about one round every 3.5 days, 10–15 new bugs per round, 1–2 of them hallucinations, and occasional impressive findings.)
Opposition (u/Littlepharaoh): 0.5 tok/s is too slow; the same task can be completed in a few minutes with serverless Runpod or a cheap API, at a cost of a few cents. Running locally is suitable for people whose countries prohibit international payments or who do not want to hand over their codebase.
Support (u/SV_SV_SV): a local 24/7 background audit does not give the code or data to anyone else, so the philosophical and privacy value is real.
Ecosystem: u/misanthrophiccunt mentioned that colibrì is adding DeepSeek V4 Flash support; u/Refinery73 asked whether small or medium-sized MoE versions are available—the current engine was built specifically for the GLM series (proof of concept).
What the conclusion covers: GLM 5.2 (744B MoE) can run on a machine with no GPU using only RAM + NVMe (pure CPU), making it suitable for “slow, background, long-running” code-audit tasks; its real throughput is far below that of cloud APIs, so it is not suitable for interactive development.
Parameter naming: The post calls it 744B (other sources, such as the GLM-5.3 release post, call it 743B; this may be a rounding difference); after int4 quantization, it is about 370GB.
Boundary: This is an experience report from a single-user environment with 132GB RAM; hit rate and throughput can drop substantially on machines with less memory. “10–15 bugs/3.5 days” is an observation from one user repeatedly running a single prompt, not a controlled benchmark.
Reproduction: The same order of magnitude can be reproduced with the colibrì engine, GLM-5.2 int4 weights, and the hardware described above; see the original post and video for the specific parameters.
GLM-5.2
