A patched llama.cpp-style CPU inference stack with custom quantization and speculative decoding that claims to run DeepSeek-V4-Flash 284B in 64GB of RAM.
llama-modchip, a GitHub project from a developer called Synchopate, claims to run DeepSeek-V4-Flash, a 284-billion-parameter mixture-of-experts model, on an ordinary 64GB desktop with no GPU. It reached the Hacker News front page on November 8 with more than 260 points, a few days after its v2.3.0 and v2.4.0 releases on November 3 and 4.
Why it matters: running a frontier-scale open model locally usually means a rack of GPUs or a very expensive high-memory workstation. If the numbers hold up, a high-end consumer CPU and 64GB of RAM would be enough to run a model of this class slowly but fully offline. The catch is that every figure so far comes from the project itself.
How it fits
DeepSeek-V4-Flash has 284.33B total parameters but activates only about 13.17B per token, spread across 256 routed experts plus a shared one in each of its 43 layers, according to the project's README as summarized by WindowsForum. Only a small part of the model does work on each token, which is what makes CPU inference plausible at all.
The project's main tricks, per its documentation and both write-ups:
- Custom compression. A "SwiGLU-H2" scheme and a home-grown Fractal-64 quantization format, at about 3.75 bits per weight, cut the model file from 76.87 GiB (an existing IQ2_XXS build) to 56.56 GiB.
- Hand-tuned CPU kernels that tile work across AVX-512 registers and the L1, L2 and L3 caches. The project credits these with a 77% prefill gain over its own earlier build.
- Speculative decoding. A tiny 1.64MB draft component called RouteCrack guesses tokens, and a scheme the author calls "Stockfish-PVS" has the main model verify them.
- Huge pages and fewer experts. The architecture document says 2MB huge pages cut decode latency by 32.8% in its tests, and routing each token to four experts instead of six cut expert memory traffic by a third.
The claimed numbers
All results are from one AMD Ryzen 9 9955HX (16-core Zen 5) with 64GB of DDR5-5600 and no GPU.
| DeepSeek-V4-Flash, as reported | Tokens/sec |
|---|---|
| Prefill (reading the prompt) | 43.2–44.4 |
| Generation, no speculation | 9.1–10.7 |
| Generation with Stockfish-PVS | 11.5–14.7 |
| "Turbo" profile with speculation | 16.0 |
Resident memory at a 256K context is 57.6–58.9 GiB. Both write-ups note that a 64GB machine leaves only about 59.6 GiB usable, so there are roughly 0.7–2 GiB to spare for the operating system and everything else. WindowsForum also points out that the README's own server example runs with a 32K context, not 256K.
On a much smaller model, NVIDIA's Nemotron 3.5 30B-A3B, the architecture document reports 25–33 tokens/sec of pure decode on the same chip and about 20.7 tokens/sec average in a live Claude Code session. The v2.3.0 release notes cite a community result of 111.62 tokens/sec on a Threadripper PRO 7985WX.
What to be careful about
- Unverified. WindowsForum says the project's own evidence brief found no independent reproduction of the 284B-on-64GB result. LavX likewise labels it project-reported.
- Quality checks are narrow. The architecture document admits its pruning check used a four-case test suite. On the Nemotron model, its fast approximate-attention mode scored 0 of 10 on passkey retrieval, versus 10 of 10 with exact attention, which remains the default.
- Custom file format. Fractal-64 files use a nonstandard tensor type (ID 40). Unpatched llama.cpp, Ollama and LM Studio reject them, so you are tied to the project's runtime.
- Hardware floor. The project requires a CPU with AVX-512BW. The v2.4.0 notes add a Windows hybrid GPU+CPU pack, but the 284B result was measured on Linux.
- Release hygiene. The v2.2.0 release notes are still marked "DRAFT," so check which build you're getting.
Who should try it
If you have a recent AVX-512 desktop or laptop chip, 64GB of RAM and patience for about 10 tokens a second, this is an interesting experiment for private, offline use of a very large open model. Run it on a machine with nothing else open, and test it on your own prompts before trusting the quality. For everyday interactive use, the project's smaller-model profiles are more realistic.
Sources
- GitHub — synchopate/llama-modchip ARCHITECTURE.md github.com
- GitHub — synchopate/llama-modchip releases github.com
- WindowsForum — llama-modchip claims DeepSeek V4 Flash 284B runs on 64GB RAM without a GPU windowsforum.com
- LavX News — Synchopate says llama-modchip runs a 284B model on a 64GB CPU system news.lavx.hu
- Hacker News — Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU news.ycombinator.com