Llama-modchip: 284B DeepSeek-V4-Flash Inference on 64GB CPU Without GPUs

The Llama-modchip project enables running 284B DeepSeek-V4-Flash and 30B MoE models on 64GB RAM CPUs without GPUs, achieving 44.4 tok/s prefill and 111.6 tok/s decode speeds using techniques like SwiGLU-H2 and Stockfish-PVS. It introduces hardware-specific optimizations for CPU-only inference, including fractal quantization and cache isolation, breaking previous memory constraints.

Comments

Read original article