Neutrino-1 8B: 8.19B-Parameter Decoder-Only Transformer with Ternary-Family Weight Format

Neutrino-1 8B is an 8.19B-parameter decoder-only transformer using a proprietary ternary-family weight format that reduces model size by 8× compared to fp16, enabling cross-platform deployment on GPUs, CPUs, and Apple M5 devices. It achieves 763 tok/s throughput on H100 and 33.7 tok/s on M5 via optimized speculative decoding and hardware-specific implementations.

Comments

Read original article