Advancing Ternary Large Language Models: Efficient Bit-Width Optimization

The paper introduces BITCOS, a distribution-adaptive storage format for ternary LLMs that reduces bit-width requirements below the 1.58-bit theoretical minimum by exploiting zero density, achieving 1.485 bits per weight and up to 1.28× performance gains in matrix operations.

Comments

Read original article