Explorative Modeling: A Third Pretraining Axis for Enhanced Generative Models

Introduces Explorative Modeling (XMs) as a third pretraining axis that improves generative models by training them to explore multiple valid outputs, enhancing sample, FLOP, and parameter efficiency. Enables end-to-end generation with significant performance gains over traditional methods.

Comments

Read original article