Explorative Modeling: A Third Pretraining Axis for Enhanced Generative Models

Explorative Modeling introduces a third pretraining axis that enhances generative models by enabling end-to-end generation through multi-step exploration, achieving significant efficiency gains. It addresses exposure bias and improves performance across images, video, and language by training models to generate multiple plausible guesses rather than averaging outputs.

Comments

Read original article