2 ms·
You’re mostly right but conflating attention with autoregressive/causal which is the real issue that prevents you from using more compute You can use diffusion
by BarakWidawsky 5mo ago
You’re mostly right but conflating attention with autoregressive/causal which is the real issue that prevents you from using more compute
You can use diffusion with attention, and this model does in fact use attention
- samuelknight 5mo agoYes, I should have said autoregressive.