2 ms·
> If replicated, it's arguably the first linear-time architecture with Transformer-quality performance!! RetNet was published in August (although I only learne
by smaddox 3y ago
> If replicated, it's arguably the first linear-time architecture with Transformer-quality performance!!
RetNet was published in August (although I only learned of it last week): https://arxiv.org/abs/2307.08621 https://arxiv.org/abs/2307.08621