10 ms·
Any autoregressive model can do what you are describing. transformers are generating one token at a time too, not all at once.
by yshui 2y ago
Any autoregressive model can do what you are describing. transformers are generating one token at a time too, not all at once.
- whimsicalism 2y agoyes but transformers are much slower than state space models
- intalentive 2y agoTrue but memory requirements grow with sequence length. For recurrent models the memory requirement is constant. This is why I qualified with "low memory".