6 ms·
Yeah, that's just the training stability part to my knowledge
by VHRanger 3y ago
Yeah, that's just the training stability part to my knowledge
- whimsicalism 3y agothey're also just less capable models. like just adding attention on top of an RNN made them a lot better
- SpaceManNabs 3y agoCalculating self-attention is still quadratic though. So you get the negatives of transformers there too.