2 ms·
MIT and Microsoft Researchers Introduce "RetNet" - An 8X Faster Transformer Alternative for AI
by wills_forward 3y ago
MIT and Microsoft Researchers Introduce "RetNet" - An 8X Faster Transformer Alternative for AI
- mirekrusin 3y agoThe claim is parallelism for training which is not fixed speed up, different complexity for inference (constant time), and different complexity for large context inference (linear) - so nothing that can be summarised as 8x - or am I getting this summary wrong?
- whimsicalism 3y agothe words per second i believe from the first graph in the paper
- canjobear 3y agoWhat’s the MIT connection? The authors all seem to be affiliated with Tsinghua.