4 ms·
Retentive Network: A Successor to Transformer for Large Language Models
- vagabund 3y agoBrief twitter thread with performance metrics. Looks big if results hold up. https://twitter.com/arankomatsuzaki/status/1681113977500184576 https://twitter.com/arankomatsuzaki/status/16811139775001845...
- fgfm 3y agoThe repo the paper is pointing to indicates that the code will be released within ~1 week! If the sheer difference in VRAM requirements and latency holds up, this will seriously be a major breakthrough for LLM architectures! Can't wait to try it out
- deimos0x02 3y agohttps://github.com/Jamie-Stirling/RetNet https://github.com/Jamie-Stirling/RetNet