4 ms·
Retentive Network: A Successor to Transformer Implemented in PyTorch
- panabee 3y agonote: this is not from the original authors of the paper.
- ttctciyf 3y agoA retnet commit has now appeared in Microsoft's torchscale repo: https://github.com/microsoft/torchscale/commit/bf65397b26469ac9c24d83a9b779b285c1ec640b https://github.com/microsoft/torchscale/commit/bf65397b26469...
- westurner 3y ago"Retentive Network: A Successor to Transformer for Large Language Models" (2023) https://arxiv.org/abs/2307.08621 https://arxiv.org/abs/2307.08621 https://www.arxiv-vanity.com/papers/2307.08621/ https://www.arxiv-vanity.com/papers/2307.08621/ https://huggingface.co/papers/2307.08621 https://huggingface.co/papers/2307.08621 https://scholar.google.com/scholar?hl=en&as_sdt=0%2C33&q=Retentive+Network%3A+A+Successor+to+Transformer+for+Large+Language+Models&btnG= https://scholar.google.com/scholar?hl=en&as_sdt=0%2C33&q=Ret... - https://news.ycombinator.com/item?id=36831956 https://news.ycombinator.com/item?id=36831956 2 days ago