4 ms·
I wrote about that a while ago: https://paxamans.github.io/blog/titans/ https://paxamans.github.io/blog/titans/
by nasvay_factory 10mo ago
I wrote about that a while ago: https://paxamans.github.io/blog/titans/ https://paxamans.github.io/blog/titans/
- moffkalast 10mo agoAre there any pretrained models with this architecture yet or is it all still completely theoretical beyond Google's unverifiable claims? They published the original Titans paper last year and nobody seems to have built on the idea.
- djrhails 10mo agohttps://github.com/lucidrains/titans-pytorch https://github.com/lucidrains/titans-pytorch - is the only public iteration. But no one appears to have taken the risk/time to properly validate it.
- AlexCoventry 10mo agoThe fundamental ideas in the paper aren't particularly novel. They will probably work as advertised.