4 ms·
TTT, cannon layers, and titans seem like a stronger approach IMO. Information needs to be compressed into latent space or it becomes computationally intractabl
by mountainriver 1y ago
TTT, cannon layers, and titans seem like a stronger approach IMO.
Information needs to be compressed into latent space or it becomes computationally intractable
- searchguy 1y agodo you have references to > TTT, cannon layers, and titans
- najarvg 1y agoThis was the nearest reference I could find. Links to an unofficial pytorch implementation on Github are also linked in the threads somewhere - https://www.reddit.com/r/LocalLLaMA/comments/1i0q8nw/titans_learning_to_memorize_at_test_time/ https://www.reddit.com/r/LocalLLaMA/comments/1i0q8nw/titans_...
- vessenes 1y agois titans replicated? I feel like lucidrains couldn't replicate.
- logicchains 1y agoI think something like Titans explains Gemini's excellent long context performance. That would explain why the Titan team hasn't released the training code or hyperpameters used even though they said in the paper that they would, and why soon after that it came out that DeepMind would be holding off publishing new results for 6 months to avoid giving away competitive advantages.
- deleted 1y ago[deleted]