4 ms·
How did everyone solve it at the same time and there is no published paper (that I'm aware of) describing how to do it? It's like every AI researcher had an ep
by FergusArgyll 3y ago
How did everyone solve it at the same time and there is no published paper (that I'm aware of) describing how to do it?
It's like every AI researcher had an epiphany all at once
- tempusalaria 3y agoFirms are hiring from each other all the time. Plus there’s the fact that the base pertaining is being done at higher context lengths, so then the context extending fine tuning is working from a larger base
- fancyfredbot 3y agoA paper describing how you might do it published in December last year. The paper was "Mamba: Linear-Time Sequence Modeling with Selective State Spaces". To be clear I don't know if Claude and Gemini actually use this technique but I would not be surprised if they did something similar: https://arxiv.org/abs/2312.00752 https://arxiv.org/abs/2312.00752 https://github.com/state-spaces/mamba https://github.com/state-spaces/mamba