4 ms·
Very nice list. I didn´t knew about Heinsen Routing, looks very interesting. From my tests, SSMs are a very promising line of research and on my (small) tests
by bratao 3y ago
Very nice list. I didn´t knew about Heinsen Routing, looks very interesting.
From my tests, SSMs are a very promising line of research and on my (small) tests on S4, it really has better characteristics than transformers, as it learned faster, a larger context and with smaller dataset.
- cs702 3y agoAgree on SSMs: they look promising. They're on my list of "things to explore more thoroughly." I've done very little with them so far. I'm still making my way through the related papers, trying to get a superficial but accurate/intuitive understanding of these models.