3 ms·
It has good potential and nobody ever thought that RNN's would be able to scale as they have led by BlinkDL. Well we did at Stability AI which is why we provid
by emadm 3y ago
It has good potential and nobody ever thought that RNN's would be able to scale as they have led by BlinkDL.
Well we did at Stability AI which is why we provided and scaled compute for it.
They have the potential for far better edge inference and also benefits longer context windows.
There are other architectures out there too we are supporting, I would be surprised if the current transformer LLM paradigm is what we use in a few years.
- polygamous_bat 3y agoWith all due respect, I consider you as one of the "RWKV folks" I mentioned in the initial comment :) (assuming you are Emad Mostaque?) > They have the potential for far better edge inference and also benefits longer context windows. ... I would be surprised if the current transformer LLM paradigm is what we use in a few years. What are the studies/evidence that convinced you? Given the current landscape, the last claim is quite bold.
- pico_creator 3y agoWhat other architectures should we be taking note of (that stability is supporting)? Especially in the text space that you feel will change the current transformer LLM paradigm?