2 ms·
It sounds like what he has always been talking about. Did he get new funding? It seems like there could be a loose relationship between having "many brains" an
by ilaksh 2y ago
It sounds like what he has always been talking about. Did he get new funding?
It seems like there could be a loose relationship between having "many brains" and having many experts in MoE. That's why I thought something like the Arctic Snowflake LLM or whatever they called it, or a model with LOTs of experts, could have an advantage. Although that one hasn't taken over any leaderboard, obviously.
The new DeepMind paper about transformers with NARs is really interesting also.
It seems like there are eventually going to be several really powerful but different approaches to general purpose AI.
- AlanYx 2y agoHe got new funding, but it's relatively modest ($2.69 million spread over two years).