3 ms·

> I've been adamantly against MoE for a set of reasons Such as?

• lucubratory 3y ago
  It was completely unsubstantiated, based on rumours from a blog, but everyone repeated it as fact.
  • whimsicalism 3y ago
    I think it is pretty compelling that almost all of the people doing research into switch transformers at Google were hired into OAI. I am not sure if that is ouboicly reported but once Ghotz leaked those details about the models, I went to check where the authirs of those papers are now and.... yep