3 ms·
Or another question, do they still publish any research that’s relevant for the field nowadays?
by balder1991 2y ago
Or another question, do they still publish any research that’s relevant for the field nowadays?
- awestroke 2y agoNo. They publish PDFs that hype up their models, but they do not publish anything even resembling a high-level overview of model architecture
- jacobgorm 2y agoGiven that you can download and use the weights, the model architecture has to be includded as part of that. And I did read a paper from them recently describing their MoE architecture and how it differs from the original GShard.