3 ms·
No. They publish PDFs that hype up their models, but they do not publish anything even resembling a high-level overview of model architecture
by awestroke 2y ago
No. They publish PDFs that hype up their models, but they do not publish anything even resembling a high-level overview of model architecture
- jacobgorm 2y agoGiven that you can download and use the weights, the model architecture has to be includded as part of that. And I did read a paper from them recently describing their MoE architecture and how it differs from the original GShard.