3 ms·
> It's not actually a 600B+ model. It's a mixture of experts. Is this described in the paper or was this inferred from the model itself ? Just curious, especi
by ur-whale 2y ago
> It's not actually a 600B+ model. It's a mixture of experts.
Is this described in the paper or was this inferred from the model itself ?
Just curious, especially if the latter.
- lopuhin 2y agoIt's a 600B+ mixture of experts and yes it's described in the paper, GitHub, etc.