20 ms·
Not bad for a start by the US government. A bit more than half as powerful as the 1.7 trillion parameter GPT4 model.
by Toolbox1337 3y ago
Not bad for a start by the US government. A bit more than half as powerful as the 1.7 trillion parameter GPT4 model.
- danpalmer 3y agoA bit more than half the size, it remains to be seen how powerful it is. There's clearly a non-linear relationship to model size, and it's also clear that it's hard to assess the power of these models anyway.
- Toolbox1337 3y agoAh, an important distinction to make. Good point.
- omneity 3y agoGPT-4 is unlikely to be 1.7T params. This is a number floating around in the internet with no justification. The largest US open model is Google’s Switch-C which is 1.6T and only because it is a Mixture of experts model, i.e. it is constituted of many small models working together.