4 ms·
It's worth noting that the GPT-3 paper did train models with more sane sizes (e.g. 1.5B) as a point of comparison. I am surprised/annoyed they never released th
by minimaxir 6y ago
It's worth noting that the GPT-3 paper did train models with more sane sizes (e.g. 1.5B) as a point of comparison. I am surprised/annoyed they never released them though.
- stellaathena 6y agoIt's because OpenAI sells them for profit. The "Ada" model is the same size as the larger of these two EleutherAI models.
- minimaxir 6y agoHuh, I was wondering what the size of the non-davinci models were; guess that make sense. It's still telling that a "small" GPT-3 model can risk cannibalizing a larger model.
- stellaathena 6y agoAda is 2.7B, Babbage is 6.7B, Curie is 13.0B, and DaVinci is 175B. The new one they announced last month is in the 20-50B range I think, not totally sure though.