2 ms·
It is a big deal because the major version numbers usually represent a new model trained from scratch, vs just a base model that has been given extra training (
by MattRix 4y ago
It is a big deal because the major version numbers usually represent a new model trained from scratch, vs just a base model that has been given extra training (which is what happened from GPT 3.0 -> 3.5)
- nl 4y agoInstructGPT (which ChatGPT/GPT-3.5) is based on was trained from scratch: > We train three model sizes (1.3B, 6B, and 175B parameters)[1] It does use the same model architecture as GPT-3 though so I guess there is a bit of reasoning there maybe. [1] https://arxiv.org/pdf/2203.02155.pdf https://arxiv.org/pdf/2203.02155.pdf