2 ms·
I don't really get the obsession with if the MS models is some variant of GPT-3.5 vs GPT-4. It's just a version number, and there aren't really meaningful reas
by nl 4y ago
I don't really get the obsession with if the MS models is some variant of GPT-3.5 vs GPT-4.
It's just a version number, and there aren't really meaningful reasons why something is a 0.5 increment vs a 1.0 release increment, other than "there are lots of changes" where lots ~= something.
- chessgecko 4y agoThe gap in quality between gpt2 and gpt3 was much much larger than the gap between gpt3 and 3.5. If they could get the gap between 2 -> 3 again when moving to 4 it would be crazy
- deleted 4y ago[deleted]
- MattRix 4y agoIt is a big deal because the major version numbers usually represent a new model trained from scratch, vs just a base model that has been given extra training (which is what happened from GPT 3.0 -> 3.5)
- nl 4y agoInstructGPT (which ChatGPT/GPT-3.5) is based on was trained from scratch: > We train three model sizes (1.3B, 6B, and 175B parameters)[1] It does use the same model architecture as GPT-3 though so I guess there is a bit of reasoning there maybe. [1] https://arxiv.org/pdf/2203.02155.pdf https://arxiv.org/pdf/2203.02155.pdf