4 ms·
ChatGPT is 20B according to Microsoft researchers, also the fact that big AI models are trillion parameter models is mostly speculation, about GPT-4 it was spre
by GaggiX 3y ago
ChatGPT is 20B according to Microsoft researchers, also the fact that big AI models are trillion parameter models is mostly speculation, about GPT-4 it was spread by geohot.
- speedgoose 3y agoTo be precise, ChatGPT 3.5 turbo being 20B is officially a mistake from a Microsoft Researcher, quoting a wrong source published before the release of chatgpt3.5 turbo. Up to you to believe it or not. But I wouldn’t claim it’s a 20B according to Microsoft Researchers. The withdrawn paper: https://arxiv.org/abs/2310.17680 https://arxiv.org/abs/2310.17680 The wrong source: https://www.forbes.com/sites/forbestechcouncil/2023/02/17/is-bigger-better-why-the-chatgpt-vs-gpt-3-vs-gpt-4-battle-is-just-a-family-chat/ https://www.forbes.com/sites/forbestechcouncil/2023/02/17/is... The discussion: https://www.reddit.com/r/LocalLLaMA/comments/17jrj82/new_microsoft_codediffusion_paper_suggests_gpt35/ https://www.reddit.com/r/LocalLLaMA/comments/17jrj82/new_mic...
- GaggiX 3y agoIt's interesting how the paper was completely retracted instead of just being corrected.
- razodactyl 3y agoYep. It feels like a 20B parameter model.
- HarHarVeryFunny 3y agoGPT-3 was 175B, so it'd be a bit odd if GPT-4 wasn't at least 5x larger (1T), especially since it's apparently a mixture of experts.
- razodactyl 3y agoI think it became apparent when mixtral came out. I've noticed too during training that my model overwrites useful information so it makes sense for these types of models to have emerged.