3 ms·
> I failed to find the reason why ChatGPT worked as in contrast that gpt-3/2/1 never really worked (well) The answer may be as simple as "it has a lot more par
by blackbear_ 4y ago
> I failed to find the reason why ChatGPT worked as in contrast that gpt-3/2/1 never really worked (well)
The answer may be as simple as "it has a lot more parameters", plus the additional fine-tuning from human conversation data.
- GaggiX 4y agoGPT-3 has the same number of parameters as ChatGPT, it's just the finetuning (or alignment).
- est 4y agoI kind of doubt it? https://arxiv.org/abs/2203.02155 https://arxiv.org/abs/2203.02155 it says > outputs from the 1.3B parameter InstructGPT model are preferred to outputs from the 175B GPT-3, despite having 100x fewer parameters
- GaggiX 4y agohttps://openai.com/blog/chatgpt/ https://openai.com/blog/chatgpt/ >ChatGPT is fine-tuned from a model in the GPT-3.5 series https://platform.openai.com/docs/model-index-for-researchers/models-referred-to-as-gpt-3-5 https://platform.openai.com/docs/model-index-for-researchers... >GPT-3.5 series is a series of models that was trained on a blend of text and code from before Q4 2021. The following models are in the GPT-3.5 series: code-davinci-002 is a base model, so good for pure code-completion tasks text-davinci-002 is an InstructGPT model based on code-davinci-002 text-davinci-003 is an improvement on text-davinci-002 the davinci models are the biggest models by OpenAI, they have 175b parameters.