3 ms·
After spending a lot of time working with GPT-3/the OpenAI API (https://github.com/minimaxir/gpt-3-experiments https://github.com/minimaxir/gpt-3-experiments ),
by minimaxir 6y ago
After spending a lot of time working with GPT-3/the OpenAI API (https://github.com/minimaxir/gpt-3-experiments https://github.com/minimaxir/gpt-3-experiments ), one notable part of GPT-3 is the high signal-to-noise ratio in generated output.
When finetuning GPT-2, only about 5-10% of the generated output is usable/coherent. But with GPT-3, easily 30%-40% of the generated text is usable/coherent, which is a big boost in quality.
- deleted 6y ago[deleted]