3 ms·
This isn't strictly true. This is comparing to "GPT3 as a few-shot learner"[1] as opposed to the fine tuned models that everyone else use. Few-shot GPT3 outper
by nl 6y ago
This isn't strictly true. This is comparing to "GPT3 as a few-shot learner"[1] as opposed to the fine tuned models that everyone else use.
Few-shot GPT3 outperforms a BERT-based baseline.
[1] https://github.com/openai/gpt-3 https://github.com/openai/gpt-3
- empiko 6y agoYeah, but they actually do fine-tune their model and compare it to in-batch GPT3 "learning". I called them out on it on Reddit and they claim that they use the same amount of data. I.e. GPT use X examples in the sample and they use the same amount of X samples to fine-tune. However, I am still not convinced that it is a fair comparison.