3 ms·
Teaching is used in the colloquial way, not in a technical ML sense. The model isn't being retrained or fine tuned.
by arram 6y ago
Teaching is used in the colloquial way, not in a technical ML sense. The model isn't being retrained or fine tuned.
- typon 6y agoThe prompt is essentially fine-tuning with extra steps
- minimaxir 6y agoNot really. Fine-tuning fundamentally changes the model weights to be more amendable to a particular use case/domain; the few-shot prompts for GPT-3 is just a guide (and it's very easy for the model to ignore said guides and go on a tangent). If you could finetune the 175B model, you'd likely get even better results for these Q&A prompts. (unclear how the OpenAI API is implementing its finetuning demo but I believe it's not on the 175B model).
- IshKebab 6y agoI think they only fine tune the top layers.