3 ms·
What does “pretrain” mean in this context? It sounds like normal training
by Havoc 3y ago
What does “pretrain” mean in this context? It sounds like normal training
- Filligree 3y agoAs opposed to fine-tuning or in-context learning. It really is normal training.
- rodonn 3y agoGPT stands for Generative Pre-trained Transformer. The "main" training step using huge amounts of inputs is called pre-training. The idea is that after that pre-training, you might fine tune the model for your specific use case.
- Havoc 3y agoI see...that makes sense. Thanks for explaining