3 ms·
It’s how we trained roughly 40 GPT 1.5B models. The technique works; it’s up to you to try it out.
by sillysaurusx 4y ago
It’s how we trained roughly 40 GPT 1.5B models. The technique works; it’s up to you to try it out.
- zone411 4y agoThe abstract mentions fine-tuning, not full pre-training?
- sillysaurusx 4y agoYeah, sorry for not being precise. We used the technique to fine tune around 40 GPT 1.5B models, including the chess one. It was very apparent that the technique was working well. The kiss curve suddenly started dropping dramatically the first day we got it working.