3 ms·
Note that by "train" here they mean "finetune that network a bit", not train from scratch.
by MasterScrat 3y ago
Note that by "train" here they mean "finetune that network a bit", not train from scratch.
- UncleEntity 3y agoFor the 1.3 billion parameter one they do mean train from scratch on your 2 hour “coffee break”. Which totally begs the question, Microsoft AI researchers get two hour coffee breaks? —edit— The way they word it is confusing but, yeah, fine tune the model on your two hour coffee break.
- MasterScrat 3y agoNo, you can see the time breakdown in the table under the "coffee break" quote: it is the time for the 3-step RLHF process only. Training a 1.3B parameter model from scratch is still a very large undertaking.