2 ms·
this is the rl run, not the pretraining run
by leothetechguy 15d ago
this is the rl run, not the pretraining run
- ahmadyan 15d agoeven in pre-training, usually 30%-50% is code these days.
- leothetechguy 15d agoThat would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.