3 ms·
even in pre-training, usually 30%-50% is code these days.
by ahmadyan 17d ago
even in pre-training, usually 30%-50% is code these days.
- leothetechguy 17d agoThat would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.