3 ms·
I didn't know 2 thirds of the training data would be source code.
by speedgoose 10d ago
I didn't know 2 thirds of the training data would be source code.
- jerrygenser 10d agothat is the the "data used to improve the model" when signing up for the subscription plans
- leothetechguy 10d agothis is the rl run, not the pretraining run
- ahmadyan 9d agoeven in pre-training, usually 30%-50% is code these days.
- leothetechguy 9d agoThat would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.