3 ms·
It's a kind of an open secret that there's no 'training' protocol for these state of the art models. Researchers behave like alchemists when training these mod
by fooker 2y ago
It's a kind of an open secret that there's no 'training' protocol for these state of the art models.
Researchers behave like alchemists when training these models, and the actions are not really reproducible.
- rockinghigh 2y agoThey could provide access to the training code. It's useful for training smaller models or distilling larger ones. They don't need to release every details involved in tuning the optimization parameters during the pre-training stage.
- fooker 2y agoThere is no training 'code' that will get you anything close to a usable result.