3 ms·
>Open source in AI means we should be able to reproduce their exact model. like: Use this specific RNG seed and train for 10 trillion cycles in this data to ge
by koolala 2y ago
>Open source in AI means we should be able to reproduce their exact model.
like: Use this specific RNG seed and train for 10 trillion cycles in this data to get X duplicate model?
- solveit 2y agoPracticality of reproducing training runs that cost tens of millions aside, it's hopeless. Determinism is hard enough with a single GPU, fixing a seed isn't going to be much help when training is distributed across hundreds of GPUs.