Prior to that work on hyperparameter transfer though things would go wrong much more often training the big expensive model and require restarts back to checkpo

Prior to that work on hyperparameter transfer though things would go wrong much more often training the big expensive model and require restarts back to checkpoints and retuning parameters much more often though. Yeah it's still a lot of fiddling around but it is orders of magnitude less expensive. But that work also went far beyond transfer of fiddling around and mathematically solved a lot of stuff with avoiding overfitting, great video overview of it here: https://www.youtube.com/watch?v=1aXOXHA7Jcw https://www.youtube.com/watch?v=1aXOXHA7Jcw