It's moot in practice, because there's additional sources of randomness from executing in parallel. Though in principle saving random seeds is a lot less hassl

It's moot in practice, because there's additional sources of randomness from executing in parallel. Though in principle saving random seeds is a lot less hassle than keeping entire checkpoints around: your random seeds would fit on a floppy disk or even a tweet. The checkpoint is basically as big as the model.