Did you do any estimate of how much storage is required? On the face of it, I would expect the gradients to take about as much space as the weights. So you’d b

Did you do any estimate of how much storage is required? On the face of it, I would expect the gradients to take about as much space as the weights. So you’d be checkpointing your network at every batch, in effect.