> They also don't post their code so I can't check that their problems with ADAM aren't due to using L2 regularization, The article indicated they used the sam

> They also don't post their code so I can't check that their problems with ADAM aren't due to using L2 regularization, The article indicated they used the same implementation on all device types, so I don't think L2 regularization would explain why the results are fine on CPU+GPU and only bad on TPU.