> Consider the standard MSE loss in x-space with the popular l2-regularization on the weights (aka weight decay)... Am I missing something? I thought weight de

> Consider the standard MSE loss in x-space with the popular l2-regularization on the weights (aka weight decay)... Am I missing something? I thought weight decay and l2 regularization could be equivalent when using SGD[1], but are not in adaptive gradient scenarios(the paper mentions using Adam later on). [1] - https://arxiv.org/pdf/1711.05101.pdf https://arxiv.org/pdf/1711.05101.pdf