I also forgot that you have to constrain U and V to be orthonormal - otherwise you end up with the same low rank approximation D', but exactly which numbers go

I also forgot that you have to constrain U and V to be orthonormal - otherwise you end up with the same low rank approximation D', but exactly which numbers go in U, S, or V can shift as it's underdetermined without that constraint.