> the same loss can be achieved with many different combinations of parameters Perhaps it can be, but it isn't. The loss in a given model was achieved with th

> the same loss can be achieved with many different combinations of parameters Perhaps it can be, but it isn't. The loss in a given model was achieved with that particular combination of parameters that the model has, and there exists no other combination of parameters which which that model can appeal to for more information. To have the other combinations, we would need to train more models and then somehow combine them so that the combinations are available; but that's just conceptually the same as making one larger model with more parameters, and lower loss.