I thought MSE was tied to the predictions it makes and not the structure itself (i.e. if you get the same value out for the same input, using different structur

I thought MSE was tied to the predictions it makes and not the structure itself (i.e. if you get the same value out for the same input, using different structures, you should at least get the same MSE).