Occam's razor in a lot of ways is a proxy for Bayes' theorem. The theorem "punishes" more complex models by shrinking the prior term and increasing the marginal

Occam's razor in a lot of ways is a proxy for Bayes' theorem. The theorem "punishes" more complex models by shrinking the prior term and increasing the marginal term, while the likelihood is pegged at P = 1.0 at most. You can come up with a model that perfectly explains the outcome ( P(e|H) = 1 ) but you have to offset the insane improbability of pegging all those free parameters at the value they are at ( P(H) ).