Volumes have been written about this topic (principle of least assumption, parsimonious representation, Occam's razor, etc.). It underlies all statistical learn

Volumes have been written about this topic (principle of least assumption, parsimonious representation, Occam's razor, etc.). It underlies all statistical learning theory and information theory of which science is a special case.