I like this explanation that likens the search for principles to compression: The natural human urge to look for simple explanations is a compression problem.

I like this explanation that likens the search for principles to compression: The natural human urge to look for simple explanations is a compression problem. If you can find a simple mechanism for operating in a complex universe, you can teach it to your children more easily. Training AIs has a similar motivation — a simpler function is cheaper to store and compute. Perhaps we can call the simplest explanations we've found so far "Principles". We want to believe they are universally true. For example, conservation of energy, or humans should have equal rights. They seem to fit our current reality, and they are vast simplifications over doing things ad-hoc, but are they more than lucky simplifications of a complex universe?