4 ms·

It's the mechanism by which Occam's razor works. A monstrous spaghetti codebase could absolutely produce the universe we see. The problem is that many, many suc

It's the mechanism by which Occam's razor works. A monstrous spaghetti codebase could absolutely produce the universe we see. The problem is that many, many such codebases fit equally well. If you wanted to you could say that the universe just is the way it is, specifying the state completely. But you could always say that, no matter what the universe is! As it turns out though, huge swaths of the universe's behavior can be produced without all that individual specification. There are massive holes in our understanding, but at pretty much every turn for the last few centuries we've gotten better at predicting more stuff with fewer equations. Does that mean it'll continue? No idea! But that's why there's a real sense that a simple universe is more 'probable' than any particular complicated one. Now maybe no simple universe can possibly fit ours, in which case we're just kinda fucked.

akiselev 6y ago
Occam's razor isn't a law of nature, a proven theorem, or even a testable hypothesis. It's a rule of thumb and a poor one for analyzing systems with extreme unknowns like the nature of the universe.
- ravi-delia 6y ago
  I get why you're saying that, I don't think I explained what I was trying to say well. Occam's razor doesn't say that the universe is simple, only that of two theories that make identical predictions you should choose the simpler. That much makes sense on better grounds than a rule of thumb. You're correct regarding the extreme unknowns, since of all possible universe-rules most are complex. However, should a simple universe-rule correspond with our own, it is more likely to be correct than any single complex universe-rule. More importantly, the whole testable hypothesis thing? That relies heavily on the same intuition. Test results mean nothing if the universe can cheat, and in fact the universe can cheat. But we disregard the billions of possible cheating universes eating the same slice of the probability pie in favor of the non-cheating universe, should it exist.
  - akiselev 6y ago
    > should a simple universe-rule correspond with our own, it is more likely to be correct than any single complex universe-rule. Occam's razor does not apply to evolutionary systems because it would require analyzing every step of the chain like every mutation that increased the probability of survival in the environment it occurred in. The path from archae to amphibians, for example, was very convoluted, complex, and nonsensical, let alone specific examples like amphibians that became hooved mammals that crawled back into the ocean to become marine mammals. The simple explanation would be a fish that evolves lungs to outcompete gills, not an auroch that learns to swim! Fact is, we don't even know if the universe is an evolutionary system or not. We don't know whether our universe just happened or if we exist in a universe that is the conclusion of a bunch of previous universes that fizzled out of existing or didn't support life and evolved into this one or if we exist in an infinity of universes with an infinite permutation of the laws of nature. We don't even know what "simple" really means in this context. We just don't know. O(explanation) = ??? and P(explanation) = ??? for every value of `explanation`.
    - ravi-delia 6y ago
      I get the sense we're talking past each other here. It seems pretty likely that's because you kindly assume I'm not a complete moron, and I'm not performing. I'm confused about the evolution example; it seems to me that evolution is something of a triumph of Occam's razor. After all, for all it's myriad complexity the simple rule of natural selection explains pretty much all of it. My gut says that the analogous component to 'natural selection' would be the laws of physics, but I might not be understanding that last bit. Are you going for more of an anthropological principle thing?
      - akiselev 6y ago
        > I'm confused about the evolution example; it seems to me that evolution is something of a triumph of Occam's razor. After all, for all it's myriad complexity the simple rule of natural selection explains pretty much all of it. The theory of natural selection is the simplest explanation to our observations of the fossil record, genetics, etc., not an explanation for why marine mammals evolved from hooved land mammals instead of fish directly - except in the most trivial sense of "it is because it is" (a la anthropological principle). Why did nature select for that path instead of the more direct one? We can't even begin to formulate that question for the universe and its laws as a whole. The laws of physics in this case are the mammals and the process of natural selection of universes (if there even is one) is a complete mystery because we don't have anything to compare it to establish probabilities. Whether that is even a valid question to ask is still in the realm of philosophy and Ivory tower theoretical physics.
- chriswarbo 6y ago
  Sounds like you may enjoy Algorithmic Information Theory. In particular, Solomonoff Induction is the most information-efficient method of inducing computable rules from observations https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_inductive_inference https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_induc... It uses Bayes rule to update hypotheses based on observations; and its initial hypothesis is the "universal prior", which assigns probabilities to observations based on how likely they are to be output by a randomly-generated computer program (more specifically, an infinite sequence of uniformly-random bits, interpreted as a self-delimiting, Turing-complete language). The universal prior is uncomputable, but has been proven to dominate all computable probability distributions. Whilst computable approximations of the universal prior, like https://en.wikipedia.org/wiki/Speed_prior https://en.wikipedia.org/wiki/Speed_prior , aren't as information-efficient, they are more time-efficient, and approach the universal prior when given more time to run (they're identical in the limit of using infinite time). This is relevant for Occam's Razor, since the universal prior itself is dominated by the shortest program that outputs an observation, i.e. the best prediction we can make about what any computable system will do next, is to look for the simplest description of (AKA shortest program outputting) everything we've seen it do so far, and make a prediction based on what that program does next. When we say the shortest program "dominates", this means that even if every longer program agreed on some different prediction, the shortest program's prediction is still more likely (it's related to the Zeno-like summation https://en.wikipedia.org/wiki/1/2_%2B_1/4_%2B_1/8_%2B_1/16_%2B_%E2%8B%AF https://en.wikipedia.org/wiki/1/2_%2B_1/4_%2B_1/8_%2B_1/16_%... ). Hence Occam's Razor is proven to be the optimal method of predicting the behaviour of any computable system. Whether or not it's a "law of nature" depends on how we define that phrase philosophically; however, I would say the Church-Turing thesis certainly counts as one, and others agree; e.g. from https://existentialtype.wordpress.com/2012/08/09/churchs-law https://existentialtype.wordpress.com/2012/08/09/churchs-law > This is all just another instance of Church’s Law, the scientific law stating that any formalism for defining computable functions will turn out to be equivalent to, say, the λ-calculus when it comes to definability of number-theoretic functions. (Ordinarily Church’s Law is called Church’s Thesis, but for reasons given in my Practical Foundations book, I prefer to give it the full status of a scientific law.) (I highly recommend that book BTW; preprints are available online) The "physical Church-Turing thesis" is a physical law akin to relativity (it states that a particular mathematical formalism corresponds to "the real world"; despite the latter having no precise definition, and hence being mathematically unprovable, but empirically testable) https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#Variations https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#V... Hence we have a physical law which tells us the universe is computable (or, if you prefer a slightly weaker statement, that no computable method can distinguish observations of a computable univese from those of an uncomputable universe, given a finite number of observations; hence even if "the real world" were uncomputable, that wouldn't have any impact on our observations) Combining these two: Occam's Razor is a provable result of a rich and rigorous mathematical theory, and applies empirically to the universe via accepted physical law. As far as testable hypotheses go, we can perform a grue/bleen experiment quite easily ( https://en.wikipedia.org/wiki/New_riddle_of_induction https://en.wikipedia.org/wiki/New_riddle_of_induction ). Here's one to try at home: - Take any existing simulation S of a physical system P which we can compare against experiment, e.g. a computational fluid dynamics simulation of our kitchen sink. - Ensure that a "prior run" of P agrees with a "prior run" of S, e.g. starting from the same initial conditions and lasting one minute. If they don't agree, go back to step 1 and alter the setup. - Alter the code of S to get a new simulation S', which behaves identically to S during a prior run (e.g. the first minute), but differently when allowed to run longer. - Try to shrink both S and S'. We can't do this exactly (since Kolmogorov Complexity is uncomputable), but we can use systematic approaches, e.g. refactorings, inlining, applying compression algorithms, etc. which should hopefully be unbiased. - Our null hypothesis is that whichever ends up shorter (S or S'), its behaviour following the "prior run" is more likely to match the real behaviour of P than the longer program (of course neither might match; but if one of them does, we'd predict it to be the shorter)
enkid 6y ago
I totally disagree that we are predicting more stuff with fewer equations. The list of equations we have have exponentially grown since the start of the 20th Century.
- ravi-delia 6y ago
  Has it? I'd contest that it's grown at all, let alone exponentially. How would that work? I guess there's an argument to be made that equations beget equations, but it's hard to think of evidence of that. Gas equations are emergent from statistical mechanics are emergent from quantum mechanics, Optics are emergent from quantum mechanics, gravitation is emergent from relativity. Now sure, relativity is a good deal more complicated than Newtonian mechanics, but saying it makes better predictions does it a disservice. It explains all the normal stuff, and also a bunch of weird stuff, while still being pretty simple and IMO way more elegant than what came before.
  - enkid 6y ago
    Its not like we get rid of the old equations. We still have the Newtonian equations, in addition to special/general relativity. We still have Maxwell's equations in addition to quantum mechanics. They are useful in different situations, but they are still useful.
    - ravi-delia 6y ago
      Ohh OK, I see what you're saying. I more meant to say that the smallest possible set of equations from which the rest could be derived is shrinking.
    - ssivark 6y ago
      The “more fundamental” equations replace the “less fundamental” ones, which can then be derived as approximations or limiting cases valid in restricted regimes (Eg: Newton’s laws of motion + gravitation from the weak field approximation of General Relativity + the low speeds regime).
JetSetWilly 6y ago
Occam’s razor isn’t applicable here, though: If you have two explanations for a phenomenon, one of complexity level 5 and one of complexity level 10, Occam’s razor would say choose the simplest. But if you have one explanation of complexity level 10, Occam’s razor does not say that you should reject it because a simpler explanation must exist. It makes no statement that hypotheses have to be simple.
- ravi-delia 6y ago
  > Now maybe no simple universe can possibly fit ours, in which case we're just kinda fucked. Last paragraph, last sentence. Now between you and me, I'm suspicious that we'll just keep describing more universe with simpler models forever, but you're right that Occam's razor doesn't guarantee that.
harperlee 6y ago
Sounds more Kolgomorovian than Occamian to me... https://en.m.wikipedia.org/wiki/Kolmogorov_complexity https://en.m.wikipedia.org/wiki/Kolmogorov_complexity Edit: so the argument would be something along the lines of: the less we need to describe the universe, the more probable it is that it exists (due to the fact that, given that it does in fact exist, the more complex it is, the more different universes could exist in its place). That's of course assuming there is one universe! If we assume that there can be multiple universes then there is no easy way to posit that a simpler one is more probable.
- ravi-delia 6y ago
  You caught me!

akiselev 6y ago

Occam's razor isn't a law of nature, a proven theorem, or even a testable hypothesis. It's a rule of thumb and a poor one for analyzing systems with extreme unknowns like the nature of the universe.

ravi-delia 6y ago
I get why you're saying that, I don't think I explained what I was trying to say well. Occam's razor doesn't say that the universe is simple, only that of two theories that make identical predictions you should choose the simpler. That much makes sense on better grounds than a rule of thumb. You're correct regarding the extreme unknowns, since of all possible universe-rules most are complex. However, should a simple universe-rule correspond with our own, it is more likely to be correct than any single complex universe-rule. More importantly, the whole testable hypothesis thing? That relies heavily on the same intuition. Test results mean nothing if the universe can cheat, and in fact the universe can cheat. But we disregard the billions of possible cheating universes eating the same slice of the probability pie in favor of the non-cheating universe, should it exist.
- akiselev 6y ago
  > should a simple universe-rule correspond with our own, it is more likely to be correct than any single complex universe-rule. Occam's razor does not apply to evolutionary systems because it would require analyzing every step of the chain like every mutation that increased the probability of survival in the environment it occurred in. The path from archae to amphibians, for example, was very convoluted, complex, and nonsensical, let alone specific examples like amphibians that became hooved mammals that crawled back into the ocean to become marine mammals. The simple explanation would be a fish that evolves lungs to outcompete gills, not an auroch that learns to swim! Fact is, we don't even know if the universe is an evolutionary system or not. We don't know whether our universe just happened or if we exist in a universe that is the conclusion of a bunch of previous universes that fizzled out of existing or didn't support life and evolved into this one or if we exist in an infinity of universes with an infinite permutation of the laws of nature. We don't even know what "simple" really means in this context. We just don't know. O(explanation) = ??? and P(explanation) = ??? for every value of `explanation`.
  - ravi-delia 6y ago
    I get the sense we're talking past each other here. It seems pretty likely that's because you kindly assume I'm not a complete moron, and I'm not performing. I'm confused about the evolution example; it seems to me that evolution is something of a triumph of Occam's razor. After all, for all it's myriad complexity the simple rule of natural selection explains pretty much all of it. My gut says that the analogous component to 'natural selection' would be the laws of physics, but I might not be understanding that last bit. Are you going for more of an anthropological principle thing?
    - akiselev 6y ago
      > I'm confused about the evolution example; it seems to me that evolution is something of a triumph of Occam's razor. After all, for all it's myriad complexity the simple rule of natural selection explains pretty much all of it. The theory of natural selection is the simplest explanation to our observations of the fossil record, genetics, etc., not an explanation for why marine mammals evolved from hooved land mammals instead of fish directly - except in the most trivial sense of "it is because it is" (a la anthropological principle). Why did nature select for that path instead of the more direct one? We can't even begin to formulate that question for the universe and its laws as a whole. The laws of physics in this case are the mammals and the process of natural selection of universes (if there even is one) is a complete mystery because we don't have anything to compare it to establish probabilities. Whether that is even a valid question to ask is still in the realm of philosophy and Ivory tower theoretical physics.
chriswarbo 6y ago
Sounds like you may enjoy Algorithmic Information Theory. In particular, Solomonoff Induction is the most information-efficient method of inducing computable rules from observations https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_inductive_inference https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_induc... It uses Bayes rule to update hypotheses based on observations; and its initial hypothesis is the "universal prior", which assigns probabilities to observations based on how likely they are to be output by a randomly-generated computer program (more specifically, an infinite sequence of uniformly-random bits, interpreted as a self-delimiting, Turing-complete language). The universal prior is uncomputable, but has been proven to dominate all computable probability distributions. Whilst computable approximations of the universal prior, like https://en.wikipedia.org/wiki/Speed_prior https://en.wikipedia.org/wiki/Speed_prior , aren't as information-efficient, they are more time-efficient, and approach the universal prior when given more time to run (they're identical in the limit of using infinite time). This is relevant for Occam's Razor, since the universal prior itself is dominated by the shortest program that outputs an observation, i.e. the best prediction we can make about what any computable system will do next, is to look for the simplest description of (AKA shortest program outputting) everything we've seen it do so far, and make a prediction based on what that program does next. When we say the shortest program "dominates", this means that even if every longer program agreed on some different prediction, the shortest program's prediction is still more likely (it's related to the Zeno-like summation https://en.wikipedia.org/wiki/1/2_%2B_1/4_%2B_1/8_%2B_1/16_%2B_%E2%8B%AF https://en.wikipedia.org/wiki/1/2_%2B_1/4_%2B_1/8_%2B_1/16_%... ). Hence Occam's Razor is proven to be the optimal method of predicting the behaviour of any computable system. Whether or not it's a "law of nature" depends on how we define that phrase philosophically; however, I would say the Church-Turing thesis certainly counts as one, and others agree; e.g. from https://existentialtype.wordpress.com/2012/08/09/churchs-law https://existentialtype.wordpress.com/2012/08/09/churchs-law > This is all just another instance of Church’s Law, the scientific law stating that any formalism for defining computable functions will turn out to be equivalent to, say, the λ-calculus when it comes to definability of number-theoretic functions. (Ordinarily Church’s Law is called Church’s Thesis, but for reasons given in my Practical Foundations book, I prefer to give it the full status of a scientific law.) (I highly recommend that book BTW; preprints are available online) The "physical Church-Turing thesis" is a physical law akin to relativity (it states that a particular mathematical formalism corresponds to "the real world"; despite the latter having no precise definition, and hence being mathematically unprovable, but empirically testable) https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#Variations https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#V... Hence we have a physical law which tells us the universe is computable (or, if you prefer a slightly weaker statement, that no computable method can distinguish observations of a computable univese from those of an uncomputable universe, given a finite number of observations; hence even if "the real world" were uncomputable, that wouldn't have any impact on our observations) Combining these two: Occam's Razor is a provable result of a rich and rigorous mathematical theory, and applies empirically to the universe via accepted physical law. As far as testable hypotheses go, we can perform a grue/bleen experiment quite easily ( https://en.wikipedia.org/wiki/New_riddle_of_induction https://en.wikipedia.org/wiki/New_riddle_of_induction ). Here's one to try at home: - Take any existing simulation S of a physical system P which we can compare against experiment, e.g. a computational fluid dynamics simulation of our kitchen sink. - Ensure that a "prior run" of P agrees with a "prior run" of S, e.g. starting from the same initial conditions and lasting one minute. If they don't agree, go back to step 1 and alter the setup. - Alter the code of S to get a new simulation S', which behaves identically to S during a prior run (e.g. the first minute), but differently when allowed to run longer. - Try to shrink both S and S'. We can't do this exactly (since Kolmogorov Complexity is uncomputable), but we can use systematic approaches, e.g. refactorings, inlining, applying compression algorithms, etc. which should hopefully be unbiased. - Our null hypothesis is that whichever ends up shorter (S or S'), its behaviour following the "prior run" is more likely to match the real behaviour of P than the longer program (of course neither might match; but if one of them does, we'd predict it to be the shorter)

enkid 6y ago

I totally disagree that we are predicting more stuff with fewer equations. The list of equations we have have exponentially grown since the start of the 20th Century.

ravi-delia 6y ago
Has it? I'd contest that it's grown at all, let alone exponentially. How would that work? I guess there's an argument to be made that equations beget equations, but it's hard to think of evidence of that. Gas equations are emergent from statistical mechanics are emergent from quantum mechanics, Optics are emergent from quantum mechanics, gravitation is emergent from relativity. Now sure, relativity is a good deal more complicated than Newtonian mechanics, but saying it makes better predictions does it a disservice. It explains all the normal stuff, and also a bunch of weird stuff, while still being pretty simple and IMO way more elegant than what came before.
- enkid 6y ago
  Its not like we get rid of the old equations. We still have the Newtonian equations, in addition to special/general relativity. We still have Maxwell's equations in addition to quantum mechanics. They are useful in different situations, but they are still useful.
  - ravi-delia 6y ago
    Ohh OK, I see what you're saying. I more meant to say that the smallest possible set of equations from which the rest could be derived is shrinking.
  - ssivark 6y ago
    The “more fundamental” equations replace the “less fundamental” ones, which can then be derived as approximations or limiting cases valid in restricted regimes (Eg: Newton’s laws of motion + gravitation from the weak field approximation of General Relativity + the low speeds regime).

JetSetWilly 6y ago

Occam’s razor isn’t applicable here, though: If you have two explanations for a phenomenon, one of complexity level 5 and one of complexity level 10, Occam’s razor would say choose the simplest. But if you have one explanation of complexity level 10, Occam’s razor does not say that you should reject it because a simpler explanation must exist. It makes no statement that hypotheses have to be simple.

ravi-delia 6y ago
> Now maybe no simple universe can possibly fit ours, in which case we're just kinda fucked. Last paragraph, last sentence. Now between you and me, I'm suspicious that we'll just keep describing more universe with simpler models forever, but you're right that Occam's razor doesn't guarantee that.

harperlee 6y ago

Sounds more Kolgomorovian than Occamian to me... https://en.m.wikipedia.org/wiki/Kolmogorov_complexity https://en.m.wikipedia.org/wiki/Kolmogorov_complexity Edit: so the argument would be something along the lines of: the less we need to describe the universe, the more probable it is that it exists (due to the fact that, given that it does in fact exist, the more complex it is, the more different universes could exist in its place). That's of course assuming there is one universe! If we assume that there can be multiple universes then there is no easy way to posit that a simpler one is more probable.

ravi-delia 6y ago
You caught me!