4 ms·

Occam's razor isn't a law of nature, a proven theorem, or even a testable hypothesis. It's a rule of thumb and a poor one for analyzing systems with extreme unk

Occam's razor isn't a law of nature, a proven theorem, or even a testable hypothesis. It's a rule of thumb and a poor one for analyzing systems with extreme unknowns like the nature of the universe.

ravi-delia 6y ago
I get why you're saying that, I don't think I explained what I was trying to say well. Occam's razor doesn't say that the universe is simple, only that of two theories that make identical predictions you should choose the simpler. That much makes sense on better grounds than a rule of thumb. You're correct regarding the extreme unknowns, since of all possible universe-rules most are complex. However, should a simple universe-rule correspond with our own, it is more likely to be correct than any single complex universe-rule. More importantly, the whole testable hypothesis thing? That relies heavily on the same intuition. Test results mean nothing if the universe can cheat, and in fact the universe can cheat. But we disregard the billions of possible cheating universes eating the same slice of the probability pie in favor of the non-cheating universe, should it exist.
- akiselev 6y ago
  > should a simple universe-rule correspond with our own, it is more likely to be correct than any single complex universe-rule. Occam's razor does not apply to evolutionary systems because it would require analyzing every step of the chain like every mutation that increased the probability of survival in the environment it occurred in. The path from archae to amphibians, for example, was very convoluted, complex, and nonsensical, let alone specific examples like amphibians that became hooved mammals that crawled back into the ocean to become marine mammals. The simple explanation would be a fish that evolves lungs to outcompete gills, not an auroch that learns to swim! Fact is, we don't even know if the universe is an evolutionary system or not. We don't know whether our universe just happened or if we exist in a universe that is the conclusion of a bunch of previous universes that fizzled out of existing or didn't support life and evolved into this one or if we exist in an infinity of universes with an infinite permutation of the laws of nature. We don't even know what "simple" really means in this context. We just don't know. O(explanation) = ??? and P(explanation) = ??? for every value of `explanation`.
  - ravi-delia 6y ago
    I get the sense we're talking past each other here. It seems pretty likely that's because you kindly assume I'm not a complete moron, and I'm not performing. I'm confused about the evolution example; it seems to me that evolution is something of a triumph of Occam's razor. After all, for all it's myriad complexity the simple rule of natural selection explains pretty much all of it. My gut says that the analogous component to 'natural selection' would be the laws of physics, but I might not be understanding that last bit. Are you going for more of an anthropological principle thing?
    - akiselev 6y ago
      > I'm confused about the evolution example; it seems to me that evolution is something of a triumph of Occam's razor. After all, for all it's myriad complexity the simple rule of natural selection explains pretty much all of it. The theory of natural selection is the simplest explanation to our observations of the fossil record, genetics, etc., not an explanation for why marine mammals evolved from hooved land mammals instead of fish directly - except in the most trivial sense of "it is because it is" (a la anthropological principle). Why did nature select for that path instead of the more direct one? We can't even begin to formulate that question for the universe and its laws as a whole. The laws of physics in this case are the mammals and the process of natural selection of universes (if there even is one) is a complete mystery because we don't have anything to compare it to establish probabilities. Whether that is even a valid question to ask is still in the realm of philosophy and Ivory tower theoretical physics.
chriswarbo 6y ago
Sounds like you may enjoy Algorithmic Information Theory. In particular, Solomonoff Induction is the most information-efficient method of inducing computable rules from observations https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_inductive_inference https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_induc... It uses Bayes rule to update hypotheses based on observations; and its initial hypothesis is the "universal prior", which assigns probabilities to observations based on how likely they are to be output by a randomly-generated computer program (more specifically, an infinite sequence of uniformly-random bits, interpreted as a self-delimiting, Turing-complete language). The universal prior is uncomputable, but has been proven to dominate all computable probability distributions. Whilst computable approximations of the universal prior, like https://en.wikipedia.org/wiki/Speed_prior https://en.wikipedia.org/wiki/Speed_prior , aren't as information-efficient, they are more time-efficient, and approach the universal prior when given more time to run (they're identical in the limit of using infinite time). This is relevant for Occam's Razor, since the universal prior itself is dominated by the shortest program that outputs an observation, i.e. the best prediction we can make about what any computable system will do next, is to look for the simplest description of (AKA shortest program outputting) everything we've seen it do so far, and make a prediction based on what that program does next. When we say the shortest program "dominates", this means that even if every longer program agreed on some different prediction, the shortest program's prediction is still more likely (it's related to the Zeno-like summation https://en.wikipedia.org/wiki/1/2_%2B_1/4_%2B_1/8_%2B_1/16_%2B_%E2%8B%AF https://en.wikipedia.org/wiki/1/2_%2B_1/4_%2B_1/8_%2B_1/16_%... ). Hence Occam's Razor is proven to be the optimal method of predicting the behaviour of any computable system. Whether or not it's a "law of nature" depends on how we define that phrase philosophically; however, I would say the Church-Turing thesis certainly counts as one, and others agree; e.g. from https://existentialtype.wordpress.com/2012/08/09/churchs-law https://existentialtype.wordpress.com/2012/08/09/churchs-law > This is all just another instance of Church’s Law, the scientific law stating that any formalism for defining computable functions will turn out to be equivalent to, say, the λ-calculus when it comes to definability of number-theoretic functions. (Ordinarily Church’s Law is called Church’s Thesis, but for reasons given in my Practical Foundations book, I prefer to give it the full status of a scientific law.) (I highly recommend that book BTW; preprints are available online) The "physical Church-Turing thesis" is a physical law akin to relativity (it states that a particular mathematical formalism corresponds to "the real world"; despite the latter having no precise definition, and hence being mathematically unprovable, but empirically testable) https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#Variations https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#V... Hence we have a physical law which tells us the universe is computable (or, if you prefer a slightly weaker statement, that no computable method can distinguish observations of a computable univese from those of an uncomputable universe, given a finite number of observations; hence even if "the real world" were uncomputable, that wouldn't have any impact on our observations) Combining these two: Occam's Razor is a provable result of a rich and rigorous mathematical theory, and applies empirically to the universe via accepted physical law. As far as testable hypotheses go, we can perform a grue/bleen experiment quite easily ( https://en.wikipedia.org/wiki/New_riddle_of_induction https://en.wikipedia.org/wiki/New_riddle_of_induction ). Here's one to try at home: - Take any existing simulation S of a physical system P which we can compare against experiment, e.g. a computational fluid dynamics simulation of our kitchen sink. - Ensure that a "prior run" of P agrees with a "prior run" of S, e.g. starting from the same initial conditions and lasting one minute. If they don't agree, go back to step 1 and alter the setup. - Alter the code of S to get a new simulation S', which behaves identically to S during a prior run (e.g. the first minute), but differently when allowed to run longer. - Try to shrink both S and S'. We can't do this exactly (since Kolmogorov Complexity is uncomputable), but we can use systematic approaches, e.g. refactorings, inlining, applying compression algorithms, etc. which should hopefully be unbiased. - Our null hypothesis is that whichever ends up shorter (S or S'), its behaviour following the "prior run" is more likely to match the real behaviour of P than the longer program (of course neither might match; but if one of them does, we'd predict it to be the shorter)

Occam's razor isn't a law of nature, a proven theorem, or even a testable hypothesis. It's a rule of thumb and a poor one for analyzing systems with extreme unknowns like the nature of the universe.

ravi-delia 6y ago

I get why you're saying that, I don't think I explained what I was trying to say well. Occam's razor doesn't say that the universe is simple, only that of two theories that make identical predictions you should choose the simpler. That much makes sense on better grounds than a rule of thumb. You're correct regarding the extreme unknowns, since of all possible universe-rules most are complex. However, should a simple universe-rule correspond with our own, it is more likely to be correct than any single complex universe-rule. More importantly, the whole testable hypothesis thing? That relies heavily on the same intuition. Test results mean nothing if the universe can cheat, and in fact the universe can cheat. But we disregard the billions of possible cheating universes eating the same slice of the probability pie in favor of the non-cheating universe, should it exist.

akiselev 6y ago
> should a simple universe-rule correspond with our own, it is more likely to be correct than any single complex universe-rule. Occam's razor does not apply to evolutionary systems because it would require analyzing every step of the chain like every mutation that increased the probability of survival in the environment it occurred in. The path from archae to amphibians, for example, was very convoluted, complex, and nonsensical, let alone specific examples like amphibians that became hooved mammals that crawled back into the ocean to become marine mammals. The simple explanation would be a fish that evolves lungs to outcompete gills, not an auroch that learns to swim! Fact is, we don't even know if the universe is an evolutionary system or not. We don't know whether our universe just happened or if we exist in a universe that is the conclusion of a bunch of previous universes that fizzled out of existing or didn't support life and evolved into this one or if we exist in an infinity of universes with an infinite permutation of the laws of nature. We don't even know what "simple" really means in this context. We just don't know. O(explanation) = ??? and P(explanation) = ??? for every value of `explanation`.
- ravi-delia 6y ago
  I get the sense we're talking past each other here. It seems pretty likely that's because you kindly assume I'm not a complete moron, and I'm not performing. I'm confused about the evolution example; it seems to me that evolution is something of a triumph of Occam's razor. After all, for all it's myriad complexity the simple rule of natural selection explains pretty much all of it. My gut says that the analogous component to 'natural selection' would be the laws of physics, but I might not be understanding that last bit. Are you going for more of an anthropological principle thing?
  - akiselev 6y ago
    > I'm confused about the evolution example; it seems to me that evolution is something of a triumph of Occam's razor. After all, for all it's myriad complexity the simple rule of natural selection explains pretty much all of it. The theory of natural selection is the simplest explanation to our observations of the fossil record, genetics, etc., not an explanation for why marine mammals evolved from hooved land mammals instead of fish directly - except in the most trivial sense of "it is because it is" (a la anthropological principle). Why did nature select for that path instead of the more direct one? We can't even begin to formulate that question for the universe and its laws as a whole. The laws of physics in this case are the mammals and the process of natural selection of universes (if there even is one) is a complete mystery because we don't have anything to compare it to establish probabilities. Whether that is even a valid question to ask is still in the realm of philosophy and Ivory tower theoretical physics.

chriswarbo 6y ago

Sounds like you may enjoy Algorithmic Information Theory. In particular, Solomonoff Induction is the most information-efficient method of inducing computable rules from observations https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_inductive_inference https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_induc... It uses Bayes rule to update hypotheses based on observations; and its initial hypothesis is the "universal prior", which assigns probabilities to observations based on how likely they are to be output by a randomly-generated computer program (more specifically, an infinite sequence of uniformly-random bits, interpreted as a self-delimiting, Turing-complete language). The universal prior is uncomputable, but has been proven to dominate all computable probability distributions. Whilst computable approximations of the universal prior, like https://en.wikipedia.org/wiki/Speed_prior https://en.wikipedia.org/wiki/Speed_prior , aren't as information-efficient, they are more time-efficient, and approach the universal prior when given more time to run (they're identical in the limit of using infinite time). This is relevant for Occam's Razor, since the universal prior itself is dominated by the shortest program that outputs an observation, i.e. the best prediction we can make about what any computable system will do next, is to look for the simplest description of (AKA shortest program outputting) everything we've seen it do so far, and make a prediction based on what that program does next. When we say the shortest program "dominates", this means that even if every longer program agreed on some different prediction, the shortest program's prediction is still more likely (it's related to the Zeno-like summation https://en.wikipedia.org/wiki/1/2_%2B_1/4_%2B_1/8_%2B_1/16_%2B_%E2%8B%AF https://en.wikipedia.org/wiki/1/2_%2B_1/4_%2B_1/8_%2B_1/16_%... ). Hence Occam's Razor is proven to be the optimal method of predicting the behaviour of any computable system. Whether or not it's a "law of nature" depends on how we define that phrase philosophically; however, I would say the Church-Turing thesis certainly counts as one, and others agree; e.g. from https://existentialtype.wordpress.com/2012/08/09/churchs-law https://existentialtype.wordpress.com/2012/08/09/churchs-law > This is all just another instance of Church’s Law, the scientific law stating that any formalism for defining computable functions will turn out to be equivalent to, say, the λ-calculus when it comes to definability of number-theoretic functions. (Ordinarily Church’s Law is called Church’s Thesis, but for reasons given in my Practical Foundations book, I prefer to give it the full status of a scientific law.) (I highly recommend that book BTW; preprints are available online) The "physical Church-Turing thesis" is a physical law akin to relativity (it states that a particular mathematical formalism corresponds to "the real world"; despite the latter having no precise definition, and hence being mathematically unprovable, but empirically testable) https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#Variations https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis#V... Hence we have a physical law which tells us the universe is computable (or, if you prefer a slightly weaker statement, that no computable method can distinguish observations of a computable univese from those of an uncomputable universe, given a finite number of observations; hence even if "the real world" were uncomputable, that wouldn't have any impact on our observations) Combining these two: Occam's Razor is a provable result of a rich and rigorous mathematical theory, and applies empirically to the universe via accepted physical law. As far as testable hypotheses go, we can perform a grue/bleen experiment quite easily ( https://en.wikipedia.org/wiki/New_riddle_of_induction https://en.wikipedia.org/wiki/New_riddle_of_induction ). Here's one to try at home: - Take any existing simulation S of a physical system P which we can compare against experiment, e.g. a computational fluid dynamics simulation of our kitchen sink. - Ensure that a "prior run" of P agrees with a "prior run" of S, e.g. starting from the same initial conditions and lasting one minute. If they don't agree, go back to step 1 and alter the setup. - Alter the code of S to get a new simulation S', which behaves identically to S during a prior run (e.g. the first minute), but differently when allowed to run longer. - Try to shrink both S and S'. We can't do this exactly (since Kolmogorov Complexity is uncomputable), but we can use systematic approaches, e.g. refactorings, inlining, applying compression algorithms, etc. which should hopefully be unbiased. - Our null hypothesis is that whichever ends up shorter (S or S'), its behaviour following the "prior run" is more likely to match the real behaviour of P than the longer program (of course neither might match; but if one of them does, we'd predict it to be the shorter)