5 ms·

I guess that's a good overview. It's related to the phenomenon of overfitting in machine learning: you can always easily find a sufficiently complex (or complic

I guess that's a good overview. It's related to the phenomenon of overfitting in machine learning: you can always easily find a sufficiently complex (or complicated, large if you prefer) theory fitting all data points. Because this theory simply encodes each observed case (including progressive sophistications of encoding), you naturally expect it to fail on unobserved cases -- it makes no effort at generalization. The simpler theories have a greater chance of generalization, they're more likely to be the "true" mechanism of the process that doesn't simply encode "edge cases" as you cited, and thus are more likely to also work on unobserved data. Honestly I haven't seen attempts at making this process more rigorous, when applied to physics. There's the large corpus of machine learning study which provides concrete results and even concrete comparison tools, but the times I've asked a physicist I've been dismissed; while it seems incredibly valuable in the face of lack or large cost of experimental data, which is quite relevant today.

chriswarbo 8y ago
> Honestly I haven't seen attempts at making this process more rigorous, when applied to physics. Marcus Hutter has expressed this idea quite well ( https://arxiv.org/pdf/0912.5434 https://arxiv.org/pdf/0912.5434 ) arguing that (a) smaller/simpler theories have more predictive power and (b) the "size" of a theory includes the complexity of its equations and the parameters needed to specify some result. The latter is important because some theories trade off between these two: e.g. a multiverse theory might have simple equations ("every possibility happens somewhere") but require very precise "coordinates" to pin-point the actual possibility that we observe. Not sure if other physicists know of or take it seriously though.
- darkmighty 8y ago
  Interesting. Apparently he needs to assume a particular multiverse theory to prove it. While I don't object to those on principle, I don't believe they're needed to prove heuristic, good enough versions of Ockham’s Razor that work in the real world (albeit without guarantees), based on the arguments outlined on the previous comments. > The latter is important because some theories trade off between these two: e.g. a multiverse theory might have simple equations ("every possibility happens somewhere") but require very precise "coordinates" to pin-point the actual possibility that we observe. I think this is an important observation that's quite obvious for ML researchers et al but again seems to escape current physics discussions. An example is the endless drama about "Fine tuning": if your new theory requires many less bits for equation description, that it requires fine tuning is irrelevant as long as the additional model parameter precision uses less bits -- then it should be the preferred candidate. W.r.t. [computational] multiverse theories (and variants such as Tegmark's MUH, Schimidhuber's, and others), I do believe they're an inevitable progression of physics/philosophy. I just think it's a bit pretentious to have any certain about a particular flavor. I feel there's still much philosophical and mathematical ground to be covered; it tests the limits of our imagination. It seriously feels like a very important step for humanity at large though -- finally approaching metaphysical theories that actually make sense, and explain the basis of much about humanity, existence, ethics, etc. I think it's an important void to be filled after the decline of religion, hopefully in coonjunction with the spread of humanism.

chriswarbo 8y ago

> Honestly I haven't seen attempts at making this process more rigorous, when applied to physics. Marcus Hutter has expressed this idea quite well ( https://arxiv.org/pdf/0912.5434 https://arxiv.org/pdf/0912.5434 ) arguing that (a) smaller/simpler theories have more predictive power and (b) the "size" of a theory includes the complexity of its equations and the parameters needed to specify some result. The latter is important because some theories trade off between these two: e.g. a multiverse theory might have simple equations ("every possibility happens somewhere") but require very precise "coordinates" to pin-point the actual possibility that we observe. Not sure if other physicists know of or take it seriously though.

darkmighty 8y ago
Interesting. Apparently he needs to assume a particular multiverse theory to prove it. While I don't object to those on principle, I don't believe they're needed to prove heuristic, good enough versions of Ockham’s Razor that work in the real world (albeit without guarantees), based on the arguments outlined on the previous comments. > The latter is important because some theories trade off between these two: e.g. a multiverse theory might have simple equations ("every possibility happens somewhere") but require very precise "coordinates" to pin-point the actual possibility that we observe. I think this is an important observation that's quite obvious for ML researchers et al but again seems to escape current physics discussions. An example is the endless drama about "Fine tuning": if your new theory requires many less bits for equation description, that it requires fine tuning is irrelevant as long as the additional model parameter precision uses less bits -- then it should be the preferred candidate. W.r.t. [computational] multiverse theories (and variants such as Tegmark's MUH, Schimidhuber's, and others), I do believe they're an inevitable progression of physics/philosophy. I just think it's a bit pretentious to have any certain about a particular flavor. I feel there's still much philosophical and mathematical ground to be covered; it tests the limits of our imagination. It seriously feels like a very important step for humanity at large though -- finally approaching metaphysical theories that actually make sense, and explain the basis of much about humanity, existence, ethics, etc. I think it's an important void to be filled after the decline of religion, hopefully in coonjunction with the spread of humanism.