3 ms·

> Guessing the right theory of physics is equivalent to minimising predictive loss. No it's not. It's minimising "predictive loss" only under extreme non-stati

> Guessing the right theory of physics is equivalent to minimising predictive loss. No it's not. It's minimising "predictive loss" only under extreme non-statistical conditions imposed on the data. The world itself can be measured an infinite number of ways. There are an infinite number of irrelevant measures. There are an infinite number of low-reliability relevant measures. And so on. Yes, you can formulate the extremely narrow task of modelling "exactly the right dataset" as loss minimization. But you cannot model the production of that dataset this way. Data is a product of experiments.

XorNot 4y ago
This is just you declaring "no you can't" without supporting that in any way. How is a theory of physics not a loss minimisation process? The history of science is literally described in these terms i.e. the Bohr model of the atom is wrong, but also so useful that we still use it to describe NMR spectroscopy. Why did we come up with it? Because their aren't infinite ways to measure the universe, there are in fact very limited ways defined by our technology. Good ones, high loss minimisation, generally then let us build better technology to find more data. You're invoking infinities which don't exist as a handwave for "understanding is a unique part of humanity" to try and hide that this is all metaphysical special pleading.
- mjburgess 4y ago
  Alright... What loss was being minimised to find F=GMm/r^2? Or any law of physics you like.
  - XorNot 4y ago
    Gravitation was literally about predicting future positions of the stars, and was successful because it did so much better then any geocentric model. How is that not a loss minimization activity? And before we had it, epicycles were steadily increasing in complexity to explain every new local astronomical observation, but that model was popular because it gives a very efficient initial fit of the easiest data to obtain (i.e. the moon actually does go around the Earth, and with only 1 reference point the Sun appears to go round the Earth too). But of course once you have a heliocentric theory, you can throw all those parameters and every new prediction lines up nearly perfectly (accounting for how much longer it would take before we had precise enough orbital measurements to need Relativity to fully model it).
    - sudosysgen 4y ago
      When the law of gravitation was formulated, it could not in fact be used to predict orbits reliably (Kepler's ellipses are the solution to the two body problem anyways, and for a more complex system integration was impossible to any useful precision at the time), and Kepler's theories came out long before it did. It took more than 70 years after its formulation for the law to actually be conclusively tested against observations in a conclusive manner.
      - XorNot 4y ago
        Sure but it was a better fit, and before that heliocentric models were definitely the only way forward that didn't keep adding terms every time someone spotted a moon. Occam's razor - do not multiply terms without necessity - is essentially a loss function.
        sudosysgen 4y ago
        You're talking about Kepler's model here, not about the gravitational equation. The gravitational equation was not a better fit than Kepler at that time, especially since it used unknown constants.
        XorNot 4y ago
        So would you care to comment on how this relates to the original contention, which is the claim that a loss function could not discover Newton's law of gravitation? Because what you're arguing, extensively, is that due to lack of fit, Newton's Law of Gravitation wasn't settled science until observational data was of sufficient fidelity to clearly distinguish it. Which sure sounds like a loss function.
        mjburgess 4y ago
        Formulate the loss function -- you'll find it's just loss(the-right-answer(perfect-x) - perfect-y) The most important aspect of "the-right-answer" is its ability to ignore almost all the data. The existence of planets is "predictable" from the difference between the data and the theory -- if the theory is just a model of the data, it has no capacity to do this. If you want to "do physics" by brute force optimization you'd need to have all possible measures, all possible data, and then a way of selecting relevant causal structures in that data -- and then able to try every possible model. loss(Model(all-data|relevant-causal-structures) - Filter(...|...))) forall Model Of course, (1) this is trivially not computable (eqv. to computing the reals) -- (2) "all possible data with all possible measures" doesn't exist and (3) selecting relevant causal structure requires having a primitive theory not derived from this very process animals solve this in reverse order: (3) is provided by the body's causal structure; (2) is obtained by using the body to experiment; and (1) we imagine simulated ways-the-world-might-be to reduce the search space down to a finite size. ie., we DO NOT make theories out of data. We first make theories then use the data to select between them. This is necessary, since a model of the data (ie., modern AI, ie., automated statistics, etc.) doesnt decide between an infinite number of theories of how the data came to be.

XorNot 4y ago

This is just you declaring "no you can't" without supporting that in any way. How is a theory of physics not a loss minimisation process? The history of science is literally described in these terms i.e. the Bohr model of the atom is wrong, but also so useful that we still use it to describe NMR spectroscopy. Why did we come up with it? Because their aren't infinite ways to measure the universe, there are in fact very limited ways defined by our technology. Good ones, high loss minimisation, generally then let us build better technology to find more data. You're invoking infinities which don't exist as a handwave for "understanding is a unique part of humanity" to try and hide that this is all metaphysical special pleading.

mjburgess 4y ago
Alright... What loss was being minimised to find F=GMm/r^2? Or any law of physics you like.
- XorNot 4y ago
  Gravitation was literally about predicting future positions of the stars, and was successful because it did so much better then any geocentric model. How is that not a loss minimization activity? And before we had it, epicycles were steadily increasing in complexity to explain every new local astronomical observation, but that model was popular because it gives a very efficient initial fit of the easiest data to obtain (i.e. the moon actually does go around the Earth, and with only 1 reference point the Sun appears to go round the Earth too). But of course once you have a heliocentric theory, you can throw all those parameters and every new prediction lines up nearly perfectly (accounting for how much longer it would take before we had precise enough orbital measurements to need Relativity to fully model it).
  - sudosysgen 4y ago
    When the law of gravitation was formulated, it could not in fact be used to predict orbits reliably (Kepler's ellipses are the solution to the two body problem anyways, and for a more complex system integration was impossible to any useful precision at the time), and Kepler's theories came out long before it did. It took more than 70 years after its formulation for the law to actually be conclusively tested against observations in a conclusive manner.
    - XorNot 4y ago
      Sure but it was a better fit, and before that heliocentric models were definitely the only way forward that didn't keep adding terms every time someone spotted a moon. Occam's razor - do not multiply terms without necessity - is essentially a loss function.
      - sudosysgen 4y ago
        You're talking about Kepler's model here, not about the gravitational equation. The gravitational equation was not a better fit than Kepler at that time, especially since it used unknown constants.
        XorNot 4y ago
        So would you care to comment on how this relates to the original contention, which is the claim that a loss function could not discover Newton's law of gravitation? Because what you're arguing, extensively, is that due to lack of fit, Newton's Law of Gravitation wasn't settled science until observational data was of sufficient fidelity to clearly distinguish it. Which sure sounds like a loss function.
        mjburgess 4y ago
        Formulate the loss function -- you'll find it's just loss(the-right-answer(perfect-x) - perfect-y) The most important aspect of "the-right-answer" is its ability to ignore almost all the data. The existence of planets is "predictable" from the difference between the data and the theory -- if the theory is just a model of the data, it has no capacity to do this. If you want to "do physics" by brute force optimization you'd need to have all possible measures, all possible data, and then a way of selecting relevant causal structures in that data -- and then able to try every possible model. loss(Model(all-data|relevant-causal-structures) - Filter(...|...))) forall Model Of course, (1) this is trivially not computable (eqv. to computing the reals) -- (2) "all possible data with all possible measures" doesn't exist and (3) selecting relevant causal structure requires having a primitive theory not derived from this very process animals solve this in reverse order: (3) is provided by the body's causal structure; (2) is obtained by using the body to experiment; and (1) we imagine simulated ways-the-world-might-be to reduce the search space down to a finite size. ie., we DO NOT make theories out of data. We first make theories then use the data to select between them. This is necessary, since a model of the data (ie., modern AI, ie., automated statistics, etc.) doesnt decide between an infinite number of theories of how the data came to be.