4 ms·

Science isn't interested in this kind of prediction. That's just engineering. Causal models give counter-factual predictions for existence claims (eg., that a

Science isn't interested in this kind of prediction. That's just engineering. Causal models give counter-factual predictions for existence claims (eg., that a planet exists because the orbit of two other planets doesn't follow the causal model). Science, in most cases, prefers models with poor "engineering predictions" (ie., point estimates of observables) because they have vastly superior explanatory power. In most cases it would be a catastrophe for a scientific model to be making good estimates of observables, because we know a priori, that observables aren't fully determined by the model (eg., just consider that F=GMm/r^2 basically didnt apply to most observations of the solar system when it was formulated by newton; nor really does it much today). Explanatory power is not a property of compression, nor association, nor "prediction" in this engineering sense. Consider here that a lossless model of the solar system would never have yielded newton's law of graviton (since most of the objects in the solar system are unknown). This entire project is just, "what if science were like ML?" -- an interesting question only because how vast the gap is; and how absurd the suggestion.

dr_dshiv 3y ago
How can you tell the difference between a causal model and a predictive model? Isn’t it just the elegance of the model? And isn’t elegance just succinctness? Because another approach could focus on human comprehensibility. Indeed, a leading theory of scientific progress focuses on the advancement of noetic understanding. But, then we should really be irritated by things like quantum mechanics and love things like nutrition (we might get poor predictions but good understanding). I’m not sure myself.
- mjburgess 3y ago
  No, it's a completely different semantics. F = GMm/r^2 does not mean anything about {P(F|M), P(F|m), ..) -- this this the semantics of associative statistical models, as used in ML. Rather it means a gravitational force is caused by the interaction of two masses over a distance r. Here `F` refers to a force via a scientific model, etc. The formula is a short-hand conceuqnece of a family of explanatory models about mass, inertia, gravity, forces, etc. and is only valid when used in the context of those models. Eg., you cannot equate F = GMm/r^2 to F = kQq/r^2. Not least since we have no gravitational model which applies to tiny charged particles. The formula used in scientific modelling do not have either mathematical or statistical semantics (ie., they dont refer to numbers, nor to associations). They refer to bits of the world via explanations; and are only valid insofar as these explanations apply. Though science seems obsessed by formula, in terms of the goal of science, it's the least important part of the scientific model. Explanations are the goal, and highly circumstantial consequences of these are given formula partly for illustration, partly for engineering. There's rarely anything in any dataset whose model would even be useful for building an explanation. Explanations are build via counter-factuals, and these are resolved by experimental data -- they are not made by it. Indeed, in many cases, the experimental data would be no where nearly modelled lossessly by the candidate hypothesis.
  - dr_dshiv 3y ago
    Here’s a toy example based on a project I’d like to do. I love faraday waves and Chladni plates — roughly speaking, cymatics. Now, if I record high res video of the waves on water in a dish, vibrating at different frequencies, I could likely create a diffusion model that had some latent “understanding” of the relationship between the proportional frequencies of sound and the waves in a particular sized dish. I could test this by holding out certain frequencies and observing whether the diffusion model could recreate them. So, I can ask the question of whether the AI model learned wave physics. Here, there would be no formula, per se, and no explanation. Merely a computational model that could make predictions about physical phenomena. Now, what makes this less scientific than a formula and set of explanations for the experiments? Is it because I can relate the the explanations linguistically to everything else I know about science? So, in that case, If I jointly developed a language model to do the same thing as the diffusion model, ie make predictions based on data—but linguistically capable of connecting the outcomes to scientific concepts, then would then it be scientific?
    - mjburgess 3y ago
      The relationship between terms in scientific models and reality isn't linguistic. You cannot create explanatory models from associative models of pixels -- you can create associative models that may have some limited engineering use. (Because, by highly fragile engineered conditions, the pixels track unknown+unstated properties of the physical system). In the case of waves, the governing dynamical wave eq, ie., f(x, t) = potential(x, t) + kinetic(x, t) needs to have terms related to the physics of the system, eg., the properties of the material, to count as even a basic explanation. Broadly, you'd need to describe the dynamics of material properties and how they give rise to the dynamics of sound properties, which requires a family of explanatory models. You would produce those explanatory models by creating novel materials, novel experimental conditions, reasoning counter-factually, etc. over a long period of time. Eventually you may be able to formalise small, circumstantial, parts of those explanations and refute them by using logically entitled experimental data. The relevant relationships here are: explanation, causation, counter-factual possibility, necessity and logical entialment. No where is "association", nor should it ever be if it's science. An associative model of data isn't an explanation, it's a sort of pseudo-empiricism also present just in superstition. You can associate personality markers with positions of constellations with an associative model and get arbitrary predictive accuracy (since, eg., there are enough stars in the sky to choose ones which correlate with anything). This has nothing to do with science, and as even a claim to science, it's outright pseudoscience.

dr_dshiv 3y ago

How can you tell the difference between a causal model and a predictive model? Isn’t it just the elegance of the model? And isn’t elegance just succinctness? Because another approach could focus on human comprehensibility. Indeed, a leading theory of scientific progress focuses on the advancement of noetic understanding. But, then we should really be irritated by things like quantum mechanics and love things like nutrition (we might get poor predictions but good understanding). I’m not sure myself.

mjburgess 3y ago
No, it's a completely different semantics. F = GMm/r^2 does not mean anything about {P(F|M), P(F|m), ..) -- this this the semantics of associative statistical models, as used in ML. Rather it means a gravitational force is caused by the interaction of two masses over a distance r. Here `F` refers to a force via a scientific model, etc. The formula is a short-hand conceuqnece of a family of explanatory models about mass, inertia, gravity, forces, etc. and is only valid when used in the context of those models. Eg., you cannot equate F = GMm/r^2 to F = kQq/r^2. Not least since we have no gravitational model which applies to tiny charged particles. The formula used in scientific modelling do not have either mathematical or statistical semantics (ie., they dont refer to numbers, nor to associations). They refer to bits of the world via explanations; and are only valid insofar as these explanations apply. Though science seems obsessed by formula, in terms of the goal of science, it's the least important part of the scientific model. Explanations are the goal, and highly circumstantial consequences of these are given formula partly for illustration, partly for engineering. There's rarely anything in any dataset whose model would even be useful for building an explanation. Explanations are build via counter-factuals, and these are resolved by experimental data -- they are not made by it. Indeed, in many cases, the experimental data would be no where nearly modelled lossessly by the candidate hypothesis.
- dr_dshiv 3y ago
  Here’s a toy example based on a project I’d like to do. I love faraday waves and Chladni plates — roughly speaking, cymatics. Now, if I record high res video of the waves on water in a dish, vibrating at different frequencies, I could likely create a diffusion model that had some latent “understanding” of the relationship between the proportional frequencies of sound and the waves in a particular sized dish. I could test this by holding out certain frequencies and observing whether the diffusion model could recreate them. So, I can ask the question of whether the AI model learned wave physics. Here, there would be no formula, per se, and no explanation. Merely a computational model that could make predictions about physical phenomena. Now, what makes this less scientific than a formula and set of explanations for the experiments? Is it because I can relate the the explanations linguistically to everything else I know about science? So, in that case, If I jointly developed a language model to do the same thing as the diffusion model, ie make predictions based on data—but linguistically capable of connecting the outcomes to scientific concepts, then would then it be scientific?
  - mjburgess 3y ago
    The relationship between terms in scientific models and reality isn't linguistic. You cannot create explanatory models from associative models of pixels -- you can create associative models that may have some limited engineering use. (Because, by highly fragile engineered conditions, the pixels track unknown+unstated properties of the physical system). In the case of waves, the governing dynamical wave eq, ie., f(x, t) = potential(x, t) + kinetic(x, t) needs to have terms related to the physics of the system, eg., the properties of the material, to count as even a basic explanation. Broadly, you'd need to describe the dynamics of material properties and how they give rise to the dynamics of sound properties, which requires a family of explanatory models. You would produce those explanatory models by creating novel materials, novel experimental conditions, reasoning counter-factually, etc. over a long period of time. Eventually you may be able to formalise small, circumstantial, parts of those explanations and refute them by using logically entitled experimental data. The relevant relationships here are: explanation, causation, counter-factual possibility, necessity and logical entialment. No where is "association", nor should it ever be if it's science. An associative model of data isn't an explanation, it's a sort of pseudo-empiricism also present just in superstition. You can associate personality markers with positions of constellations with an associative model and get arbitrary predictive accuracy (since, eg., there are enough stars in the sky to choose ones which correlate with anything). This has nothing to do with science, and as even a claim to science, it's outright pseudoscience.