5 ms·

“As another example, consider the Newtonian model of gravitational attraction, which says that the force between two objects of mass m1 and m2 a distance r apar

by aj7 4y ago

“As another example, consider the Newtonian model of gravitational attraction, which says that the force between two objects of mass m1 and m2 a distance r apart is given by F = Gm1m2/r^2 where G is the universal gravitational constant. This is a trained model because the gravitational constant G is determined by statistical inference over the results of a series of experiments that contain stochastic experimental error.” No, it’s not. Period. It’s a THEORY that is supported “by statistical inference over the results of a series of experiments that contain stochastic experimental error.” It’s been checked hundreds of times and its accuracy has gotten better and better. The technology of the checking led to new science. By making it consistent with special relativity, and POSTULATING the principle of equivalence, a new THEORY was born, general relativity, also known as the theory of gravitation. IT has been checked hundreds of times, ”the technology of the checking has led to new science,” and it is a current very active field of RESEARCH, (not model training.) It’s a little horrifying that Norvig doesn’t seem to understand these nuances. The same arguments apply to the solid state physics underlying the machines that run large language models. That too, is a “THEORY. It has been checked hundreds of times, the technology of the checking has led to new science, and it is a current very active field of RESEARCH, (not model training.)”

ckrapu 4y ago
I disagree. It’s true that we are used to thinking of gravitation as a theory that was double checked with data, but it is also pretty clear that good physical arguments are strong priors. Then, with experimental data, we have a classic prior + data posterior update which does cast all of this as a statistical model.
- pfortuny 4y ago
  That is not at all the way Newton thought, though.
- aj7 4y ago
  Then all scientific theories are “models.” And we are just very advanced “computers.” Nope. The first sentence is wrong. And the second sentence is why we play the game.
  - xapata 4y ago
    Is the use of those quoted words what you dislike? Would you have a different sense of it if we replaced those words with "foo" and "bar" so that we could use more nuanced concepts?
  - pdonis 4y ago
    > all scientific theories are “models.” That's correct. They are. As the saying goes, all models are wrong but some are useful. > And we are just very advanced “computers.” To a physicalist, yes, this is true.
  - dekhn 4y ago
    I think you're coming at this from a dogmatic interpretation and it's making it hard for you to see the alternate perspective. All scientific theories are models; that's generally considered true by nearly all scientists. What would a theory be if it's not a model?
  - bmitc 4y ago
    > Then all scientific theories are “models.” I may be missing the context of what you mean by model and somewhat agree with your original critique of Norvig's comment. However, scientific theories are nothing but models. For example, general relativity is not gravity but is instead a model of gravity. Quantum mechanics describes quantum behavior but it is not quantum behavior. It is a model of quantum behavior. When physicists say "is", they mean "is modeled by" or "is a model of", whether they think so or not. None of the mathematics, or physics, or what have you is the actual thing. They are nothing but models of the actual thing that have varying degrees of accuracy and precision.
college_physics 4y ago
> It’s a little horrifying that Norvig doesn’t seem to understand these nuances. yes. its quite incredible actually. It is one thing to sort of tribulate about brain / cognition domains that nobody knows much about using a pseudo-language of "learning" or "intelligence" (or even the original sin: "neural networks") and it is another level altogether to expose publicly how much one is subject to "I have got a nice little hammer and to me everything is a nail" pathologies Conflating the statistical verification of physical laws / theories with the theories themselves reveals in much clearer terms how limited the conceptual model the AI crowd want to inflict on us. We do know about more about physical laws than about the brain and they don't come about like they say. Go back to school.
- adw 4y ago
  Newtonian gravitation is absolutely “we propose a model, we fit it, here is the result, here are the error bars on _g_”. Einsteinian gravitation is a better model. It fits the data better, that’s how we know. These are models, exactly in line with Breiman’s “first culture”, data modelling - the model has been designed according to intuition about the underlying mechanism. You seem to be arguing that if a model is not derived from first principles it is not science. This is ahistorical nonsense. Science has always been full of phenomenological models which haven’t lent themselves to direct interpretation. That doesn’t make the model wrong, it makes it uninterpretable. Some of them become interpretable later, some are replaced by interpretable models, often they give key insight to underlying relationships which must hold which then guide the search for a mechanism, and nearly always they’re profoundly useful. Consider Lev Landau’s order parameter model. “Physical law” is just a synonym for “extremely high-confidence interpretable model”.
  - college_physics 4y ago
    > “Physical law” is just a synonym for “extremely high-confidence interpretable model”. You are abusing the term "model" to the point of ridicule. What is the "model" in Newtonian gravity? An iconic relation to which you might (over)fit some data "AI style"? The associated concepts (action/reaction, momentum) that float somewhere around the "formula" and are absolutely necessary to understand and use it but are not reflected in the slightest in this quantitative expression? The drastically alternative ways you can re-express the formula, which neverthless all contain exactly the same physical content? The explanatory (modeling) potential of the concepts (what happens when I split a mass in two?) that suggests a more profound link to what is actually being described? At a very basic level it is true that we verify the conceptual constructs that we call physical laws by deriving certain statistically testable relations. But to argue that these derivations is all there is to it is simply... wrong.
hackinthebochs 4y ago
You're just using different words to say the same thing. The word "theory" doesn't have the significance you are attributing to it.
pdonis 4y ago
> No, it’s not. What's "not"? His statement about the gravitational constant is exactly correct: we have no theory that tells us what the value of this constant should be, we have to get its value from measurements, and doing that is a process of statistical inference. It is also true that the form of the theory itself--its equations--is not given to us magically but has to be developed by comparing predictions with experimental data, i.e., by a process of statistical inference. Physicists don't usually describe these processes as "model training", but that doesn't mean such a description is wrong.
- asimpletune 4y ago
  I guess one difference between the two is explainability. As we predict, test, and update, there is an ongoing narrative along the way of why we are making this change or the other. We can explain why we thought some scientific model was the correct one, even if it turns out to be wrong and needs to be udpated. In model training as it's used here, I don't think there is any such a thing.
  - adw 4y ago
    That's really the divergence Norvig points to between Bregman's two cultures. And there are plenty of cases where we have had essentially purely phenomenological models in physics (I mentioned this in another thread, but Landau theory) which only _later_ were systematized, so it's a set of processes on a continuum, not two binary opposites. People _treat_ physical laws as absolutely true, because with high confidence they are and it's intellectually convenient, but they're really just models. We build models on those models, and mental models using our _interpretation_ of those models; no-one is denying the importance of model interpretability in all of this – but it's absolutely a tower of model-building, with more-or-less convincing explanations for the model parameters at various points. Physics, or at least some domains of physics, are tractable using what one might term the "axiomatic style" because the models we have are a) staggeringly explainable and b) extremely robustly supported by measurement. Even that statement exists on a continuum: MOND is pretty damn phenomenological, for one example, and while we have pretty good quantum chemistry methods in the small, we certainly don't have practical ways of using those ab-initio methods for even mesoscale problems in solid state physics. Does that make mesoscale physics "not science"? A science which rejects all phenomenology and which insists on building everything up from first principles isn't a science; it's mathematics. That's totally fine! It's just a different—related, but distinct—domain of study.
usgroup 4y ago
Yeah, I feel similarly. It comes across as if Norvig just doesn’t understand the perspective. He’s translating it to his terms and making strawmen by doing so.
ramrunner0xff 4y ago
I came to argue in favor of aj7's argument as a physicist. I think we are all missing the forest for the trees here. what elevates the gravitational law to it's status and differentiates it from a simple model, and what i think the point that aj7 is trying to make, and also the point that norvig is choosing not to focus on here, is not the gravitational constant G. Noone really cares about the value of the constant and how it is approximated in this part of our spacetime etc [edit: of course they do care if it is postive, zero, or negative for reasons consistent to what i am saying later]. The "juice" here is the inverse square relation to length. The form of the equation is what is the deep insight and what also provides for that insight to transfer to other systems. Knowing the nature of the differential equation is of uttermost importance because then we can reason about possible and impossible outcomes of it just from it's mathematical (and only) nature. If these models in the future can be built in ways that they can be reasoned about symbolically like these systems that we strive for in physics, i think they will be elevated to laws (or proper theories as aj7 says) rather than models. and noone will care if they have a billion parameters as long as they have a certain structure that we can work with. I hope this makes some sense to support the counter-norvig argument in this (and i personally respect both chomsky and norvig infinitely of course. i just think norvig's wording in this specific part of the article is indeed crude and shallow, maybe just to make his counterargument)

by aj7 4y ago

ckrapu 4y ago

I disagree. It’s true that we are used to thinking of gravitation as a theory that was double checked with data, but it is also pretty clear that good physical arguments are strong priors. Then, with experimental data, we have a classic prior + data posterior update which does cast all of this as a statistical model.

pfortuny 4y ago
That is not at all the way Newton thought, though.
aj7 4y ago
Then all scientific theories are “models.” And we are just very advanced “computers.” Nope. The first sentence is wrong. And the second sentence is why we play the game.
- xapata 4y ago
  Is the use of those quoted words what you dislike? Would you have a different sense of it if we replaced those words with "foo" and "bar" so that we could use more nuanced concepts?
- pdonis 4y ago
  > all scientific theories are “models.” That's correct. They are. As the saying goes, all models are wrong but some are useful. > And we are just very advanced “computers.” To a physicalist, yes, this is true.
- dekhn 4y ago
  I think you're coming at this from a dogmatic interpretation and it's making it hard for you to see the alternate perspective. All scientific theories are models; that's generally considered true by nearly all scientists. What would a theory be if it's not a model?
- bmitc 4y ago
  > Then all scientific theories are “models.” I may be missing the context of what you mean by model and somewhat agree with your original critique of Norvig's comment. However, scientific theories are nothing but models. For example, general relativity is not gravity but is instead a model of gravity. Quantum mechanics describes quantum behavior but it is not quantum behavior. It is a model of quantum behavior. When physicists say "is", they mean "is modeled by" or "is a model of", whether they think so or not. None of the mathematics, or physics, or what have you is the actual thing. They are nothing but models of the actual thing that have varying degrees of accuracy and precision.

college_physics 4y ago

> It’s a little horrifying that Norvig doesn’t seem to understand these nuances. yes. its quite incredible actually. It is one thing to sort of tribulate about brain / cognition domains that nobody knows much about using a pseudo-language of "learning" or "intelligence" (or even the original sin: "neural networks") and it is another level altogether to expose publicly how much one is subject to "I have got a nice little hammer and to me everything is a nail" pathologies Conflating the statistical verification of physical laws / theories with the theories themselves reveals in much clearer terms how limited the conceptual model the AI crowd want to inflict on us. We do know about more about physical laws than about the brain and they don't come about like they say. Go back to school.

adw 4y ago
Newtonian gravitation is absolutely “we propose a model, we fit it, here is the result, here are the error bars on _g_”. Einsteinian gravitation is a better model. It fits the data better, that’s how we know. These are models, exactly in line with Breiman’s “first culture”, data modelling - the model has been designed according to intuition about the underlying mechanism. You seem to be arguing that if a model is not derived from first principles it is not science. This is ahistorical nonsense. Science has always been full of phenomenological models which haven’t lent themselves to direct interpretation. That doesn’t make the model wrong, it makes it uninterpretable. Some of them become interpretable later, some are replaced by interpretable models, often they give key insight to underlying relationships which must hold which then guide the search for a mechanism, and nearly always they’re profoundly useful. Consider Lev Landau’s order parameter model. “Physical law” is just a synonym for “extremely high-confidence interpretable model”.
- college_physics 4y ago
  > “Physical law” is just a synonym for “extremely high-confidence interpretable model”. You are abusing the term "model" to the point of ridicule. What is the "model" in Newtonian gravity? An iconic relation to which you might (over)fit some data "AI style"? The associated concepts (action/reaction, momentum) that float somewhere around the "formula" and are absolutely necessary to understand and use it but are not reflected in the slightest in this quantitative expression? The drastically alternative ways you can re-express the formula, which neverthless all contain exactly the same physical content? The explanatory (modeling) potential of the concepts (what happens when I split a mass in two?) that suggests a more profound link to what is actually being described? At a very basic level it is true that we verify the conceptual constructs that we call physical laws by deriving certain statistically testable relations. But to argue that these derivations is all there is to it is simply... wrong.

hackinthebochs 4y ago

You're just using different words to say the same thing. The word "theory" doesn't have the significance you are attributing to it.

pdonis 4y ago

> No, it’s not. What's "not"? His statement about the gravitational constant is exactly correct: we have no theory that tells us what the value of this constant should be, we have to get its value from measurements, and doing that is a process of statistical inference. It is also true that the form of the theory itself--its equations--is not given to us magically but has to be developed by comparing predictions with experimental data, i.e., by a process of statistical inference. Physicists don't usually describe these processes as "model training", but that doesn't mean such a description is wrong.

asimpletune 4y ago
I guess one difference between the two is explainability. As we predict, test, and update, there is an ongoing narrative along the way of why we are making this change or the other. We can explain why we thought some scientific model was the correct one, even if it turns out to be wrong and needs to be udpated. In model training as it's used here, I don't think there is any such a thing.
- adw 4y ago
  That's really the divergence Norvig points to between Bregman's two cultures. And there are plenty of cases where we have had essentially purely phenomenological models in physics (I mentioned this in another thread, but Landau theory) which only _later_ were systematized, so it's a set of processes on a continuum, not two binary opposites. People _treat_ physical laws as absolutely true, because with high confidence they are and it's intellectually convenient, but they're really just models. We build models on those models, and mental models using our _interpretation_ of those models; no-one is denying the importance of model interpretability in all of this – but it's absolutely a tower of model-building, with more-or-less convincing explanations for the model parameters at various points. Physics, or at least some domains of physics, are tractable using what one might term the "axiomatic style" because the models we have are a) staggeringly explainable and b) extremely robustly supported by measurement. Even that statement exists on a continuum: MOND is pretty damn phenomenological, for one example, and while we have pretty good quantum chemistry methods in the small, we certainly don't have practical ways of using those ab-initio methods for even mesoscale problems in solid state physics. Does that make mesoscale physics "not science"? A science which rejects all phenomenology and which insists on building everything up from first principles isn't a science; it's mathematics. That's totally fine! It's just a different—related, but distinct—domain of study.

usgroup 4y ago

Yeah, I feel similarly. It comes across as if Norvig just doesn’t understand the perspective. He’s translating it to his terms and making strawmen by doing so.

ramrunner0xff 4y ago

I came to argue in favor of aj7's argument as a physicist. I think we are all missing the forest for the trees here. what elevates the gravitational law to it's status and differentiates it from a simple model, and what i think the point that aj7 is trying to make, and also the point that norvig is choosing not to focus on here, is not the gravitational constant G. Noone really cares about the value of the constant and how it is approximated in this part of our spacetime etc [edit: of course they do care if it is postive, zero, or negative for reasons consistent to what i am saying later]. The "juice" here is the inverse square relation to length. The form of the equation is what is the deep insight and what also provides for that insight to transfer to other systems. Knowing the nature of the differential equation is of uttermost importance because then we can reason about possible and impossible outcomes of it just from it's mathematical (and only) nature. If these models in the future can be built in ways that they can be reasoned about symbolically like these systems that we strive for in physics, i think they will be elevated to laws (or proper theories as aj7 says) rather than models. and noone will care if they have a billion parameters as long as they have a certain structure that we can work with. I hope this makes some sense to support the counter-norvig argument in this (and i personally respect both chomsky and norvig infinitely of course. i just think norvig's wording in this specific part of the article is indeed crude and shallow, maybe just to make his counterargument)