4 ms·

Discovering underlying principles and predicting outcomes is two sides of the same coin in that there is no way to confirm you have discovered underlying princi

by ak_111 2y ago

Discovering underlying principles and predicting outcomes is two sides of the same coin in that there is no way to confirm you have discovered underlying principles unless they have some predictive power. Some had tried to come up with other criteria to confirm you have discovered an underlying principle without predictive power, such as on aesthetics - but this is seen by the majority of scientists as basically a cop out. See debate around string theory. Note that this comment is summarizing a massive debate in the philosophy of science.

thfuran 2y ago
>there is no way to confirm you have discovered underlying principles unless they have some predictive power. Yes, but a perfect oracle has no explanatory power, only predictive.
- nkingsy 2y ago
  increasing the volume of predictions produces patterns that often lead to underlying principles.
  - mikeyouse 2y ago
    And much of the 20th century was characterized by a very similar progression - we had no clue what the actual mechanism of action was for hundreds of life saving drugs until relatively recently, and we still only have best guesses for many. That doesn’t diminish the value that patients received in any way even though it would be more satisfying to make predictions and design something to interact in a way that exactly matches your theory.
    - qp11 2y ago
      We were using the compass for navigation for thousands of years, without any clue about what it was doing or why. Ofcourse lot of people got lost cause compasses are not perfect. And the same will happen here. Theory of Bounded Rationality applies.
chasd00 2y ago
If all you can do is predict an outcome without being able to explain how then what have you really discovered? Asking someone to just believe you can predict outcomes without any reasoning as to how, even if you're always right, sounds like the concept of faith in religion.
- pas 2y ago
  it's still an extremely valuable tool. just as we see in mathematics, closed forms (and short and elegant proofs) are much coveted luxury items. for many basic/fundamental mathematical objects we don't (yet) have simple mechanistic ways to compute them. so if a probabilistic model spits out something very useful, we can slap a nice label on it and call it a day. that's how engineering works anyway. and then hopefully someday someone will be able to derive that result from "first principles" .. maybe it'll be even more funky/crazy/interesting ... just like mathematics arguably became more exciting by the fact that someone noticed that many things are not provable/constructable without an explicit Axiom of Choice. https://en.wikipedia.org/wiki/Nonelementary_integral#Examples https://en.wikipedia.org/wiki/Nonelementary_integral#Example...
  - thfuran 2y ago
    >closed forms (and short and elegant proofs) are much coveted luxury items. Yes, but we're taking about roughly the opposite of a proof
    - pas 2y ago
      but in usual natural sciences we don't have proofs, only data and models, and then we do model selection (and through careful experiments we end up with confidence intervals) and it seems with these molecular biology problems we constantly have the problem of specificity (model prediction quality) vs sensitivity (model applicability), right? but due to information theory constraints there's also a dimension along model size/complexity. so if a ML model can push the ROC curve toward the magic left-up corner then likely it's getting more and more complex. and at one point we simply are left with models that are completely parametrized by data and there's virtually zero (direct) influence of the first principles. (I mean that at one point as we get more data even to do model selection we can't use "first principles" because what we know through that is already incorporated into previous versions of the models. Ie. the information we gained from those principles we already used to make decisions in earlier iterations.) Of course then in theory we can do model distillation, and if there's some hidden small/elegant theory we can probably find it. (Which would be like a proof through contradiction, because it would mean that we found model with the same predictive power but with smaller complexity than expected.) // NB: it's 01:30 here, but independent of ignorance-o-clock ... it's quite possible I'm totally wrong about this, happy to read any criticism/replies

by ak_111 2y ago

thfuran 2y ago

>there is no way to confirm you have discovered underlying principles unless they have some predictive power. Yes, but a perfect oracle has no explanatory power, only predictive.

nkingsy 2y ago
increasing the volume of predictions produces patterns that often lead to underlying principles.
- mikeyouse 2y ago
  And much of the 20th century was characterized by a very similar progression - we had no clue what the actual mechanism of action was for hundreds of life saving drugs until relatively recently, and we still only have best guesses for many. That doesn’t diminish the value that patients received in any way even though it would be more satisfying to make predictions and design something to interact in a way that exactly matches your theory.
  - qp11 2y ago
    We were using the compass for navigation for thousands of years, without any clue about what it was doing or why. Ofcourse lot of people got lost cause compasses are not perfect. And the same will happen here. Theory of Bounded Rationality applies.

chasd00 2y ago

If all you can do is predict an outcome without being able to explain how then what have you really discovered? Asking someone to just believe you can predict outcomes without any reasoning as to how, even if you're always right, sounds like the concept of faith in religion.

pas 2y ago
it's still an extremely valuable tool. just as we see in mathematics, closed forms (and short and elegant proofs) are much coveted luxury items. for many basic/fundamental mathematical objects we don't (yet) have simple mechanistic ways to compute them. so if a probabilistic model spits out something very useful, we can slap a nice label on it and call it a day. that's how engineering works anyway. and then hopefully someday someone will be able to derive that result from "first principles" .. maybe it'll be even more funky/crazy/interesting ... just like mathematics arguably became more exciting by the fact that someone noticed that many things are not provable/constructable without an explicit Axiom of Choice. https://en.wikipedia.org/wiki/Nonelementary_integral#Examples https://en.wikipedia.org/wiki/Nonelementary_integral#Example...
- thfuran 2y ago
  >closed forms (and short and elegant proofs) are much coveted luxury items. Yes, but we're taking about roughly the opposite of a proof
  - pas 2y ago
    but in usual natural sciences we don't have proofs, only data and models, and then we do model selection (and through careful experiments we end up with confidence intervals) and it seems with these molecular biology problems we constantly have the problem of specificity (model prediction quality) vs sensitivity (model applicability), right? but due to information theory constraints there's also a dimension along model size/complexity. so if a ML model can push the ROC curve toward the magic left-up corner then likely it's getting more and more complex. and at one point we simply are left with models that are completely parametrized by data and there's virtually zero (direct) influence of the first principles. (I mean that at one point as we get more data even to do model selection we can't use "first principles" because what we know through that is already incorporated into previous versions of the models. Ie. the information we gained from those principles we already used to make decisions in earlier iterations.) Of course then in theory we can do model distillation, and if there's some hidden small/elegant theory we can probably find it. (Which would be like a proof through contradiction, because it would mean that we found model with the same predictive power but with smaller complexity than expected.) // NB: it's 01:30 here, but independent of ignorance-o-clock ... it's quite possible I'm totally wrong about this, happy to read any criticism/replies