4 ms·

> The advantage of physical systems is that it's straightforward to collect data from natural systems. For a lot of physical systems it might be "straightforwa

> The advantage of physical systems is that it's straightforward to collect data from natural systems. For a lot of physical systems it might be "straightforward" but expensive. Testing new building designs for energy efficiency is just a matter of building a new test building, why not just build 1000? Testing new car designs to see if they reach the safety qualifications is just a matter of crashing a few of your prototypes, why not crash 1000? It's just a matter of getting more telescope time. It's just a matter of getting more robots to try more materials or proteins. Yes you can do theoretically do it, it's just cost prohibitive to get Google-level datasets for many scientific and engineering problems. Of course that's not true for all domains (high-throughput sequencing for bioinformatics is the prime example of something that became cheaply data-rich for standard machine learning), but there are many domains where getting another data point is always possible but just would cost another million.

dekhn 4y ago
yes, but it's still unclear that for data-poor physics problems (the joke in grad school is that you could graduate when you collected your second data point, since that was enough to draw a straight line, and we all know any interesting physical phenomenon is linear in the engineering regime, right?) that NNs really provide anything more than a bayesian prior that physics folks already knew how to incorporate. High throughput sequeuncing didn't solve any problems for bioinformatics- in fact, the problem is that we have so much data we don't know how to process it to extract the actual value from the data.
- uoaei 4y ago
  The general principle can be referred to as "inductive bias": out of the space of all functions that can be represented by computational means, how does one restrict the search space to only those functions which are relevant for the data/system of study? Developing physical theories is nothing more than the process of whittling down what space of functions can represent the data most accurately, using analytical and scientific techniques to get there. Bioinformaticists, with sufficiently accurate and detailed models of how genes translate to $BEHAVIOR_OR_OUTCOME, would need far fewer data to extract meaningful insights. So of course throughput hasn't solved the problem. That wasn't the point of the comment above.
  - dekhn 4y ago
    I don't think there's any deep relationship between the functions we find that fit observed data well, and those functions being generally useful at making out of class predictions. Really, I was making a comment about https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness_of_Mathematics_in_the_Natural_Sciences https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness... which in my mind is still unresolved (that is: why do the math models we use for physical systems generalize so well, if we wouldn't expect them to do so?)

dekhn 4y ago

yes, but it's still unclear that for data-poor physics problems (the joke in grad school is that you could graduate when you collected your second data point, since that was enough to draw a straight line, and we all know any interesting physical phenomenon is linear in the engineering regime, right?) that NNs really provide anything more than a bayesian prior that physics folks already knew how to incorporate. High throughput sequeuncing didn't solve any problems for bioinformatics- in fact, the problem is that we have so much data we don't know how to process it to extract the actual value from the data.

uoaei 4y ago
The general principle can be referred to as "inductive bias": out of the space of all functions that can be represented by computational means, how does one restrict the search space to only those functions which are relevant for the data/system of study? Developing physical theories is nothing more than the process of whittling down what space of functions can represent the data most accurately, using analytical and scientific techniques to get there. Bioinformaticists, with sufficiently accurate and detailed models of how genes translate to $BEHAVIOR_OR_OUTCOME, would need far fewer data to extract meaningful insights. So of course throughput hasn't solved the problem. That wasn't the point of the comment above.
- dekhn 4y ago
  I don't think there's any deep relationship between the functions we find that fit observed data well, and those functions being generally useful at making out of class predictions. Really, I was making a comment about https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness_of_Mathematics_in_the_Natural_Sciences https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness... which in my mind is still unresolved (that is: why do the math models we use for physical systems generalize so well, if we wouldn't expect them to do so?)