3 ms·

Does anyone? What % of people make any connection at all between the fields? Firstly this is not a CS or Physics only forum so I would guess the subject could

Does anyone? What % of people make any connection at all between the fields? Firstly this is not a CS or Physics only forum so I would guess the subject could easily have never come up for many (if so for the simpler/non-ML connections see https://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_information_theory https://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_...). Secondly I think general awareness beyond formal education has increased in a non-linear way only over the last 20 years only because there has been so much science press around the celebrity of Steven Hawking. His celebrity alone raises awareness, however it was compounded by drama depicted between himself and others and some really interesting observations along the way (https://en.wikipedia.org/wiki/Holographic_principle https://en.wikipedia.org/wiki/Holographic_principle). When all of this publicity has various inevitable Kevin Bacon trails back to the fundamental connections, I’d say the population is not only not everyone but could be much less if things unfolded a little differently.

mlthoughts2018 8y ago
I mean it more from a Joel Spolsky sort of too much abstraction point of view, e.g. from [0]: > “When great thinkers think about problems, they start to see patterns. They look at the problem of people sending each other word-processor files, and then they look at the problem of people sending each other spreadsheets, and they realize that there’s a general pattern: sending files. That’s one level of abstraction already. Then they go up one more level: people send files, but web browsers also “send” requests for web pages. Those are both sending operations, so our clever thinker invents a new, higher, broader abstraction called messaging, but now it’s getting really vague and nobody really knows what they’re talking about any more. And if you go too far up, abstraction-wise, you run out of oxygen. Sometimes smart thinkers just don’t know when to stop, and they create these absurd, all-encompassing, high-level pictures of the universe that are all good and fine, but don’t actually mean anything at all.” [0]: < https://www.joelonsoftware.com/2000/07/22/microsoft-goes-bonkers/ https://www.joelonsoftware.com/2000/07/22/microsoft-goes-bon... >
- WhitneyLand 8y ago
  I can relate to your point more in that way. Joel has a gift for exlplaining intangibles. Agree with his point also having had to spend time debating whether an abstraction choice adds value. Still worth noting I think though that he does say “sometimes”, because no one is going anywhere without abstractions or generalizations. The problem is they’re hard to do really well and we know historically really smart people can make small mistakes and fall into the not cogent/no value/Joel trap #234.
- elcritch 8y ago
  That seems to be a poor analogy to relate in this case. I’m not going to claim that the parent article is factual or even useful but at first glance it seems interesting. In general the difference between a tinkerer and an engineer or scientist is that the latter uses appropriate mathematical models to garner deeper insight into problems than available to general tinkerers and "smart thinkers". If the linked article succeeds in linking any of the concepts of thermodynamics — which is one of the core building blocks of modern physics and relevant to almost every field of practical engineering — to DL models then that’d be huge boon. There are a lot of well honed techniques and tools available in the thermodynamics and statistical physics toolboxes which could vastly improve the ability of DL to be tuned for useful tasks in a scientific manner. Most application of DL appears to be "dark arts" or alchemy phase where only intuition is used to direct what problems can and should be tackeled using DL. It seems every other paper published in DL is just happenstance. Joel Spolsky‘s comment really fits more with smart "tinkerers" but not serious attempts at modeling of the energy minimization aspects of restricted Boltzmann machines (ahem DL) which really do borrow a lot from thermodynamics [0]. I’m glad "smart thinkers" like Geoffrey Hintonmidday know when to stop [1], and kept generalizing and tweaking Boltzmann machines. Heck even the idea of applying energy minimization techniques from the phase changes of spin glasses is one of those applications of abstract high level models that really don’t mean anything. Except when evenetually they provide entirely new insights and, well fields of study. Of course there’s a difference between serious attempts and hard work of establishing mathematical and conceptual correspondents between fields and just quackery or pseudo-sophistication. Given the amount of formula development in the linked paper I’ll take it as a good faith attempt to seriously link fields. 0: https://en.m.wikipedia.org/wiki/Boltzmann_machine#History https://en.m.wikipedia.org/wiki/Boltzmann_machine#History —- especially the first paper on spin glasses 1: https://medium.com/@andreykurenkov/a-brief-history-of-neural-nets-and-deep-learning-part-4-61be90639182 https://medium.com/@andreykurenkov/a-brief-history-of-neural...

mlthoughts2018 8y ago

I mean it more from a Joel Spolsky sort of too much abstraction point of view, e.g. from [0]: > “When great thinkers think about problems, they start to see patterns. They look at the problem of people sending each other word-processor files, and then they look at the problem of people sending each other spreadsheets, and they realize that there’s a general pattern: sending files. That’s one level of abstraction already. Then they go up one more level: people send files, but web browsers also “send” requests for web pages. Those are both sending operations, so our clever thinker invents a new, higher, broader abstraction called messaging, but now it’s getting really vague and nobody really knows what they’re talking about any more. And if you go too far up, abstraction-wise, you run out of oxygen. Sometimes smart thinkers just don’t know when to stop, and they create these absurd, all-encompassing, high-level pictures of the universe that are all good and fine, but don’t actually mean anything at all.” [0]: < https://www.joelonsoftware.com/2000/07/22/microsoft-goes-bonkers/ https://www.joelonsoftware.com/2000/07/22/microsoft-goes-bon... >

WhitneyLand 8y ago
I can relate to your point more in that way. Joel has a gift for exlplaining intangibles. Agree with his point also having had to spend time debating whether an abstraction choice adds value. Still worth noting I think though that he does say “sometimes”, because no one is going anywhere without abstractions or generalizations. The problem is they’re hard to do really well and we know historically really smart people can make small mistakes and fall into the not cogent/no value/Joel trap #234.
elcritch 8y ago
That seems to be a poor analogy to relate in this case. I’m not going to claim that the parent article is factual or even useful but at first glance it seems interesting. In general the difference between a tinkerer and an engineer or scientist is that the latter uses appropriate mathematical models to garner deeper insight into problems than available to general tinkerers and "smart thinkers". If the linked article succeeds in linking any of the concepts of thermodynamics — which is one of the core building blocks of modern physics and relevant to almost every field of practical engineering — to DL models then that’d be huge boon. There are a lot of well honed techniques and tools available in the thermodynamics and statistical physics toolboxes which could vastly improve the ability of DL to be tuned for useful tasks in a scientific manner. Most application of DL appears to be "dark arts" or alchemy phase where only intuition is used to direct what problems can and should be tackeled using DL. It seems every other paper published in DL is just happenstance. Joel Spolsky‘s comment really fits more with smart "tinkerers" but not serious attempts at modeling of the energy minimization aspects of restricted Boltzmann machines (ahem DL) which really do borrow a lot from thermodynamics [0]. I’m glad "smart thinkers" like Geoffrey Hintonmidday know when to stop [1], and kept generalizing and tweaking Boltzmann machines. Heck even the idea of applying energy minimization techniques from the phase changes of spin glasses is one of those applications of abstract high level models that really don’t mean anything. Except when evenetually they provide entirely new insights and, well fields of study. Of course there’s a difference between serious attempts and hard work of establishing mathematical and conceptual correspondents between fields and just quackery or pseudo-sophistication. Given the amount of formula development in the linked paper I’ll take it as a good faith attempt to seriously link fields. 0: https://en.m.wikipedia.org/wiki/Boltzmann_machine#History https://en.m.wikipedia.org/wiki/Boltzmann_machine#History —- especially the first paper on spin glasses 1: https://medium.com/@andreykurenkov/a-brief-history-of-neural-nets-and-deep-learning-part-4-61be90639182 https://medium.com/@andreykurenkov/a-brief-history-of-neural...