3 ms·

Yes, the foundations for ML are somewhat old, but the explosion of progress has put everyone on the edge of a fresh domain for which we are only now developing

Yes, the foundations for ML are somewhat old, but the explosion of progress has put everyone on the edge of a fresh domain for which we are only now developing a theoretical framework. At the very least you must concede that applied machine learning is effectively a new field. Yes I'm sure a handful of neuron-like components have probably been assembled here or there in the past to do something interesting, but no one was going to college trying to make it big with a career in ML.

YeGoblynQueenne 6y ago
>> Yes I'm sure a handful of neuron-like components have probably been assembled here or there in the past to do something interesting, but no one was going to college trying to make it big with a career in ML. I'm at a looss as to how to respond to your comment. You seem to really believe that before the big success of CNNs in 2012 there were no neural netwroks to speak of. That could not be farther from the truth. Wikipedia has a decent introduction to the history of neural networks: https://en.wikipedia.org/wiki/History_of_artificial_neural_networks https://en.wikipedia.org/wiki/History_of_artificial_neural_n... I suggest you start from there, then follow the links etc.
- mtgp1000 6y ago
  You're underappreciating how much more we are doing with neural networks now. Image recognition, general purpose contextual searching, NLP, accelerated 3D modeling, abstract problem solving (applied derivatives of alphago and other RL), and we're on the cusp of image synthesis, music synthesis, news summarization, not to mention translation... All of this powered by dozens of exotic architectures and hundreds of discovered tweaks and optimizations - we are finally digging into the iceberg of which research before the 10s only scratch the surface. That's not to discount the visionary work of the past! But if you plotted some general measure of progress in this field, you would see an enormous discontinuity in the derivative starting sometime in the 10s with GPUs and the open source/open science initiatives. For all the shit I give to Google and Facebook and FAANG in general, they absolutely brought immense good to society by democratizing practical general function approximators. If there aren't any major roadblocks ML will touch every aspect of our near future lives, much like the internet. There's just too much potential - the cutting edge is just hitting the industry, our tech is already proven and it goes far deeper than classifying cats and dogs or digits. This is real science and it's huge, and it's a fact that even the money has never been remotely there for the kind of progress we are making; not to even mention the drastic difference between running ML on 1980s hardware vs a modern GPU. The kind of science we can now do from the comfort of our homes was impossible until very recently. It's like coming out of the bronze age and starting to work with steel. We build the tools as we go along. This is part of the process and we're actually doing a great job.
  - YeGoblynQueenne 6y ago
    >> You're underappreciating how much more we are doing with neural networks now. From your comment above, that "a handful of neuron-like components have probably been assembled here or there" I understand that you do not have any background in AI or machine learning etc. I am curious then, where all this enthusiasm in the form of "we are doing X" statements comes from. In particular I'm interested in your use of "we". Clearly, you're not doing any of that stuff, so where does the "we" come in? Is it really prudent to express such strong views, without good understanding or personal experience of the subject matter? Are you adding anything to the conversation, by asserting all those things with such impetuousness, other than noise? I imagine that your source for all this information are articles you've read in the lay press. Unfortunately, such articles can't very well represent the real state of research in deep learning. The truth is that there has been an enormous increase in the numbers of work on deep learning being published every month- there's probably thousands of articles written in that period and uploaded on arxiv or even submitted to reputable venues- and even researches in the field have trouble keeping up. What is abundantly clear however is that the vast majority of this work is very poor quality and even the published work is not much better. It's clear also that the vast majority of this work has no lasting impact and is superseded within weeks anyway. The truth is that deep learning research is in a deep crisis and despite appearances and breathless announcements by large companies, progress has stalled and no new things are really being done. Many of the luminaries of the field, including Geoff Hinton and Yoshua Bengio, have said this in various ways.

YeGoblynQueenne 6y ago

>> Yes I'm sure a handful of neuron-like components have probably been assembled here or there in the past to do something interesting, but no one was going to college trying to make it big with a career in ML. I'm at a looss as to how to respond to your comment. You seem to really believe that before the big success of CNNs in 2012 there were no neural netwroks to speak of. That could not be farther from the truth. Wikipedia has a decent introduction to the history of neural networks: https://en.wikipedia.org/wiki/History_of_artificial_neural_networks https://en.wikipedia.org/wiki/History_of_artificial_neural_n... I suggest you start from there, then follow the links etc.

mtgp1000 6y ago
You're underappreciating how much more we are doing with neural networks now. Image recognition, general purpose contextual searching, NLP, accelerated 3D modeling, abstract problem solving (applied derivatives of alphago and other RL), and we're on the cusp of image synthesis, music synthesis, news summarization, not to mention translation... All of this powered by dozens of exotic architectures and hundreds of discovered tweaks and optimizations - we are finally digging into the iceberg of which research before the 10s only scratch the surface. That's not to discount the visionary work of the past! But if you plotted some general measure of progress in this field, you would see an enormous discontinuity in the derivative starting sometime in the 10s with GPUs and the open source/open science initiatives. For all the shit I give to Google and Facebook and FAANG in general, they absolutely brought immense good to society by democratizing practical general function approximators. If there aren't any major roadblocks ML will touch every aspect of our near future lives, much like the internet. There's just too much potential - the cutting edge is just hitting the industry, our tech is already proven and it goes far deeper than classifying cats and dogs or digits. This is real science and it's huge, and it's a fact that even the money has never been remotely there for the kind of progress we are making; not to even mention the drastic difference between running ML on 1980s hardware vs a modern GPU. The kind of science we can now do from the comfort of our homes was impossible until very recently. It's like coming out of the bronze age and starting to work with steel. We build the tools as we go along. This is part of the process and we're actually doing a great job.
- YeGoblynQueenne 6y ago
  >> You're underappreciating how much more we are doing with neural networks now. From your comment above, that "a handful of neuron-like components have probably been assembled here or there" I understand that you do not have any background in AI or machine learning etc. I am curious then, where all this enthusiasm in the form of "we are doing X" statements comes from. In particular I'm interested in your use of "we". Clearly, you're not doing any of that stuff, so where does the "we" come in? Is it really prudent to express such strong views, without good understanding or personal experience of the subject matter? Are you adding anything to the conversation, by asserting all those things with such impetuousness, other than noise? I imagine that your source for all this information are articles you've read in the lay press. Unfortunately, such articles can't very well represent the real state of research in deep learning. The truth is that there has been an enormous increase in the numbers of work on deep learning being published every month- there's probably thousands of articles written in that period and uploaded on arxiv or even submitted to reputable venues- and even researches in the field have trouble keeping up. What is abundantly clear however is that the vast majority of this work is very poor quality and even the published work is not much better. It's clear also that the vast majority of this work has no lasting impact and is superseded within weeks anyway. The truth is that deep learning research is in a deep crisis and despite appearances and breathless announcements by large companies, progress has stalled and no new things are really being done. Many of the luminaries of the field, including Geoff Hinton and Yoshua Bengio, have said this in various ways.