4 ms·

>they don't work. In the same way that ANNs didn't work for quite some time, until we've had the compute and the data to train them successfully? I get that i

by b3kart 6y ago

>they don't work. In the same way that ANNs didn't work for quite some time, until we've had the compute and the data to train them successfully? I get that it's important to prove that an idea is worthwhile, and the easiest way to do that is to use it to solve a practical problem. At the same time, I am conscious that we shouldn't put all our eggs in the deep learning basket: who knows where the ceiling is going to be. Don't get me wrong, I like deep learning, and you have to be silly not to admit how successful it has been. But the field would be so much more boring if not for the people with alternative views and ideas.

p1esk 6y ago
No, not in the same way. Convnets and lstms worked great when they were invented in the 90s. Yes, they were limited by the available compute power but they did work well right away.
- dougabug 6y ago
  ConvNets were invented in the 80s.
  - p1esk 6y ago
    The first convnet that is similar to what we use today was described in 1989 [1], and immediately became the best method to do a handwritten digit recognition. [1] http://yann.lecun.com/exdb/publis/pdf/lecun-89e.pdf http://yann.lecun.com/exdb/publis/pdf/lecun-89e.pdf
- lacker 6y ago
  When I was in grad school in 1999 studying AI the general consensus was that neural networks didn't work very well, and that you were better off with the more mathematically grounded methods like support vector machines. TD-Gammon was just about the only success story for neural networks, there hadn't been much else since 1992.
  - p1esk 6y ago
    I don't know where that "general consensus" came from, because in 1998 Lecun clearly showed [1] convnets beating all other models, including SVMs, at image recognition. [1] http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf
    - Componica 6y ago
      It like other neural network research was ignored because neural networks were considered a dead-end at the time. In the early 2000s I recreated his LeNet-5 implementation, and no one was interested despite the great results I was getting in OCR and medical image processing with tumor detection. Younger people don't realize there was strong bias against using neural networks in the late 90s up until Hinton's talk on NNs around 2007. I get the feeling we're going through the same thing where novel research is becoming ignored because everything must fit the deep learning paradigm to be noticed.
      - p1esk 6y ago
        Most research will be ignored if it does not have good results, regardless of whether it's novel, or whether it's DL. In fact, a lot of recently published DL research is ignored for exactly this reason. Top DL conferences are so competitive currently that the quality bar is pretty high. There are lots of ideas floating around in AI field. Some of them might be good, most are not. If you have an idea and want others to look at it you better demonstrate how it outperforms every other method when applied to some task.
        BlueTemplar 6y ago
        OP's point is that research is/was being ignored despite having good results. But this is normal, it just takes time, and a critical mass of good results for most researchers to switch to the new paradigm. (A decade is a very short time in the history of science.)
        6y ago

by b3kart 6y ago

p1esk 6y ago

No, not in the same way. Convnets and lstms worked great when they were invented in the 90s. Yes, they were limited by the available compute power but they did work well right away.

dougabug 6y ago
ConvNets were invented in the 80s.
- p1esk 6y ago
  The first convnet that is similar to what we use today was described in 1989 [1], and immediately became the best method to do a handwritten digit recognition. [1] http://yann.lecun.com/exdb/publis/pdf/lecun-89e.pdf http://yann.lecun.com/exdb/publis/pdf/lecun-89e.pdf
lacker 6y ago
When I was in grad school in 1999 studying AI the general consensus was that neural networks didn't work very well, and that you were better off with the more mathematically grounded methods like support vector machines. TD-Gammon was just about the only success story for neural networks, there hadn't been much else since 1992.
- p1esk 6y ago
  I don't know where that "general consensus" came from, because in 1998 Lecun clearly showed [1] convnets beating all other models, including SVMs, at image recognition. [1] http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf
  - Componica 6y ago
    It like other neural network research was ignored because neural networks were considered a dead-end at the time. In the early 2000s I recreated his LeNet-5 implementation, and no one was interested despite the great results I was getting in OCR and medical image processing with tumor detection. Younger people don't realize there was strong bias against using neural networks in the late 90s up until Hinton's talk on NNs around 2007. I get the feeling we're going through the same thing where novel research is becoming ignored because everything must fit the deep learning paradigm to be noticed.
    - p1esk 6y ago
      Most research will be ignored if it does not have good results, regardless of whether it's novel, or whether it's DL. In fact, a lot of recently published DL research is ignored for exactly this reason. Top DL conferences are so competitive currently that the quality bar is pretty high. There are lots of ideas floating around in AI field. Some of them might be good, most are not. If you have an idea and want others to look at it you better demonstrate how it outperforms every other method when applied to some task.
      - BlueTemplar 6y ago
        OP's point is that research is/was being ignored despite having good results. But this is normal, it just takes time, and a critical mass of good results for most researchers to switch to the new paradigm. (A decade is a very short time in the history of science.)
        6y ago