6 ms·

Machine learning

tnecniv 9y ago
Yup You can trace the origins of the ANN to the 40s and 50s. Since then, statistical ML has come into vogue a few times, but not like today. Phones and other data sources (plus the algorithms to properly deal with all the data) have ushered in our current renaissance.
- bllguo 9y ago
  Huh. Why do you single out phones as a data source? That seems like a weird statement to make. Neural nets are in vogue because everyone has a phone?
  - nly 9y ago
    Machine learning and neural networks are in vogue because it's easier than ever before to collect data for them to be trained upon and process. Phones are a part of that for sure.
  - nostrademons 9y ago
    Everybody has a camera in their pocket that takes pictures for free. That's increased the number of images that you can train a neural network on (and the number of images that you might want to classify with it) dramatically. I remember when I was a kid, you had huge binders of photo albums, and the total number of pictures that a family might own numbered in the hundreds. Now I take that many photos on a week-long vacation, or sometimes just in ordinary life, and my cloud backup has maybe tens of thousands. Multiply that by a billion cell phones and that's a lot of images out there.
    - Houshalter 9y ago
      Data was never the issue for neural nets. It was the compute power required to run them. The most popular image recognition dataset today was mostly taken pre-smartphones.
      - xapata 9y ago
        When I was in school, studying ML, we were always able to beat NN performance with SVMs on the datasets available. It's only through larger training sets that NNs start to shine.
        Houshalter 9y ago
        NNs will still be beat by svms on simple datasets, even big ones. Where NNs shine is when the data has an underlying structure that can be exploited. Like how convolutions take advantage of the 2 dimensional nature of images, or RNNs take advantage of sequential data. Algorithmic improvements like dropout and proper weight initialization have also made them bunch better than they used to be.
        xapata 9y ago
        If you know what the structure is, then you could write a proper kernel and then SVMs are better again. NNs win when you don't know what the structure is and it's too complex to approximate with a standard kernel.
        Houshalter 9y ago
        I'm not aware of anything like a convolutional SVM, but I'm not terribly familiar with them. As I understand it SVMs are fundamentally shallow learners, and if you try to hack them to make them deep and recurrent or convolutional, you just get weird NNs.
        xapata 9y ago
        Shallow in the sense that SVMs are not layered, but the layering of a NN is only to enable the humble perceptron (logistic or sigmoid, usually) to model more complex functions. In contrast, the SVM doesn't need to layer, because the kernel can be arbitrarily complex.
drpgq 9y ago
Meh Duda and Hart came out in 1973. I could just be somewhat old and bitter.

Machine learning

Yup You can trace the origins of the ANN to the 40s and 50s. Since then, statistical ML has come into vogue a few times, but not like today. Phones and other data sources (plus the algorithms to properly deal with all the data) have ushered in our current renaissance.

bllguo 9y ago
Huh. Why do you single out phones as a data source? That seems like a weird statement to make. Neural nets are in vogue because everyone has a phone?
- nly 9y ago
  Machine learning and neural networks are in vogue because it's easier than ever before to collect data for them to be trained upon and process. Phones are a part of that for sure.
- nostrademons 9y ago
  Everybody has a camera in their pocket that takes pictures for free. That's increased the number of images that you can train a neural network on (and the number of images that you might want to classify with it) dramatically. I remember when I was a kid, you had huge binders of photo albums, and the total number of pictures that a family might own numbered in the hundreds. Now I take that many photos on a week-long vacation, or sometimes just in ordinary life, and my cloud backup has maybe tens of thousands. Multiply that by a billion cell phones and that's a lot of images out there.
  - Houshalter 9y ago
    Data was never the issue for neural nets. It was the compute power required to run them. The most popular image recognition dataset today was mostly taken pre-smartphones.
    - xapata 9y ago
      When I was in school, studying ML, we were always able to beat NN performance with SVMs on the datasets available. It's only through larger training sets that NNs start to shine.
      - Houshalter 9y ago
        NNs will still be beat by svms on simple datasets, even big ones. Where NNs shine is when the data has an underlying structure that can be exploited. Like how convolutions take advantage of the 2 dimensional nature of images, or RNNs take advantage of sequential data. Algorithmic improvements like dropout and proper weight initialization have also made them bunch better than they used to be.
        xapata 9y ago
        If you know what the structure is, then you could write a proper kernel and then SVMs are better again. NNs win when you don't know what the structure is and it's too complex to approximate with a standard kernel.
        Houshalter 9y ago
        I'm not aware of anything like a convolutional SVM, but I'm not terribly familiar with them. As I understand it SVMs are fundamentally shallow learners, and if you try to hack them to make them deep and recurrent or convolutional, you just get weird NNs.
        xapata 9y ago
        Shallow in the sense that SVMs are not layered, but the layering of a NN is only to enable the humble perceptron (logistic or sigmoid, usually) to model more complex functions. In contrast, the SVM doesn't need to layer, because the kernel can be arbitrarily complex.

drpgq 9y ago

Meh Duda and Hart came out in 1973. I could just be somewhat old and bitter.