5 ms·
What general ideas did we sow in the past 20-30 years that, through minor iterations, are coming to (surprising) fruition today?
by roymurdock 9y ago
What general ideas did we sow in the past 20-30 years that, through minor iterations, are coming to (surprising) fruition today?
- santaclaus 9y agoMachine learning
- tnecniv 9y agoYup You can trace the origins of the ANN to the 40s and 50s. Since then, statistical ML has come into vogue a few times, but not like today. Phones and other data sources (plus the algorithms to properly deal with all the data) have ushered in our current renaissance.
- bllguo 9y agoHuh. Why do you single out phones as a data source? That seems like a weird statement to make. Neural nets are in vogue because everyone has a phone?
- nly 9y agoMachine learning and neural networks are in vogue because it's easier than ever before to collect data for them to be trained upon and process. Phones are a part of that for sure.
- nostrademons 9y agoEverybody has a camera in their pocket that takes pictures for free. That's increased the number of images that you can train a neural network on (and the number of images that you might want to classify with it) dramatically. I remember when I was a kid, you had huge binders of photo albums, and the total number of pictures that a family might own numbered in the hundreds. Now I take that many photos on a week-long vacation, or sometimes just in ordinary life, and my cloud backup has maybe tens of thousands. Multiply that by a billion cell phones and that's a lot of images out there.
- Houshalter 9y agoData was never the issue for neural nets. It was the compute power required to run them. The most popular image recognition dataset today was mostly taken pre-smartphones.
- xapata 9y agoWhen I was in school, studying ML, we were always able to beat NN performance with SVMs on the datasets available. It's only through larger training sets that NNs start to shine.
- Houshalter 9y agoNNs will still be beat by svms on simple datasets, even big ones. Where NNs shine is when the data has an underlying structure that can be exploited. Like how convolutions take advantage of the 2 dimensional nature of images, or RNNs take advantage of sequential data. Algorithmic improvements like dropout and proper weight initialization have also made them bunch better than they used to be.
- xapata 9y agoIf you know what the structure is, then you could write a proper kernel and then SVMs are better again. NNs win when you don't know what the structure is and it's too complex to approximate with a standard kernel.
- Houshalter 9y agoI'm not aware of anything like a convolutional SVM, but I'm not terribly familiar with them. As I understand it SVMs are fundamentally shallow learners, and if you try to hack them to make them deep and recurrent or convolutional, you just get weird NNs.
- xapata 9y ago
- drpgq 9y agoMeh Duda and Hart came out in 1973. I could just be somewhat old and bitter.
- contingencies 9y agoFree unix. Cryptocurrencies. Military robotics. Personal drones. Augmented reality. Freedom of information. Pervasive media piracy. Darknets. State level hacking. Technofascist dystopia in general.
- patrickk 9y agoIncreasingly cheap wind, solar and battery hardware. Smartphones (also enables vastly cheaper batteries and drone components).