4 ms·
Another nice example are Fourier transforms. It took quite a while from 1822 until we had great applications for it in microcontrollers and DSPs for sound proce
by carlmr 8y ago
Another nice example are Fourier transforms. It took quite a while from 1822 until we had great applications for it in microcontrollers and DSPs for sound processing.
However I can also kind of see the argument here that the deep learning ideas don't seem to have such a fundamental idea behind them. We modeled how biology works at a very simple and abstract level. Now we have more computing power (and especially more data) so we could find a few applications.
But to me at least it doesn't look like a solid theory to build upon. It doesn't deliver insights into how things work like Fourier transforms did with signals and Bayes did with probabilities.
- bunderbunder 8y ago> We modeled how biology works That's the popular story behind neural networks. Like any good popular explanation of a STEM concept, it's as useful as it is inaccurate.
- carlmr 8y ago>at a simple and abstract level It would be nice if you quoted honestly.
- bunderbunder 8y agoSorry, I wasn't meaning to quote mine. I honestly didn't think that part was relevant to the point I was making. Neural networks have taken some inspiration from biology, but that doesn't account for the vast bulk of the work. Biology analogies play a much bigger part in popular explanations than they did in the development of the technology. (edit) Here, for example, take a look at the original paper on perceptrons. Whole lotta math. This having been released during one of the AI flaps, there was also a fair bit of psychology talk. But not much indication that he was just trying to copy a brain and otherwise didn't understand how it works. https://blogs.umass.edu/brain-wars/files/2016/03/rosenblatt-1957.pdf https://blogs.umass.edu/brain-wars/files/2016/03/rosenblatt-... Similar happens with deep learning. They are grounded in theory, most notably a mathematical proof that deep MLPs are, in principle, capable of learning any mathematical function. You just won't see much of that stuff if you aren't actually reading the papers, because most people aren't that interested in vector calculus. Much easier to explain things by analogy.
- deleted 8y ago[deleted]
- nkozyra 8y ago> That's the popular story behind neural networks. Like any good popular explanation of a STEM concept, it's as useful as it is inaccurate. To be fair, our own understanding of "how it works" in practice is not necessarily set in stone. Approximating our belief in how it works is the best case scenario, and expecting it to perfectly mirror human reasoning is unrealistic and frankly not all that appealing from a problem-solving stance. Human reasoning has a lot of built-in buffers and fault tolerance, and it is, itself, a fairly narrow AI.
- twtw 8y agoFun fact: the FFT (the algorithm that is actually used in applications today) was originally developed around 1805 by Gauss, but he didn't publish the work. Sorta crazy since it precedes Fourier's work.
- dekhn 8y agoThis is a bit of a stretch. The algorithm used today is due to Tukey. Gauss's work was great, but it's not the algorithm used today. Also, another interesting fact about fourier transforms- in the old days, computing FTs was too expensive, so physicists used lenses- which perform approximate FTs- to decompose signals into spectra.
- twtw 8y agoFrom https://www.cis.rit.edu/class/simg716/Gauss_History_FFT.pdf https://www.cis.rit.edu/class/simg716/Gauss_History_FFT.pdf > Thus, Gauss' algorithm is as general and powerful as the Cooley-Tukey common factor algorithm and is, in fact, equivalent to a decimation in frequency algorithm adapted to a real data sequence.