3 ms·

Beyond that, as an outsider looking in I may be off base, but the rise of machine learning seems mostly fueled by the rise of GPGPU. So I would say it's not di

by sbov 10y ago

Beyond that, as an outsider looking in I may be off base, but the rise of machine learning seems mostly fueled by the rise of GPGPU. So I would say it's not different, in that we experienced something similar with regular CPUs until about 2007. But it may be different depending upon how much, and how long, we can continue to cheaply speed up GPUs.

Florin_Andrei 10y ago
If that is the case, if the limiting factor was hardware, then we should see continuing growth for a good stretch going forward. There are quite a few companies looking to build specialized chips for machine learning, that could outcompete GPUs the way GPUs outcompete CPUs in this field.
gumby 10y ago
It's the rise of cheap computation and the availability of enormous data sets. It's a bit of exaggeration to say so, so don't take it literally but in many ways the NN work is where it was when Papert and Minsky wrote the "Perceptrons" book in 1969. But as Stalin is supposed to have said, "quantity has a quality all its own" (thus the immense quantity of cycles and data people have at their fingertips causes a discontinuous change in functionality). Don't over-read this; I don't mean to denigrate a lot of cool work done in the past few years. But conceptually you are correct on the computation side.
- parallaxView 10y ago
  This is fair, and one should certainly acknowledge the progress we have made as a functionality all its own, but there seems to be something else at work in the deep learning paradigm. I think (ha) we are dipping our toes in the water of intelligence replication, and that deep learning is a very real crystallization of hundreds of years (using Decartes as my x-axis) of scientifically accurate data amalgamation on what it is we are doing when we maintain a thought, so while the Perceptron is definitely the architecture for passing from the realm of information to the realm of data, what fascinates me is the proximity to actual human reasoning that is occurring presently within the 'mind' of, say, Watson, which I have to say is an incredibly providential name given IBM's founder, Sherlock's best bud, and James Watt's influence on electricity. It's almost as if intelligent design isn't all that far fetched....lol

by sbov 10y ago

Florin_Andrei 10y ago

If that is the case, if the limiting factor was hardware, then we should see continuing growth for a good stretch going forward. There are quite a few companies looking to build specialized chips for machine learning, that could outcompete GPUs the way GPUs outcompete CPUs in this field.

gumby 10y ago

It's the rise of cheap computation and the availability of enormous data sets. It's a bit of exaggeration to say so, so don't take it literally but in many ways the NN work is where it was when Papert and Minsky wrote the "Perceptrons" book in 1969. But as Stalin is supposed to have said, "quantity has a quality all its own" (thus the immense quantity of cycles and data people have at their fingertips causes a discontinuous change in functionality). Don't over-read this; I don't mean to denigrate a lot of cool work done in the past few years. But conceptually you are correct on the computation side.

parallaxView 10y ago
This is fair, and one should certainly acknowledge the progress we have made as a functionality all its own, but there seems to be something else at work in the deep learning paradigm. I think (ha) we are dipping our toes in the water of intelligence replication, and that deep learning is a very real crystallization of hundreds of years (using Decartes as my x-axis) of scientifically accurate data amalgamation on what it is we are doing when we maintain a thought, so while the Perceptron is definitely the architecture for passing from the realm of information to the realm of data, what fascinates me is the proximity to actual human reasoning that is occurring presently within the 'mind' of, say, Watson, which I have to say is an incredibly providential name given IBM's founder, Sherlock's best bud, and James Watt's influence on electricity. It's almost as if intelligent design isn't all that far fetched....lol