3 ms·

I don't think it's a long-term view due to the numbers involved. For example, everyday developers might have access to 64-bit laptop computer with 4 cores @ 4

I don't think it's a long-term view due to the numbers involved. For example, everyday developers might have access to 64-bit laptop computer with 4 cores @ 4 ghz and 16 GB of RAM (literally millions of people use such a machine, zero exaggeration). When working on their own desktops, AI researches might use an 6-core or 8-core desktop with 64 GB of RAM or, rarely, 128 GB of RAM. Access to 2-32 terabytes of RAM are not in the realm of what most AI researchers get to run their algorithms on on a daily basis. Google's AlphaGo ran on[1] 1202 CPU's, 176 GPU's, and I didn't immediately find the amount of memory but on the order of magnitude you would think it is from 1 terabyte (if it's 1 GB of RAM per CPU) up to (reasonably) about 128 terabytes, with my guess being 4 TB-16 TB. However I didn't find this figure. This is hardware that exists today, but AI researchers would have to wait a few years to have normal everyday access to this kind of thing in their normal everyday devices. Right now only datacenters match the processing ability of the human mind. Although they certainly exist in one cluster, they're far from the type of thing that is in everyone's hands yet. On the other hand, once it is set up these clusters can be far faster than the human brain, which runs at really slow analog speeds. (Look up the propagation speed of neural signals.) The human brain is just slightly ahead of the curve right now without any specialized hardware. There are 7 billion extant examples of intelligent human brains. There are probably less than 10,000 clusters AI researchers can play around with for months, which have 16-128 TB of RAM and aren't used for anything more important, just whatever they feel like running on it. (Versus easily more than 15 million computers with just 16 GB of RAM, though that may not be enough.) I don't think this is very far-off. [1] http://uk.businessinsider.com/heres-how-much-computing-power-google-deepmind-needed-to-beat-lee-sedol-2016-3 http://uk.businessinsider.com/heres-how-much-computing-power...

rapnie 9y ago
Agree. I just saw doyoutrustthiscomputer.org (still on YouTube) where it was stated by some guy that Google is building a 100 million server strong datacenter for DeepMind, which has access to all their data. On the example of AlphaGo.. in 1997 a scientist and Go fan stated that it would take at least 100 yrs or longer for a computer to beat humans at Go. 20 yrs later the champion was beaten..
crote 9y ago
I'm very sceptical about it. First of all, the current state of AI and machine learning seems to be abysmal. The major breakthroughs (such as backpropagation) were made decades ago. The only reason that AI seems to be improving, is that a couple of companies are throwing an immense amount of money, data, and computing power into it. Even then, the results are relatively poor at best: systems like Google Now and Siri can only respond to a limited set of hardcoded queries and every kind of "suggestion" system seems to be actually getting worse in the last few years. Secondly, Moore's law is as good as dead. If an algorithm requires a terabyte of ram, it is all but useless for most practical purposes. I'm currently looking for a replacement of a 5-year old laptop and the newest model is at most 20% faster, with the same amount of memory! Heck, last time I got a desktop it had an AMD Phenom II X6 1055T: for the same amount of money, you'd now get an i5 8500 or a Ryzen 5 1600x. The performance gain? Well judging by the Passmark scores, it would be a jump from ~5000 to ~13000. So that's roughly 2.5 times as fast in eight years, with a jump from 45nm to 14nm. Memory progression is about the same. Considering that the progression is slowing down, it'll take quite a few decades before the performance required for current AI is generally available. Thirdly, the stuff which is impersonating AI isn't actually intelligent. Every current solution focuses on what basically boils down to a very complex set of linear regressions and similar methods. There is no adaptability, no sense of context, not even the slightest form of creativity. Everything which we consider to be intelligence is lacking. This isn't really a big surprise, because we simply don't know how it works in living creatures. Sure, we know how the neurons work, but the high-level understanding is lacking, so how could we even begin to imitate it or even improve on it? Sure, we could try simulating an actual brain, but a Blue Gene/L can only simulate half of a mouse's brain at one-tenth of the speed - and that's assuming the simulation is actually correctly modelling the biology. So no, I don't think we're on the brink of an AI revolution. We might have something with the intelligence of a service dog in a decade or three, but even that would probably be a stretch. Unless a few very major breakthroughs occur, AI is going to be a bit of a disappointment.
- logicallee 9y ago
  I understand your frustration with the results. But please bear in mind that until the wright brothers flew, the results were frustrating. What you need to look at is the raw power of internal combustion engines, and the weight they add. Versus the raw power of muscles of birds. You need to look at the raw number of neurons firing in a human brain. This is the analog device that is our benchmark. You need to look at the speed each neuron, of which there are a hundred billion, fires at: 200 hertz maximum, and more like 0.1 hertz on average. you need to look at the speed of propagation: 8 orders of magnitude slower than the interconnects in data centers. You are looking at a bird in flight, and you are looking at an internal combustion engine that outputs 1,000,000 times more power than its wings, and you are saying, "But our results are abysmal." We will, and very soon, match that output. And far, far exceed that output. For sure we will. Because it's just analog meat. And our computers are faster and wider. The poor state of algorithmic research is due to researchers having small access to the algorithms in question. We have all of the machinery for "flight" (intelligence). Personally I also think that researchers are impatient. Humans takes years to bootstrap to where they can understand the world as well as a 3-year-old. Specifically, humans take three years to understand or communicate as well as a 3-year-old. 36 months. Parity with human brains isn't the goal. The goal is achieving 144 months (12 years) of brain development closely assisted and supervised by parents and teachers, especially within the first few years, within 9 hours of unsupervised learning. But it will be done. You have the facts there in black and white.

rapnie 9y ago

Agree. I just saw doyoutrustthiscomputer.org (still on YouTube) where it was stated by some guy that Google is building a 100 million server strong datacenter for DeepMind, which has access to all their data. On the example of AlphaGo.. in 1997 a scientist and Go fan stated that it would take at least 100 yrs or longer for a computer to beat humans at Go. 20 yrs later the champion was beaten..

crote 9y ago

I'm very sceptical about it. First of all, the current state of AI and machine learning seems to be abysmal. The major breakthroughs (such as backpropagation) were made decades ago. The only reason that AI seems to be improving, is that a couple of companies are throwing an immense amount of money, data, and computing power into it. Even then, the results are relatively poor at best: systems like Google Now and Siri can only respond to a limited set of hardcoded queries and every kind of "suggestion" system seems to be actually getting worse in the last few years. Secondly, Moore's law is as good as dead. If an algorithm requires a terabyte of ram, it is all but useless for most practical purposes. I'm currently looking for a replacement of a 5-year old laptop and the newest model is at most 20% faster, with the same amount of memory! Heck, last time I got a desktop it had an AMD Phenom II X6 1055T: for the same amount of money, you'd now get an i5 8500 or a Ryzen 5 1600x. The performance gain? Well judging by the Passmark scores, it would be a jump from ~5000 to ~13000. So that's roughly 2.5 times as fast in eight years, with a jump from 45nm to 14nm. Memory progression is about the same. Considering that the progression is slowing down, it'll take quite a few decades before the performance required for current AI is generally available. Thirdly, the stuff which is impersonating AI isn't actually intelligent. Every current solution focuses on what basically boils down to a very complex set of linear regressions and similar methods. There is no adaptability, no sense of context, not even the slightest form of creativity. Everything which we consider to be intelligence is lacking. This isn't really a big surprise, because we simply don't know how it works in living creatures. Sure, we know how the neurons work, but the high-level understanding is lacking, so how could we even begin to imitate it or even improve on it? Sure, we could try simulating an actual brain, but a Blue Gene/L can only simulate half of a mouse's brain at one-tenth of the speed - and that's assuming the simulation is actually correctly modelling the biology. So no, I don't think we're on the brink of an AI revolution. We might have something with the intelligence of a service dog in a decade or three, but even that would probably be a stretch. Unless a few very major breakthroughs occur, AI is going to be a bit of a disappointment.

logicallee 9y ago
I understand your frustration with the results. But please bear in mind that until the wright brothers flew, the results were frustrating. What you need to look at is the raw power of internal combustion engines, and the weight they add. Versus the raw power of muscles of birds. You need to look at the raw number of neurons firing in a human brain. This is the analog device that is our benchmark. You need to look at the speed each neuron, of which there are a hundred billion, fires at: 200 hertz maximum, and more like 0.1 hertz on average. you need to look at the speed of propagation: 8 orders of magnitude slower than the interconnects in data centers. You are looking at a bird in flight, and you are looking at an internal combustion engine that outputs 1,000,000 times more power than its wings, and you are saying, "But our results are abysmal." We will, and very soon, match that output. And far, far exceed that output. For sure we will. Because it's just analog meat. And our computers are faster and wider. The poor state of algorithmic research is due to researchers having small access to the algorithms in question. We have all of the machinery for "flight" (intelligence). Personally I also think that researchers are impatient. Humans takes years to bootstrap to where they can understand the world as well as a 3-year-old. Specifically, humans take three years to understand or communicate as well as a 3-year-old. 36 months. Parity with human brains isn't the goal. The goal is achieving 144 months (12 years) of brain development closely assisted and supervised by parents and teachers, especially within the first few years, within 9 hours of unsupervised learning. But it will be done. You have the facts there in black and white.