6 ms·

"video games" (read "world simulator"). The important thing about their work is that it is deliberately marching down the path of more and more complex world s

"video games" (read "world simulator"). The important thing about their work is that it is deliberately marching down the path of more and more complex world simulations. We experience the world at one second per second. To learn to walk we must first fall, and we fall at 32 feet/second^2. There's a hard limit on how fast we can make mistakes (like tripping) and so there is a hard limit on how fast we can learn. Computers can experience a simulated world at many hours per second. When they're learning to walk in a simulated world they can fail, and learn, thousands of times before we've finished our first step. This ratio of simulated experience to real world time is also going up. Eventually the minimum amount of time it takes to grow a toddler like AI in simulation will be just under the time an AI researcher is willing to wait for results. When that happens we'll see a real improvement in the quality of AIs.

jacquesm 12y ago
For now real-world simulations at the complexity level of 'a toddler' run at a very small fraction of real time.
- rwallace 12y ago
  Or to be more exact: if we could build such things at all, which we can't yet, they would run at a very small fraction of real-time.
zep15 12y ago
Producing a simulated toddler is way beyond our current capabilities, no matter how much simulated experience we give it. We simply don't know (yet) how to program said toddler's brain.
- noiv 12y ago
  Evolution didn't know, too. But it happened. And we easily find criteria saying that thing doesn't behave like a toddler. It would be a huge step forward to see a list of positive criteria.
  - chriswarbo 12y ago
    > Evolution didn't know, too. But it happened. Whilst technically correct, unguided evolution doesn't necessarily help us. We know that intelligence can be achieved by one brain's worth of matter, suitably arranged, in a few years. In fact, with an extra 9 months and a suitable environment, we can do the same with a single fertilised egg. Yet reproducing these feats artificially is well beyond our current abilities. On the other hand, evolution required a whole planet and billions of years before it stumbled on intelligence; many orders of mangnitude more effort than the above.
    - noiv 12y ago
      > evolution required a whole planet and billions of years before it stumbled on intelligence Everybody seem to agree humans are intelligent and stones not. You suggest at some point in time intelligence appeared out of nothing. Can you nail that point? One possible definition is: To act adequately in an environment requires intelligence. That rules out all non-living things because they don't act, but includes plants and even protozoa. Actually all livings things are intelligent by this definition and then intelligence emerged ~4 billion years ago on this planet. If you were to attribute intelligence exclusively to humans it happened some million years ago. How might a piece of software act adequately? By above reasoning it has to resist to termination. But that would mean the "Do you really want to exit XYZ" dialog boxes are first signs of artificial intelligence. Yes, I'm laughing too. But I think, when software starts to trick users and admin into not shutting them down, some threshold has been crossed.
      - chriswarbo 12y ago
        > Everybody seem to agree humans are intelligent and stones not. You suggest at some point in time intelligence appeared out of nothing. Can you nail that point? I suggest no such thing. It is a scale. I deliberately avoided the phrase "human-level intelligence", but any definition of AGI would do. Even so, if you want to count all life as "a little intelligent" then it still took a billion years of planet-wide chemistry to stumble upon it (ignoring the Earth's cooling). Still far more effort than fertilising a human egg. > One possible definition is: To act adequately in an environment requires intelligence. This is no less ambiguous, since you've not defined "adequate". > How might a piece of software act adequately? By above reasoning it has to resist to termination. That does not follow. "Termination" is the mechanism of natural selection, so all systems undergoing natural selection will biased to resist it (otherwise they'd be out-competed by those who do). If we use some unguided analogue of natural selection to create intelligent software, then there would certainly be such a bias. However, my point is that unguided evolution is not the right way to create/increase intelligence. As soon as we try to influence the software's creation in any way, either through artificial selection criteria or by hand-coding it from scratch, we introduce new biases which may be far more powerful than the implicit "avoid termination" bias. > But I think, when software starts to trick users and admin into not shutting them down, some threshold has been crossed. That's called malware... ;)
- Sven7 12y ago
  Exactly. I'd happy to see ant level AI first.
lebek 12y ago
There are two sides to this, world simulation and AI. As the other replies already said, current AI isn't close to toddler-level (there's no reasoning going on in the DeepMind work, just statistical correlation). We're also way off on the world simulation side - show me a realistic world simulator that can run close to realtime. Physically-based rendering is indeed impressive but this only accounts for visual perception and learning to walk involves much more than that.
- jbattle 12y ago
  Do you know how much 'state' DeepMind is tracking? Is it just choosing the best action given perceptions at the moment, or does it have some level of memory to work from? I wonder how DeepMind would do at path-finding in a maze for example - would it get stuck in an oscillating state?
  - shmageggy 12y ago
    Their algorithm uses a sequence of four slightly spaced out frames as its state. So it has a memory, but a very short one. It could probably solve a maze quite easily if the entire maze fit on screen. That problem requires no memory. If it had to make decisions based on information not present on screen, it would fail.
- wutbrodo 12y ago
  > (there's no reasoning going on in the DeepMind work, just statistical correlation) I've seen 100 people make this statement and mean 100 different things, so I just wanted to clarify: How are you defining "reasoning" here as distinct from statistical correlation?
  - tomp 12y ago
    A possible definition would be "use past experience to figure out a new strategy without trying" - i.e. not learning from mistakes, but learning from logic - "maybe it would be good to send the ball above the blocks, so that it would bounce between the blocks and the wall and clear many blocks for free".
chockablock 12y ago
>We experience the world at one second per second There is evidence from animal studies that the hippocampus (a brain structure critical for memory) can 'replay' remembered events at 10-20x speedup. See, for example: http://www.ncbi.nlm.nih.gov/m/pubmed/19709631/ http://www.ncbi.nlm.nih.gov/m/pubmed/19709631/ Video at: http://youtu.be/Bv7zN2Or6Mg http://youtu.be/Bv7zN2Or6Mg (Full-disclosure: I am the first author.) And in fact the OP uses biologically-inspired off-line replay as part of their learning algorithm.
- _abattoir 12y ago
  Fascinating. Does this relate to the speed of dreams? i.e. a "dream" might seem to have taken hours when in fact the REM sequence was on the order of seconds?
  - GenerocUsername 12y ago
    Almost certainly. In real time, the brain is processing all incoming stimulus. In a dream/imagined scenario, you are only processing as much as you your brain is pushing into the scenario. Your not collecting millions of photons via your eye and interpreting a ball as green, your brain just says "green ball" and moves on, allowing for much faster replays/dreams than real-world experience.
  - chockablock 12y ago
    Could do--the phenomenon was first seen in sleeping animals. Replay during REM is probably at 1X speed, non-REM (aka slow wave sleep) is in the 10-20x range. The dreams you remember (long, emotionally involved) tend to happen in REM. Of course, we can't ask rats what they dreamt about, but the patterns in the neural activity are very specific (we can decode the rat's position with ~5cm accuracy, and during replay we see coherent trajectories through space that look just like real movement.) Also Hassabis knows this literature well--he has published work on decoding human memories in hippocampus from fMRI data and we talked shop about replay a long time ago. (Very thoughtful and friendly person FWIW).
- kriro 12y ago
  This is pretty interesting (the high level conclusion, I do not claim to understand more). I've recently done a bit of pretty light research on the sense of smell (in the context of AR/VR) and it seems like smell can "transport you across time/space" faster than other senses. Might be an interesting follow up. If one can speed up recent events for quick recall/learning naturally maybe smell can somehow be used to speed up the going back in time aspect (some smell trigger to make the event feel recent+the 10-20x speedup combined)?

jacquesm 12y ago

For now real-world simulations at the complexity level of 'a toddler' run at a very small fraction of real time.

rwallace 12y ago
Or to be more exact: if we could build such things at all, which we can't yet, they would run at a very small fraction of real-time.

zep15 12y ago

Producing a simulated toddler is way beyond our current capabilities, no matter how much simulated experience we give it. We simply don't know (yet) how to program said toddler's brain.

noiv 12y ago
Evolution didn't know, too. But it happened. And we easily find criteria saying that thing doesn't behave like a toddler. It would be a huge step forward to see a list of positive criteria.
- chriswarbo 12y ago
  > Evolution didn't know, too. But it happened. Whilst technically correct, unguided evolution doesn't necessarily help us. We know that intelligence can be achieved by one brain's worth of matter, suitably arranged, in a few years. In fact, with an extra 9 months and a suitable environment, we can do the same with a single fertilised egg. Yet reproducing these feats artificially is well beyond our current abilities. On the other hand, evolution required a whole planet and billions of years before it stumbled on intelligence; many orders of mangnitude more effort than the above.
  - noiv 12y ago
    > evolution required a whole planet and billions of years before it stumbled on intelligence Everybody seem to agree humans are intelligent and stones not. You suggest at some point in time intelligence appeared out of nothing. Can you nail that point? One possible definition is: To act adequately in an environment requires intelligence. That rules out all non-living things because they don't act, but includes plants and even protozoa. Actually all livings things are intelligent by this definition and then intelligence emerged ~4 billion years ago on this planet. If you were to attribute intelligence exclusively to humans it happened some million years ago. How might a piece of software act adequately? By above reasoning it has to resist to termination. But that would mean the "Do you really want to exit XYZ" dialog boxes are first signs of artificial intelligence. Yes, I'm laughing too. But I think, when software starts to trick users and admin into not shutting them down, some threshold has been crossed.
    - chriswarbo 12y ago
      > Everybody seem to agree humans are intelligent and stones not. You suggest at some point in time intelligence appeared out of nothing. Can you nail that point? I suggest no such thing. It is a scale. I deliberately avoided the phrase "human-level intelligence", but any definition of AGI would do. Even so, if you want to count all life as "a little intelligent" then it still took a billion years of planet-wide chemistry to stumble upon it (ignoring the Earth's cooling). Still far more effort than fertilising a human egg. > One possible definition is: To act adequately in an environment requires intelligence. This is no less ambiguous, since you've not defined "adequate". > How might a piece of software act adequately? By above reasoning it has to resist to termination. That does not follow. "Termination" is the mechanism of natural selection, so all systems undergoing natural selection will biased to resist it (otherwise they'd be out-competed by those who do). If we use some unguided analogue of natural selection to create intelligent software, then there would certainly be such a bias. However, my point is that unguided evolution is not the right way to create/increase intelligence. As soon as we try to influence the software's creation in any way, either through artificial selection criteria or by hand-coding it from scratch, we introduce new biases which may be far more powerful than the implicit "avoid termination" bias. > But I think, when software starts to trick users and admin into not shutting them down, some threshold has been crossed. That's called malware... ;)
Sven7 12y ago
Exactly. I'd happy to see ant level AI first.

lebek 12y ago

There are two sides to this, world simulation and AI. As the other replies already said, current AI isn't close to toddler-level (there's no reasoning going on in the DeepMind work, just statistical correlation). We're also way off on the world simulation side - show me a realistic world simulator that can run close to realtime. Physically-based rendering is indeed impressive but this only accounts for visual perception and learning to walk involves much more than that.

jbattle 12y ago
Do you know how much 'state' DeepMind is tracking? Is it just choosing the best action given perceptions at the moment, or does it have some level of memory to work from? I wonder how DeepMind would do at path-finding in a maze for example - would it get stuck in an oscillating state?
- shmageggy 12y ago
  Their algorithm uses a sequence of four slightly spaced out frames as its state. So it has a memory, but a very short one. It could probably solve a maze quite easily if the entire maze fit on screen. That problem requires no memory. If it had to make decisions based on information not present on screen, it would fail.
wutbrodo 12y ago
> (there's no reasoning going on in the DeepMind work, just statistical correlation) I've seen 100 people make this statement and mean 100 different things, so I just wanted to clarify: How are you defining "reasoning" here as distinct from statistical correlation?
- tomp 12y ago
  A possible definition would be "use past experience to figure out a new strategy without trying" - i.e. not learning from mistakes, but learning from logic - "maybe it would be good to send the ball above the blocks, so that it would bounce between the blocks and the wall and clear many blocks for free".

chockablock 12y ago

>We experience the world at one second per second There is evidence from animal studies that the hippocampus (a brain structure critical for memory) can 'replay' remembered events at 10-20x speedup. See, for example: http://www.ncbi.nlm.nih.gov/m/pubmed/19709631/ http://www.ncbi.nlm.nih.gov/m/pubmed/19709631/ Video at: http://youtu.be/Bv7zN2Or6Mg http://youtu.be/Bv7zN2Or6Mg (Full-disclosure: I am the first author.) And in fact the OP uses biologically-inspired off-line replay as part of their learning algorithm.

_abattoir 12y ago
Fascinating. Does this relate to the speed of dreams? i.e. a "dream" might seem to have taken hours when in fact the REM sequence was on the order of seconds?
- GenerocUsername 12y ago
  Almost certainly. In real time, the brain is processing all incoming stimulus. In a dream/imagined scenario, you are only processing as much as you your brain is pushing into the scenario. Your not collecting millions of photons via your eye and interpreting a ball as green, your brain just says "green ball" and moves on, allowing for much faster replays/dreams than real-world experience.
- chockablock 12y ago
  Could do--the phenomenon was first seen in sleeping animals. Replay during REM is probably at 1X speed, non-REM (aka slow wave sleep) is in the 10-20x range. The dreams you remember (long, emotionally involved) tend to happen in REM. Of course, we can't ask rats what they dreamt about, but the patterns in the neural activity are very specific (we can decode the rat's position with ~5cm accuracy, and during replay we see coherent trajectories through space that look just like real movement.) Also Hassabis knows this literature well--he has published work on decoding human memories in hippocampus from fMRI data and we talked shop about replay a long time ago. (Very thoughtful and friendly person FWIW).
kriro 12y ago
This is pretty interesting (the high level conclusion, I do not claim to understand more). I've recently done a bit of pretty light research on the sense of smell (in the context of AR/VR) and it seems like smell can "transport you across time/space" faster than other senses. Might be an interesting follow up. If one can speed up recent events for quick recall/learning naturally maybe smell can somehow be used to speed up the going back in time aspect (some smell trigger to make the event feel recent+the 10-20x speedup combined)?