4 ms·

It would be very hard to tell if an AGI running 1000x slower than a brain were intelligent. Humans babies don’t talk for about a year, but nobody has tried firi

by tlb 7y ago

It would be very hard to tell if an AGI running 1000x slower than a brain were intelligent. Humans babies don’t talk for about a year, but nobody has tried firing up a putative slow AI and waiting for 1000 years.

jacquesm 7y ago
Then let's run 1000 of them in parallel, right? You're on to the exact point I'm trying to make. The whole idea that 'just' complexity or 'just' speed is enough to make something intelligent that is not yet intelligent is off-base to a degree that resorting to metaphors to try to explain why also breaks down. I liken intelligence to life: once it exists it is obvious and self-perpetrating but before it exists it is non-obvious and no amount of fantasizing about what it would take to make it will get you incrementally closer until one day you've got it. But that day will start out just like the thousands of days before it. It's a quantum leap, a difference in quality, not an incremental one. Yes, enough order of magnitude change can make a quantitative change into a qualitative one. But so far the proof that this holds for intelligence is eluding us. Maybe we're just not intelligent enough, in the same way that you can't 100% describe a system from within the system.
- mlyle 7y ago
  If it's possible to implement AGI on a computer, a 4004 with enough RAM attached can do it... very slowly. So the speed/size trade-off is implicit unless we take a pseudo-spiritualist view that computing machines just can't be intelligent. There's two big options out there: either it requires much, much more computation than we can readily employ even to do it relatively "slowly", or we don't know how. The former has a little bearing on the balance between a transistor's computational capabilities and a neuron's, but even so is largely orthogonal. As to parallelism, it doesn't work that way. If you get nine women pregnant, you don't get a baby per month. If we've made a working but far-too-slow-for-us-to-realize-it AGI, making more of them doesn't help us understand the problem.
  - jacquesm 7y ago
    > we don't know how Exactly.
- tlb 7y ago
  But it doesn't parallelize. Watching 1000 babies for their first 0.365 days isn't going to reveal their potential. More practically, deep learning experiments were tried in the 80s, but the results weren't encouraging. If they'd left the experiments running for a sufficiently long time, they might have gotten great results. But they gave it a few days, saw the loss rate plateau and hit ^C, and that was that. Most likely, the early deep learning experiments had some parameters wrong. Wisdom about choosing parameters only came once people could run multiple experiments in a week. So it's likely the same with some hypothetical AGI. Early experiments will have some parameters wrong, and it'll either go into seizures or catatonia. We won't be able to get the parameters right until we can run experiments faster than human brains. Say, simulating 5 years of life in a week, or 250x real time.
  - jacquesm 7y ago
    Yes, we are indeed in violent agreement.
  - jacobush 7y ago
    Wow, that is a thought I didn't have before. Thanks! Still, we wouldn't even know what real time is, or what the "x" is, rather, until after we have achieved AGI. Also a mind bender.
    - tlb 7y ago
      Right, we have no idea how much computation it takes. Discoveries usually start with a less efficient algorithm, and then the performance gradually improves once we know what it needs to do. We might need 1000x the compute power to discover something as it'll eventually need once we deploy it. So I'm a big fan of ridiculously fast hardware like this article describes.
- crististm 7y ago
  I think that part of the idea of adding more speed and data to solve the AI problem started with Norvig's presentations on how more data resolved the search problem better than using more clever algorithms. People are using the 'more data' approach because it makes a (minor) dent at the problem. It's the only tool they have right now. It is my opinion that it is not enough to make us smarter in understanding the quantum leap necessary.

by tlb 7y ago

jacquesm 7y ago

Then let's run 1000 of them in parallel, right? You're on to the exact point I'm trying to make. The whole idea that 'just' complexity or 'just' speed is enough to make something intelligent that is not yet intelligent is off-base to a degree that resorting to metaphors to try to explain why also breaks down. I liken intelligence to life: once it exists it is obvious and self-perpetrating but before it exists it is non-obvious and no amount of fantasizing about what it would take to make it will get you incrementally closer until one day you've got it. But that day will start out just like the thousands of days before it. It's a quantum leap, a difference in quality, not an incremental one. Yes, enough order of magnitude change can make a quantitative change into a qualitative one. But so far the proof that this holds for intelligence is eluding us. Maybe we're just not intelligent enough, in the same way that you can't 100% describe a system from within the system.

mlyle 7y ago
If it's possible to implement AGI on a computer, a 4004 with enough RAM attached can do it... very slowly. So the speed/size trade-off is implicit unless we take a pseudo-spiritualist view that computing machines just can't be intelligent. There's two big options out there: either it requires much, much more computation than we can readily employ even to do it relatively "slowly", or we don't know how. The former has a little bearing on the balance between a transistor's computational capabilities and a neuron's, but even so is largely orthogonal. As to parallelism, it doesn't work that way. If you get nine women pregnant, you don't get a baby per month. If we've made a working but far-too-slow-for-us-to-realize-it AGI, making more of them doesn't help us understand the problem.
- jacquesm 7y ago
  > we don't know how Exactly.
tlb 7y ago
But it doesn't parallelize. Watching 1000 babies for their first 0.365 days isn't going to reveal their potential. More practically, deep learning experiments were tried in the 80s, but the results weren't encouraging. If they'd left the experiments running for a sufficiently long time, they might have gotten great results. But they gave it a few days, saw the loss rate plateau and hit ^C, and that was that. Most likely, the early deep learning experiments had some parameters wrong. Wisdom about choosing parameters only came once people could run multiple experiments in a week. So it's likely the same with some hypothetical AGI. Early experiments will have some parameters wrong, and it'll either go into seizures or catatonia. We won't be able to get the parameters right until we can run experiments faster than human brains. Say, simulating 5 years of life in a week, or 250x real time.
- jacquesm 7y ago
  Yes, we are indeed in violent agreement.
- jacobush 7y ago
  Wow, that is a thought I didn't have before. Thanks! Still, we wouldn't even know what real time is, or what the "x" is, rather, until after we have achieved AGI. Also a mind bender.
  - tlb 7y ago
    Right, we have no idea how much computation it takes. Discoveries usually start with a less efficient algorithm, and then the performance gradually improves once we know what it needs to do. We might need 1000x the compute power to discover something as it'll eventually need once we deploy it. So I'm a big fan of ridiculously fast hardware like this article describes.
crististm 7y ago
I think that part of the idea of adding more speed and data to solve the AI problem started with Norvig's presentations on how more data resolved the search problem better than using more clever algorithms. People are using the 'more data' approach because it makes a (minor) dent at the problem. It's the only tool they have right now. It is my opinion that it is not enough to make us smarter in understanding the quantum leap necessary.