11 ms·

Er, you're assuming that this strong AI would be smarter than a human brain, what is not at all obvious.

by mbq 13y ago

ceejayoz 13y ago
I thought the general assumption was that a strong AI would be able to rapidly self-improve. http://en.wikipedia.org/wiki/Technological_singularity http://en.wikipedia.org/wiki/Technological_singularity
- ars 13y ago
  Humans haven't managed it, so why so you assume a computer will?
  - eli_gottlieb 13y ago
    Humans are built out of neural structures, which are black-box learners. Any AI built out of a white-box learner would at least hypothetically be able to self-improve up to some fundamental physical-mathematical limit imposed by its supply of raw compute-power. The interesting question is: where is that upper limit on intelligence-per-FLOPS? EDIT: Fixed it to "FLOPS". We already have AI algorithms that Solve The Problem if given unlimited "thinking time" between steps of interaction with their environment (up to the point of needing a Halting Oracle), and we have extensions of those that are asymptotically speed-optimal modulo some additive or multiplicative constants (which are larger than our current universe). I strongly suspect that scaling down these Idealized AI Agents to real, useful programs isn't going to just involve feeding raw instances of the Ideal Agents small enough environments to be tractable, but actually finding or deriving some algorithm that can scale continuously up and down with available computing power per unit of real time, where the time units are the actual real-time length of an interaction cycle.
    - seiji 13y ago
      Check out Accelerando. Basically, mumble mumble computronium mumble mumble.
      - eli_gottlieb 13y ago
        mumble mumble stop giving them ideas mumble mumble
    - ars 13y ago
      > are black-box learners. Any AI built out of a white-box learner To have a white-box learner would imply that humans were smart enough to build it. Yet it's unlikely that humans can build something significantly smarter than a human, so this AI would have the same limitation: It's unable (not smart enough) to build a smarter AI.
      - eli_gottlieb 13y ago
        Intelligence is not magic, and in some way almost definitely scales with computing power. For instance, if you have infinite computing power, you just have to dovetail over every possible Turing Machine until one of them "wakes up" and does AI-y things. Yes, there are in fact real approaches based on a more sophisticated version of such a Universal Search. In fact, we routinely make use of machine-learning algorithms that are "smarter than a human" in particular narrow domains. Are they smarter than a human at general intelligence, of the kind necessary to start looking more like a scifi "AI" than a cheap trick? No, but they are an indication that such a thing could be possible. Overall, I would say that with Machine Learning maturing and Universal AI being a formal science at all, we are only just now beginning to find out where the actual tradeoffs lie in the design and construction of intelligent agents.
        ars 13y ago
        > Intelligence is not magic, and in some way almost definitely scales with computing power. The speed of intelligence scales. But not the quality of it. The quality of intelligence is something very hard to quantify. But as simply as I can state it, it's the quality of coming up with an idea that is not just an extension of an existing one, but an idea that came seemingly from nowhere. To someone else the idea appears magic, and no amount of thinking by that other person can lead him in a path that will end up there. > but they are an indication that such a thing could be possible. Real genius is not just "more" intelligence, it's intelligence of a different type that a non-genius simply can not do, no matter how they try. Analogously, just because a computer can do something fast does not give it that hard to define quality of true intelligence. To make (rather than randomly search for) a computer with real intelligence we need to understand what it is, and where it comes from, and we don't know either one of those things. Our best (smartest) computers are all variations of "search randomly" (for example chess engines). Those can do some amazing things, but their limitations will not be overcome by doing more of the same. > infinite computing power .. every possible Talking about infinity is a total red herring and you know that - especially because if your method of increasing intelligence is "search randomly" then an AI will not be able to make a better AI - all it can do is continue the random search humans already started.
  - ceejayoz 13y ago
    We're talking about something purely hypothetical at this point, so who knows if they're right... but I'd assume that an AI would benefit from the vastly faster processing speed of a computer, the ability to have a processing facility larger than a blob of fat that fits in our cranium, a more accurate memory, etc... plus the ability to iterate on its own operating system.
- hharrison 13y ago
  Eventually, but strong AI just means intelligence matching or exceeding human intelligence. We already have billions of entities with human intelligence, and it is taking us a long time to produce something smarter than ourselves. If the first strong AI is just a little smarter than us, and if it chooses to put its energy toward the creation of more AIs, then maybe it will eventually produce something smarter than itself. But it's not so simple that we just get to the singularity straight away.
  - seiji 13y ago
    The problem with people is multiplicative intelligence parity. (Like that phrase? Feel free to use it as your own.) You can find one smart person. But how do you find another smart person for them to work with? With AI-level smarts, you reduce the coordination problem to zero since it can spawn multiple copies of its own brain state (maybe?) and work concurrently and intelligently on the same problem. Just ignore the problem of killing off divergent brain states once the task is complete (nobody cares about the life of a kage bunshin). I sure could get a lot more done if I had five more of me (unpaid, of course) to tackle a problem all at once.
PeterisP 13y ago
From the evidence of how fast new botnets spread, and the current prices of 0-day vulnerabilities we can assume that any strong-AI can easily get access to immense computing power (percentage points of globally installed CPU's). The difference is that it can use them not for sending low-return spam, but for multiplying its power - even if it doesn't give a qualitative difference, there should be a quantitative difference (how many different tasks it can do at once) when a strong AI gains many orders of magnitude more computing power than any laboratory/datacenter where it was initially created.
- VLM 13y ago
  Latency and bandwidth are going to suck and that might be important. There's an assumption in most AI discussions that AI "must" be embarrassingly parallelizable. For example, much as my brain is parallelized, surely an AI would have to be the same way, much as all heavier than air human flight required was a faithful reproduction of bird anatomy (LOL). There's also a certain assumption that intelligence implies a level of self awareness that most humans don't have therefore AI would be highly self aware... There's also a whole industry of sophistry devoted to proving no humans are intelligent and IQ as a concept or any other numerical measure of intelligence does not exist and there is no way to compare intelligence, and right or wrong those folks will surely make life difficult for people improving / testing / upgrading an AI, either because they're right or they'll be making political protests.
- chongli 13y ago
  strong-AI can easily get access to immense computing power But is it the right kind of computing power? The human brain is a very different sort of machine, architecturally speaking, from any computer currently in existence. Current computers have a very difficult time dealing with massive, parallel, random data access. Perhaps strong AI requires billions of threads randomly accessing a database which is terabytes in size? That is going to make the memory bus very, very sad.
- mbq 13y ago
  To begin with, it may not want to harvest more computational power. Even if, it may be initially too stupid to do this. Even if, the internet speed and small reliability of those nodes may make them unusable to this purpose.
Houshalter 13y ago
Strong AI generally means smarter-than human.
- mbq 13y ago
  What does not necessary mean more capable in case of making AI systems.
  - Houshalter 13y ago
    No but it does mean it's possible to automate the work of further improving AI. It will also very likely be much smarter than human. Being exactly the same intelligence as a human is a very narrow target to hit.

Er, you're assuming that this strong AI would be smarter than a human brain, what is not at all obvious.

by mbq 13y ago

Er, you're assuming that this strong AI would be smarter than a human brain, what is not at all obvious.

I thought the general assumption was that a strong AI would be able to rapidly self-improve. http://en.wikipedia.org/wiki/Technological_singularity http://en.wikipedia.org/wiki/Technological_singularity

ars 13y ago
Humans haven't managed it, so why so you assume a computer will?
- eli_gottlieb 13y ago
  Humans are built out of neural structures, which are black-box learners. Any AI built out of a white-box learner would at least hypothetically be able to self-improve up to some fundamental physical-mathematical limit imposed by its supply of raw compute-power. The interesting question is: where is that upper limit on intelligence-per-FLOPS? EDIT: Fixed it to "FLOPS". We already have AI algorithms that Solve The Problem if given unlimited "thinking time" between steps of interaction with their environment (up to the point of needing a Halting Oracle), and we have extensions of those that are asymptotically speed-optimal modulo some additive or multiplicative constants (which are larger than our current universe). I strongly suspect that scaling down these Idealized AI Agents to real, useful programs isn't going to just involve feeding raw instances of the Ideal Agents small enough environments to be tractable, but actually finding or deriving some algorithm that can scale continuously up and down with available computing power per unit of real time, where the time units are the actual real-time length of an interaction cycle.
  - seiji 13y ago
    Check out Accelerando. Basically, mumble mumble computronium mumble mumble.
    - eli_gottlieb 13y ago
      mumble mumble stop giving them ideas mumble mumble
  - ars 13y ago
    > are black-box learners. Any AI built out of a white-box learner To have a white-box learner would imply that humans were smart enough to build it. Yet it's unlikely that humans can build something significantly smarter than a human, so this AI would have the same limitation: It's unable (not smart enough) to build a smarter AI.
    - eli_gottlieb 13y ago
      Intelligence is not magic, and in some way almost definitely scales with computing power. For instance, if you have infinite computing power, you just have to dovetail over every possible Turing Machine until one of them "wakes up" and does AI-y things. Yes, there are in fact real approaches based on a more sophisticated version of such a Universal Search. In fact, we routinely make use of machine-learning algorithms that are "smarter than a human" in particular narrow domains. Are they smarter than a human at general intelligence, of the kind necessary to start looking more like a scifi "AI" than a cheap trick? No, but they are an indication that such a thing could be possible. Overall, I would say that with Machine Learning maturing and Universal AI being a formal science at all, we are only just now beginning to find out where the actual tradeoffs lie in the design and construction of intelligent agents.
      - ars 13y ago
        > Intelligence is not magic, and in some way almost definitely scales with computing power. The speed of intelligence scales. But not the quality of it. The quality of intelligence is something very hard to quantify. But as simply as I can state it, it's the quality of coming up with an idea that is not just an extension of an existing one, but an idea that came seemingly from nowhere. To someone else the idea appears magic, and no amount of thinking by that other person can lead him in a path that will end up there. > but they are an indication that such a thing could be possible. Real genius is not just "more" intelligence, it's intelligence of a different type that a non-genius simply can not do, no matter how they try. Analogously, just because a computer can do something fast does not give it that hard to define quality of true intelligence. To make (rather than randomly search for) a computer with real intelligence we need to understand what it is, and where it comes from, and we don't know either one of those things. Our best (smartest) computers are all variations of "search randomly" (for example chess engines). Those can do some amazing things, but their limitations will not be overcome by doing more of the same. > infinite computing power .. every possible Talking about infinity is a total red herring and you know that - especially because if your method of increasing intelligence is "search randomly" then an AI will not be able to make a better AI - all it can do is continue the random search humans already started.
- ceejayoz 13y ago
  We're talking about something purely hypothetical at this point, so who knows if they're right... but I'd assume that an AI would benefit from the vastly faster processing speed of a computer, the ability to have a processing facility larger than a blob of fat that fits in our cranium, a more accurate memory, etc... plus the ability to iterate on its own operating system.
hharrison 13y ago
Eventually, but strong AI just means intelligence matching or exceeding human intelligence. We already have billions of entities with human intelligence, and it is taking us a long time to produce something smarter than ourselves. If the first strong AI is just a little smarter than us, and if it chooses to put its energy toward the creation of more AIs, then maybe it will eventually produce something smarter than itself. But it's not so simple that we just get to the singularity straight away.
- seiji 13y ago
  The problem with people is multiplicative intelligence parity. (Like that phrase? Feel free to use it as your own.) You can find one smart person. But how do you find another smart person for them to work with? With AI-level smarts, you reduce the coordination problem to zero since it can spawn multiple copies of its own brain state (maybe?) and work concurrently and intelligently on the same problem. Just ignore the problem of killing off divergent brain states once the task is complete (nobody cares about the life of a kage bunshin). I sure could get a lot more done if I had five more of me (unpaid, of course) to tackle a problem all at once.

PeterisP 13y ago

From the evidence of how fast new botnets spread, and the current prices of 0-day vulnerabilities we can assume that any strong-AI can easily get access to immense computing power (percentage points of globally installed CPU's). The difference is that it can use them not for sending low-return spam, but for multiplying its power - even if it doesn't give a qualitative difference, there should be a quantitative difference (how many different tasks it can do at once) when a strong AI gains many orders of magnitude more computing power than any laboratory/datacenter where it was initially created.

VLM 13y ago
Latency and bandwidth are going to suck and that might be important. There's an assumption in most AI discussions that AI "must" be embarrassingly parallelizable. For example, much as my brain is parallelized, surely an AI would have to be the same way, much as all heavier than air human flight required was a faithful reproduction of bird anatomy (LOL). There's also a certain assumption that intelligence implies a level of self awareness that most humans don't have therefore AI would be highly self aware... There's also a whole industry of sophistry devoted to proving no humans are intelligent and IQ as a concept or any other numerical measure of intelligence does not exist and there is no way to compare intelligence, and right or wrong those folks will surely make life difficult for people improving / testing / upgrading an AI, either because they're right or they'll be making political protests.
chongli 13y ago
strong-AI can easily get access to immense computing power But is it the right kind of computing power? The human brain is a very different sort of machine, architecturally speaking, from any computer currently in existence. Current computers have a very difficult time dealing with massive, parallel, random data access. Perhaps strong AI requires billions of threads randomly accessing a database which is terabytes in size? That is going to make the memory bus very, very sad.
mbq 13y ago
To begin with, it may not want to harvest more computational power. Even if, it may be initially too stupid to do this. Even if, the internet speed and small reliability of those nodes may make them unusable to this purpose.

Houshalter 13y ago

Strong AI generally means smarter-than human.

mbq 13y ago
What does not necessary mean more capable in case of making AI systems.
- Houshalter 13y ago
  No but it does mean it's possible to automate the work of further improving AI. It will also very likely be much smarter than human. Being exactly the same intelligence as a human is a very narrow target to hit.