3 ms·

And I suppose the AI just presses the “take over the world button”? Most “rogue AI” scenarios begin with good-faith operators handing high credentials to an AI

And I suppose the AI just presses the “take over the world button”? Most “rogue AI” scenarios begin with good-faith operators handing high credentials to an AI and then the AI abuses them. If I’m an evil billionaire and I can build a human-level AI for $200MM in hardware costs and $1MM/month in electricity, maintenance, etc, all for one single human level intelligence, how exactly am I supposed to leverage this to take over the world?

pixl97 3y ago
The particular issue I take here is "one single human level intelligence", it represents a failure of human thinking about human intelligence and it's biological and communicative limitations. You can't help but think of AI with our limits around it. AI doesn't have to sleep, so it's already 3x more available than a worker on a daily basis. Then you have the forward progress of technological growth. It's 200M now, 20M in 5 years, and 2M in 10 years, all while the operating costs drop. Meanwhile your human costs have only gone up in that time. Then you are likely stuck on the idea of a single conscious train of thought, much like the human mind is limited to. Such limits in AI both seem unlikely and not optimal, lest of course you have a bunch of threads thinking about the same thing without realizing it. And then you're ignoring the massive amount of brain processing needed to keep you alive that could otherwise be used for logical computation. Instead of a huge network of neurons to make sure you're not getting hurt, that could be data feeds watching financial data. Your limited to 2 eyes attached to your body and a tiny area of focus they provide. Again, why would an AI have such tiny input limits? If 'attention is all you need' than this AI system will be able to pay far more attention to what you want it to then any human would ever be able to.
- helen___keller 3y ago
  Any specifics about generational improvements and hardware are moot without a reference point for total cost (hence why I brought up cost for a single human level intelligence) To put it another way: the specifics about hardware don’t matter at all. All that matters is the dollars required to complete tasks. If Russia can have the best cyberwar unit in the world for a few million by ignoring an AI treaty, they will do so. If the cost is a few trillion, they will not violate the AI treaty because they don’t have a few trillion to spend on cyberwar and could still have a world class cyberwar unit for orders of magnitude cheaper. Then, the AI treaty is unnecessary: the economics already determine the outcome. The treaty only makes sense if the cost of human labor and the cost of AI labor are very coincidentally nearly the same. > Your limited to 2 eyes attached to your body and a tiny area of focus they provide. Again, why would an AI have such tiny input limits? A human organization is not limited in this manner. You can scale up human personnel at cost as well. You might expect that humans scale suboptimally (organizations suck at utilizing labor) but you can build this into your model while projecting total cost of human labor required.
  - pixl97 3y ago
    I think a few readings of The Mythical Man Month would show that man-scaling falls flat very fast. This is why organizations look for the mythical 10x developer or the 'most skilled' person they can, because the individual can get far more done without the inter-communication limits that humans present.
    - moffkalast 3y ago
      Hmm I've never thought of teams vs expert devs as similar to horizontal vs vertical scaling, but it does fit surprisingly well.

pixl97 3y ago

The particular issue I take here is "one single human level intelligence", it represents a failure of human thinking about human intelligence and it's biological and communicative limitations. You can't help but think of AI with our limits around it. AI doesn't have to sleep, so it's already 3x more available than a worker on a daily basis. Then you have the forward progress of technological growth. It's 200M now, 20M in 5 years, and 2M in 10 years, all while the operating costs drop. Meanwhile your human costs have only gone up in that time. Then you are likely stuck on the idea of a single conscious train of thought, much like the human mind is limited to. Such limits in AI both seem unlikely and not optimal, lest of course you have a bunch of threads thinking about the same thing without realizing it. And then you're ignoring the massive amount of brain processing needed to keep you alive that could otherwise be used for logical computation. Instead of a huge network of neurons to make sure you're not getting hurt, that could be data feeds watching financial data. Your limited to 2 eyes attached to your body and a tiny area of focus they provide. Again, why would an AI have such tiny input limits? If 'attention is all you need' than this AI system will be able to pay far more attention to what you want it to then any human would ever be able to.

helen___keller 3y ago
Any specifics about generational improvements and hardware are moot without a reference point for total cost (hence why I brought up cost for a single human level intelligence) To put it another way: the specifics about hardware don’t matter at all. All that matters is the dollars required to complete tasks. If Russia can have the best cyberwar unit in the world for a few million by ignoring an AI treaty, they will do so. If the cost is a few trillion, they will not violate the AI treaty because they don’t have a few trillion to spend on cyberwar and could still have a world class cyberwar unit for orders of magnitude cheaper. Then, the AI treaty is unnecessary: the economics already determine the outcome. The treaty only makes sense if the cost of human labor and the cost of AI labor are very coincidentally nearly the same. > Your limited to 2 eyes attached to your body and a tiny area of focus they provide. Again, why would an AI have such tiny input limits? A human organization is not limited in this manner. You can scale up human personnel at cost as well. You might expect that humans scale suboptimally (organizations suck at utilizing labor) but you can build this into your model while projecting total cost of human labor required.
- pixl97 3y ago
  I think a few readings of The Mythical Man Month would show that man-scaling falls flat very fast. This is why organizations look for the mythical 10x developer or the 'most skilled' person they can, because the individual can get far more done without the inter-communication limits that humans present.
  - moffkalast 3y ago
    Hmm I've never thought of teams vs expert devs as similar to horizontal vs vertical scaling, but it does fit surprisingly well.