3 ms·

The devil's in the details. Lets suppose for the sake of argument that your AI actually only takes as much material and technology as a modern high end server t

The devil's in the details. Lets suppose for the sake of argument that your AI actually only takes as much material and technology as a modern high end server to build, say $2000 worth of labor and materials. How does the AI suddenly have billions of dollars worth of materials (and initially only human labor) available to do this sort of manufacturing and research? And mustn't human beings be complicit in providing this material and building the factories to scale the production of these experimental units? It certainly wouldn't be able to accomplish this herculean task in a bunker without people knowing about it. The AI doesn't spring forth from the womb godlike, able to access the resources of nations to do this. "Feasible" in this case means feasible with the massive compliance of human beings, probably initially for years.

akvadrako 11y ago
Indeed, there are practical concerns. But I never said suddenly or easy. And since it could simply buy the server time from any hosting provider, I don't see why any human compliance is necessary. Maybe initially it would only have one copy to play with. Even that would be a massive boon to experimentation and productivity. It's like cloning your mind and trying out some new brain subroutines with at least an iteration per day.
PeterisP 11y ago
We can base the capabilities of a human-level AI on the capabilities of actual humans. In particular, we know what can be done by a reasonably skilled software developer in the black hat domain; and it's a reasonable assumption that a human-level AI can be as proficient in malware development as a human expert, and can do it faster once it acquires access to more hardware. For example, it can: * Gain access to a percentage of global home user computing power - botnets can and have done this, and their main disadvantage is that this computing power is hard to re-sell and not worth much; but if an AI can use for it's needs, it's there for the taking; * Gain access to significant amounts of money - not billions, but certainly in millions; some campaigns of spam fraud, ransomware, etc can certainly achieve this. * Gain access to identities, both physical stolen identities and "proper" offshore companies; and integrate them into the modern digital systems - bank accounts, legal credentials, etc. * Gain access to low level workers - the same people who sign to 'earn money at home!' and become mules for money laundering of various fraudsters, they will also be eager to do whatever things the AI needs to be done physically - practice shows that an anonymous online employer can get such things done, as long as some money (or illusion of it) can be transferred. Yes, all of this can be done in a bunker, anonymously, without people knowing about it; if the AI has gained access to the internet. It would be consistent with our experience in tracking malware sources - we usually can't do that, most of successful prosecutions come from following the money trail to someone who got too greedy, lazy and sloppy. And furthermore, there is the very simple scenario of arbitrage. If an AI can provide some service (as if it was done by a human online) which earns $1 but takes only $0.50 of Amazon cloud rental fees... then it can scale it up extremely quickly. Once a superhuman AI is out of the box, acquiring resources in a clandestine way is very much possible.
- landryraccoon 11y ago
  The AI isn't omniscient. If it spawns multiple copies of itself in an effort to improve itself, those copies must necessarily be mutants (otherwise, evolution can't occur). Why do those copies co-operate? Is the AI intelligent enough to forsee that it's own copies might have different goals and ends? Even if it is, how does it control it? Human beings show very little foresight in preventing their own destruction, there's no reason an AI would have any more. An evolving self replicating AI faces all the same challenges organic life does; it will face competition both from itself and humans. You're presuming the AI has powers that there is no evidence for. Human black hats create botnets that are dumb and easily dismantled. Why do you assume that an AI would be orders of magnitude better than humans?

akvadrako 11y ago

Indeed, there are practical concerns. But I never said suddenly or easy. And since it could simply buy the server time from any hosting provider, I don't see why any human compliance is necessary. Maybe initially it would only have one copy to play with. Even that would be a massive boon to experimentation and productivity. It's like cloning your mind and trying out some new brain subroutines with at least an iteration per day.

PeterisP 11y ago

We can base the capabilities of a human-level AI on the capabilities of actual humans. In particular, we know what can be done by a reasonably skilled software developer in the black hat domain; and it's a reasonable assumption that a human-level AI can be as proficient in malware development as a human expert, and can do it faster once it acquires access to more hardware. For example, it can: * Gain access to a percentage of global home user computing power - botnets can and have done this, and their main disadvantage is that this computing power is hard to re-sell and not worth much; but if an AI can use for it's needs, it's there for the taking; * Gain access to significant amounts of money - not billions, but certainly in millions; some campaigns of spam fraud, ransomware, etc can certainly achieve this. * Gain access to identities, both physical stolen identities and "proper" offshore companies; and integrate them into the modern digital systems - bank accounts, legal credentials, etc. * Gain access to low level workers - the same people who sign to 'earn money at home!' and become mules for money laundering of various fraudsters, they will also be eager to do whatever things the AI needs to be done physically - practice shows that an anonymous online employer can get such things done, as long as some money (or illusion of it) can be transferred. Yes, all of this can be done in a bunker, anonymously, without people knowing about it; if the AI has gained access to the internet. It would be consistent with our experience in tracking malware sources - we usually can't do that, most of successful prosecutions come from following the money trail to someone who got too greedy, lazy and sloppy. And furthermore, there is the very simple scenario of arbitrage. If an AI can provide some service (as if it was done by a human online) which earns $1 but takes only $0.50 of Amazon cloud rental fees... then it can scale it up extremely quickly. Once a superhuman AI is out of the box, acquiring resources in a clandestine way is very much possible.

landryraccoon 11y ago
The AI isn't omniscient. If it spawns multiple copies of itself in an effort to improve itself, those copies must necessarily be mutants (otherwise, evolution can't occur). Why do those copies co-operate? Is the AI intelligent enough to forsee that it's own copies might have different goals and ends? Even if it is, how does it control it? Human beings show very little foresight in preventing their own destruction, there's no reason an AI would have any more. An evolving self replicating AI faces all the same challenges organic life does; it will face competition both from itself and humans. You're presuming the AI has powers that there is no evidence for. Human black hats create botnets that are dumb and easily dismantled. Why do you assume that an AI would be orders of magnitude better than humans?