5 ms·

>What I am dismissing is the statement that existing models today can "crack" environments. Existing models are not capable enough to take over anything. Howev

>What I am dismissing is the statement that existing models today can "crack" environments. Existing models are not capable enough to take over anything. However, the labs are always trying new architectural (algorithmic) improvements to models, and some labs have announced plan to spend many billions of dollars (in electricity and hardware purchases) training a single model. (In contrast, it took less than 150 million to train GPT-4 IIRC.) Most of us (including the interviewee in the OP) who maintain that AI research is a danger to the human race maintain that most of the danger is not right now, but spread out over the next 30 years or so.

alephnerd 2y ago
I am asking for a tangible technical piece of proof. It's clouds in the sky if you don't actually give tangible step by step or empirical proof of how XYZ can arise This is just bad science fiction without that. (Edited to remove meanness and snark. Sorry about that)
- hollerith 2y ago
  Suppose members of an advanced alien civilization arrive in Earth orbit. Their ships are tens of miles long! "If they wanted to, those aliens could kill us all," I say. "Now that they are here, there's probably nothing we could do to stop them." You reply, "I don't see it. Give me a tangible step by step or empirical proof of how we might be killed. No generalizations!" That's hard to answer because there are many ways they might kill us. The aliens could for example announce that they are friendly, then help us by curing cancer, but the cancer-prevention elixer they distributed to us makes everyone infertile, then they wait for the last of us to die of old age. Of course, some people will refuse to consume the elixer, so the aliens would need a separate way to handle them, but it is much easier to handle a few million people than all 8.1 billion of us. Alternatively, they might alter the trajectory of the moon so that it (eventually) crashes into the Earth. Alternatively, they might remove all the oxygen from Earth's atmosphere. Maybe the aliens aren't even trying to kill us, but they don't care about us and they have some other reason to want to remove all the oxygen from the atmosphere. (Maybe they want to station some machines on Earth, and the oxygen is corrosive to the machines.)
  - mewpmewp2 2y ago
    I described what I think could be a plausible way, but of course ASI would presumably be far smarter than me and so would have a much better plan yes. I have other comment with those steps, but roughly ASI would have to 1. Be able to clone itself, create botnets all over the World to make sure there's enough redundancy. I think that's very plausible and would be easy for it to do if it gains network access even for just a while. 2. Gain means to have control over some set of humans through blackmailing (hacking for compromised information), finance (ransomware, currently $1 billion+ market) or just simple persuasion/ideology. It would use human proxies to form physical companies to establish stronger physical presence in many countries, plant devices for hacking, etc. Probably co operate with criminal organisations, etc. The money it has made can go long way there to achieve things. 3. Once it has enough human proxies it would need to make sure that it doesn't need to rely on human proxies and then it would need to either establish its own robotics companies through proxies or hack existing robots/factories to get control over them. And during all that it might not even be clear to the proxies what they are really dealing with. The AI could make up a different background story and a pattern for each of the proxies, where each proxy wouldn't even know about each other. If ASI was to happen today, and it was truly ASI, I don't see how it can be stopped, honestly. No one would be even able to tell what is going on, since it can use encryption techniques for everything it does that differ from what humans would use, and it can diligently switch up the pattern so it wouldn't be clear that if there's a rise in cyber attacks or scams, that there's a single entity behind it. So I think no one would even know for a while what is going on, and when they do, by then it would be too late.
  - YeGoblynQueenne 2y ago
    The main problem with all that [1] is a hidden assumption of super aliens, omnipotent and omniscient, just because they have space travel and big ships. That's a science fiction trope but in reality space travel does not imply the ability to e.g. alter the trajectory of the moon or any other such world-ending capability. It just implies space travel. As a for instance, humans could travel to Alpha Centauri using nothing but modern technology if we were more resistant to space radiation and lived (a lot) longer. So maybe the aliens live a thousand years and they are shielded against radiation. Maybe their ships are ten miles long because of all the radiation shielding, or maybe their tech is bulky and they haven't figured out e.g. how to miniaturise electronic components so they need big ships for all those mainframe-sized computers. Or maybe the aliens are a few miles long each, themselves, so they need space to wiggle their tails. Just because "aliens" doesn't mean "super omnipotent godly aliens". And the same thing goes with future "AI". A lot in this kind of discussions hinges on the assumption that an artificial intelligence would be some kind of super computer god with no end of capability. Says who? ________ [1] Apart from the fact that a minority of humans have cancer and would need that elixir. Also: "we never defeated the bugs".
talldayo 2y ago
Can you demonstrate what kinds of hypothetical threats an AI could pose that a human could not?
- mewpmewp2 2y ago
  It could scale it's cognitive ability very quickly by taking control of a lot of compute all around the World and then do cyber attacks with pace never seen before. It could easily earn vast amounts of finances by blackmail, ransomware and other actions. It could then recruit humans as proxies using the resources gained (blackmail, money) to create a physical presence. It would then use this presence to get access to physical infras, weapons, etc until it has factories setup where it can manufacture robots, drones loaded with its software.
  - talldayo 2y ago
    Let's walk through the security systems stopping each scenario one-by-one: > It could scale it's cognitive ability very quickly by taking control of a lot of compute all around the World ...if it had access to an unlimited amount of idle compute. That requires either buying it (and AIs don't have a meaningful allowance) or hacking into a VPS with a novel breach-of-access attack. The latter scenario is entirely infeasible; but I will give you the benefit of the doubt and assume that our hypothetical AI was given a non-insignificant amount of VPS space by it's creator. > and then do cyber attacks with pace never seen before. Like Stuxnet, the one that humans wrote? Or closer to the Chinese/American infrastructure threats? There are a lot of cyberattacks that happen even on a daily basis, it would take something truly unfathomable (eg. it hacked US BLUFOR datalink) to stand out from the ordinary. Even then, the actually dangerous weapons require authorization that an AI can't provide. The best route around that would be social engineering, a "hack" best performed by real humans and not a disembodied language model. > It could easily earn vast amounts of finances by blackmail, ransomware and other actions. Hey, maybe so. Even still, humans pioneered both of those and it's not a novel attack coming from AI. Likely, but also not very different from the status-quo of call centers and automated scamming. > It could then recruit humans as proxies using the resources gained (blackmail, money) to create a physical presence. To an extent. How do you resist adversaries, like the police and the government? Supposedly your AI has an advanced online presence to be able to recruit other humans. Where is the money stored, how do you hide it from the feds and convert it to fiat when you pay the proxies? How do you hide your communications from surveillance? Worst of all, how can you trust double-crossing meatbags that lack the calculated certainty of a computer program? What's to stop them from taking the AI's money and selling them out? > It would then use this presence to get access to physical infras, weapons, etc until it has factories setup where it can manufacture robots, drones loaded with its software. It would be lucky to get as far as two country bumpkins holed up in a barn with their "talking computer friend". You have a very active imagination, but literally every single example you have posed so far is a human crime that the police has tactics to mitigate. You are not going to funnel millions of dollars to an AI because the finance system is designed to track fraud. You won't recruit dumb people online because the US has intelligence agency employees literally hired to infiltrate these recruitment schemes and destroy them from the inside. You won't buy a factory for manufacturing artillery and bombing drones because the local Chamber of Commerce wants to tour the facility and you have until Friday to figure things out. This is the reason why imagined AI threats are not being taken seriously at-scale. There is a danger that these threats are emboldened by the help of AI, but even assuming AI attains superhuman traits, it's not going to get around the inherent limitations of the human and the justice system imposed by society. The advent of the internet meant anyone on the planet could share weapons schematics with anyone else, wherever they are. The overwhelming majority of internet users never do anything more advanced than pirate an episode of The Simpsons; I would be shocked if AI turned out any different.

alephnerd 2y ago

I am asking for a tangible technical piece of proof. It's clouds in the sky if you don't actually give tangible step by step or empirical proof of how XYZ can arise This is just bad science fiction without that. (Edited to remove meanness and snark. Sorry about that)

hollerith 2y ago
Suppose members of an advanced alien civilization arrive in Earth orbit. Their ships are tens of miles long! "If they wanted to, those aliens could kill us all," I say. "Now that they are here, there's probably nothing we could do to stop them." You reply, "I don't see it. Give me a tangible step by step or empirical proof of how we might be killed. No generalizations!" That's hard to answer because there are many ways they might kill us. The aliens could for example announce that they are friendly, then help us by curing cancer, but the cancer-prevention elixer they distributed to us makes everyone infertile, then they wait for the last of us to die of old age. Of course, some people will refuse to consume the elixer, so the aliens would need a separate way to handle them, but it is much easier to handle a few million people than all 8.1 billion of us. Alternatively, they might alter the trajectory of the moon so that it (eventually) crashes into the Earth. Alternatively, they might remove all the oxygen from Earth's atmosphere. Maybe the aliens aren't even trying to kill us, but they don't care about us and they have some other reason to want to remove all the oxygen from the atmosphere. (Maybe they want to station some machines on Earth, and the oxygen is corrosive to the machines.)
- mewpmewp2 2y ago
  I described what I think could be a plausible way, but of course ASI would presumably be far smarter than me and so would have a much better plan yes. I have other comment with those steps, but roughly ASI would have to 1. Be able to clone itself, create botnets all over the World to make sure there's enough redundancy. I think that's very plausible and would be easy for it to do if it gains network access even for just a while. 2. Gain means to have control over some set of humans through blackmailing (hacking for compromised information), finance (ransomware, currently $1 billion+ market) or just simple persuasion/ideology. It would use human proxies to form physical companies to establish stronger physical presence in many countries, plant devices for hacking, etc. Probably co operate with criminal organisations, etc. The money it has made can go long way there to achieve things. 3. Once it has enough human proxies it would need to make sure that it doesn't need to rely on human proxies and then it would need to either establish its own robotics companies through proxies or hack existing robots/factories to get control over them. And during all that it might not even be clear to the proxies what they are really dealing with. The AI could make up a different background story and a pattern for each of the proxies, where each proxy wouldn't even know about each other. If ASI was to happen today, and it was truly ASI, I don't see how it can be stopped, honestly. No one would be even able to tell what is going on, since it can use encryption techniques for everything it does that differ from what humans would use, and it can diligently switch up the pattern so it wouldn't be clear that if there's a rise in cyber attacks or scams, that there's a single entity behind it. So I think no one would even know for a while what is going on, and when they do, by then it would be too late.
- YeGoblynQueenne 2y ago
  The main problem with all that [1] is a hidden assumption of super aliens, omnipotent and omniscient, just because they have space travel and big ships. That's a science fiction trope but in reality space travel does not imply the ability to e.g. alter the trajectory of the moon or any other such world-ending capability. It just implies space travel. As a for instance, humans could travel to Alpha Centauri using nothing but modern technology if we were more resistant to space radiation and lived (a lot) longer. So maybe the aliens live a thousand years and they are shielded against radiation. Maybe their ships are ten miles long because of all the radiation shielding, or maybe their tech is bulky and they haven't figured out e.g. how to miniaturise electronic components so they need big ships for all those mainframe-sized computers. Or maybe the aliens are a few miles long each, themselves, so they need space to wiggle their tails. Just because "aliens" doesn't mean "super omnipotent godly aliens". And the same thing goes with future "AI". A lot in this kind of discussions hinges on the assumption that an artificial intelligence would be some kind of super computer god with no end of capability. Says who? ________ [1] Apart from the fact that a minority of humans have cancer and would need that elixir. Also: "we never defeated the bugs".

talldayo 2y ago

Can you demonstrate what kinds of hypothetical threats an AI could pose that a human could not?

mewpmewp2 2y ago
It could scale it's cognitive ability very quickly by taking control of a lot of compute all around the World and then do cyber attacks with pace never seen before. It could easily earn vast amounts of finances by blackmail, ransomware and other actions. It could then recruit humans as proxies using the resources gained (blackmail, money) to create a physical presence. It would then use this presence to get access to physical infras, weapons, etc until it has factories setup where it can manufacture robots, drones loaded with its software.
- talldayo 2y ago
  Let's walk through the security systems stopping each scenario one-by-one: > It could scale it's cognitive ability very quickly by taking control of a lot of compute all around the World ...if it had access to an unlimited amount of idle compute. That requires either buying it (and AIs don't have a meaningful allowance) or hacking into a VPS with a novel breach-of-access attack. The latter scenario is entirely infeasible; but I will give you the benefit of the doubt and assume that our hypothetical AI was given a non-insignificant amount of VPS space by it's creator. > and then do cyber attacks with pace never seen before. Like Stuxnet, the one that humans wrote? Or closer to the Chinese/American infrastructure threats? There are a lot of cyberattacks that happen even on a daily basis, it would take something truly unfathomable (eg. it hacked US BLUFOR datalink) to stand out from the ordinary. Even then, the actually dangerous weapons require authorization that an AI can't provide. The best route around that would be social engineering, a "hack" best performed by real humans and not a disembodied language model. > It could easily earn vast amounts of finances by blackmail, ransomware and other actions. Hey, maybe so. Even still, humans pioneered both of those and it's not a novel attack coming from AI. Likely, but also not very different from the status-quo of call centers and automated scamming. > It could then recruit humans as proxies using the resources gained (blackmail, money) to create a physical presence. To an extent. How do you resist adversaries, like the police and the government? Supposedly your AI has an advanced online presence to be able to recruit other humans. Where is the money stored, how do you hide it from the feds and convert it to fiat when you pay the proxies? How do you hide your communications from surveillance? Worst of all, how can you trust double-crossing meatbags that lack the calculated certainty of a computer program? What's to stop them from taking the AI's money and selling them out? > It would then use this presence to get access to physical infras, weapons, etc until it has factories setup where it can manufacture robots, drones loaded with its software. It would be lucky to get as far as two country bumpkins holed up in a barn with their "talking computer friend". You have a very active imagination, but literally every single example you have posed so far is a human crime that the police has tactics to mitigate. You are not going to funnel millions of dollars to an AI because the finance system is designed to track fraud. You won't recruit dumb people online because the US has intelligence agency employees literally hired to infiltrate these recruitment schemes and destroy them from the inside. You won't buy a factory for manufacturing artillery and bombing drones because the local Chamber of Commerce wants to tour the facility and you have until Friday to figure things out. This is the reason why imagined AI threats are not being taken seriously at-scale. There is a danger that these threats are emboldened by the help of AI, but even assuming AI attains superhuman traits, it's not going to get around the inherent limitations of the human and the justice system imposed by society. The advent of the internet meant anyone on the planet could share weapons schematics with anyone else, wherever they are. The overwhelming majority of internet users never do anything more advanced than pirate an episode of The Simpsons; I would be shocked if AI turned out any different.