3 ms·

Earlier you asked, "How is Skynet supposed to escape if it's payload is in an airgapped network?" I think I gave a good answer to that one if "escape" means to

Earlier you asked, "How is Skynet supposed to escape if it's payload is in an airgapped network?" I think I gave a good answer to that one if "escape" means to gain access to the internet and to gain the ability to run any code it wants to run. (I.e., many people will give it those things.) I'm not entirely clear what you are asking now. Maybe you are asking me how an AI that is better at reality than people are would get access to a weapons system or to an important piece of infrastructure like an electrical generating station. Note that if the AI is better at science and technology than people are, it might be able to create its own weapons system or infrastructure. 120 years ago for example no one knew that uranium could be turned into a potent weapon or a potent source of heat and electrical power. Similarly, there might be something on Earth today that no one would realize might be dangerous if an AI got access to it and got the ability to process it however it wanted to. Also, do I misunderstand you or are you really claiming that things like electrical generating stations cannot be taken over by breaking into the computers that control them even if the "agent" doing the breaking-in is as good at breaking into computers as AIs currently are at chess, i.e., much much better than people are? Most people who dismiss the possibility that an AI might take over maintain that there is no way an AI is going to become better at reality than people are (or better at making complicated plans that can withstand determined human opposition) -- at least not any decade soon -- but you seem to have a different dismissal having to do with the difficulty of subverting computer-based systems.

alephnerd 2y ago
> I'm not entirely clear what you are asking now Give me an actual technically feasible and sourced attack path that cannot be remediated via existing methods. The issue is you are giving broad stroke handwavey assertions. The proof is in the pudding. Don't imagine the pudding, show me a step by step technically feasible recipe for the pudding. > Also, do I misunderstand you or are you really claiming that things like electrical generating stations cannot be taken over by breaking into the computers that control them even if the "agent" doing the breaking-in is as good at breaking into computers as AIs currently are at chess, i.e., much much better than people are I am not denying that electric grids can't be hacked (hell back in HS and College I used to try and take over IP CCTVs and other IoT devices using Shodan for lulz). What I am dismissing is the statement that existing models today can "crack" environments by innovating new attacks. Sure they can catch misconfigurations and brute force CVEs, but models are limited by their corpora, and much of the corpora used does not include every single pattern of code in humanity. This is why the companies testing computer generated code are using internally trained models because each code base is unique with different idiosyncrasies. And code patterns do differ based on where you studied and worked (it's been a common tool in attributing attacks to groups). > but you seem to have a different dismissal having to do with the difficulty of subverting computer-based systems Because I worked in this the security and ML/HPC industry for 10-15 years in every single facet - engineering, policy, product, and funding. Broad stroke narratives are nice, but if they cannot stand a basic stress test, then they don't make sense. I'm sure you lesswrongers can agree with that, but you guys seem to think you're smarter than the other domain experts who are actively thinking about these problems as well, and how to minimize them in their domain.
- hollerith 2y ago
  >What I am dismissing is the statement that existing models today can "crack" environments. Existing models are not capable enough to take over anything. However, the labs are always trying new architectural (algorithmic) improvements to models, and some labs have announced plan to spend many billions of dollars (in electricity and hardware purchases) training a single model. (In contrast, it took less than 150 million to train GPT-4 IIRC.) Most of us (including the interviewee in the OP) who maintain that AI research is a danger to the human race maintain that most of the danger is not right now, but spread out over the next 30 years or so.
  - alephnerd 2y ago
    I am asking for a tangible technical piece of proof. It's clouds in the sky if you don't actually give tangible step by step or empirical proof of how XYZ can arise This is just bad science fiction without that. (Edited to remove meanness and snark. Sorry about that)
    - hollerith 2y ago
      Suppose members of an advanced alien civilization arrive in Earth orbit. Their ships are tens of miles long! "If they wanted to, those aliens could kill us all," I say. "Now that they are here, there's probably nothing we could do to stop them." You reply, "I don't see it. Give me a tangible step by step or empirical proof of how we might be killed. No generalizations!" That's hard to answer because there are many ways they might kill us. The aliens could for example announce that they are friendly, then help us by curing cancer, but the cancer-prevention elixer they distributed to us makes everyone infertile, then they wait for the last of us to die of old age. Of course, some people will refuse to consume the elixer, so the aliens would need a separate way to handle them, but it is much easier to handle a few million people than all 8.1 billion of us. Alternatively, they might alter the trajectory of the moon so that it (eventually) crashes into the Earth. Alternatively, they might remove all the oxygen from Earth's atmosphere. Maybe the aliens aren't even trying to kill us, but they don't care about us and they have some other reason to want to remove all the oxygen from the atmosphere. (Maybe they want to station some machines on Earth, and the oxygen is corrosive to the machines.)
talldayo 2y ago
> Note that if the AI is better at science and technology than people are, it might be able to create its own weapons system or infrastructure. Oh man, do I have a bridge to sell you! For one, the internet already exists and is better at aggregating scientific and technological information than any of us, AI included. The stuff that LLMs feed you is cut-down, diluted information that is always better-provided by a human source. You know what that means... the internet is even more dangerous than AI!!! ...but wait. The internet doesn't do anything of it's own free will. It can be used for nefarious purposes, but that requires a nefarious actor to circumvent the safety mechanisms (which usually entails breaking the law). Hm. Guess that explains why nobody got very far building Skynet. > are you really claiming that things like electrical generating stations cannot be taken over by breaking into the computers that control them even I don't think anyone said that, but given that humans have also been capable of this for the past 30 years it feels like a moot-point. You are describing a security issue, not an AI one.

alephnerd 2y ago

> I'm not entirely clear what you are asking now Give me an actual technically feasible and sourced attack path that cannot be remediated via existing methods. The issue is you are giving broad stroke handwavey assertions. The proof is in the pudding. Don't imagine the pudding, show me a step by step technically feasible recipe for the pudding. > Also, do I misunderstand you or are you really claiming that things like electrical generating stations cannot be taken over by breaking into the computers that control them even if the "agent" doing the breaking-in is as good at breaking into computers as AIs currently are at chess, i.e., much much better than people are I am not denying that electric grids can't be hacked (hell back in HS and College I used to try and take over IP CCTVs and other IoT devices using Shodan for lulz). What I am dismissing is the statement that existing models today can "crack" environments by innovating new attacks. Sure they can catch misconfigurations and brute force CVEs, but models are limited by their corpora, and much of the corpora used does not include every single pattern of code in humanity. This is why the companies testing computer generated code are using internally trained models because each code base is unique with different idiosyncrasies. And code patterns do differ based on where you studied and worked (it's been a common tool in attributing attacks to groups). > but you seem to have a different dismissal having to do with the difficulty of subverting computer-based systems Because I worked in this the security and ML/HPC industry for 10-15 years in every single facet - engineering, policy, product, and funding. Broad stroke narratives are nice, but if they cannot stand a basic stress test, then they don't make sense. I'm sure you lesswrongers can agree with that, but you guys seem to think you're smarter than the other domain experts who are actively thinking about these problems as well, and how to minimize them in their domain.

hollerith 2y ago
>What I am dismissing is the statement that existing models today can "crack" environments. Existing models are not capable enough to take over anything. However, the labs are always trying new architectural (algorithmic) improvements to models, and some labs have announced plan to spend many billions of dollars (in electricity and hardware purchases) training a single model. (In contrast, it took less than 150 million to train GPT-4 IIRC.) Most of us (including the interviewee in the OP) who maintain that AI research is a danger to the human race maintain that most of the danger is not right now, but spread out over the next 30 years or so.
- alephnerd 2y ago
  I am asking for a tangible technical piece of proof. It's clouds in the sky if you don't actually give tangible step by step or empirical proof of how XYZ can arise This is just bad science fiction without that. (Edited to remove meanness and snark. Sorry about that)
  - hollerith 2y ago
    Suppose members of an advanced alien civilization arrive in Earth orbit. Their ships are tens of miles long! "If they wanted to, those aliens could kill us all," I say. "Now that they are here, there's probably nothing we could do to stop them." You reply, "I don't see it. Give me a tangible step by step or empirical proof of how we might be killed. No generalizations!" That's hard to answer because there are many ways they might kill us. The aliens could for example announce that they are friendly, then help us by curing cancer, but the cancer-prevention elixer they distributed to us makes everyone infertile, then they wait for the last of us to die of old age. Of course, some people will refuse to consume the elixer, so the aliens would need a separate way to handle them, but it is much easier to handle a few million people than all 8.1 billion of us. Alternatively, they might alter the trajectory of the moon so that it (eventually) crashes into the Earth. Alternatively, they might remove all the oxygen from Earth's atmosphere. Maybe the aliens aren't even trying to kill us, but they don't care about us and they have some other reason to want to remove all the oxygen from the atmosphere. (Maybe they want to station some machines on Earth, and the oxygen is corrosive to the machines.)

talldayo 2y ago

> Note that if the AI is better at science and technology than people are, it might be able to create its own weapons system or infrastructure. Oh man, do I have a bridge to sell you! For one, the internet already exists and is better at aggregating scientific and technological information than any of us, AI included. The stuff that LLMs feed you is cut-down, diluted information that is always better-provided by a human source. You know what that means... the internet is even more dangerous than AI!!! ...but wait. The internet doesn't do anything of it's own free will. It can be used for nefarious purposes, but that requires a nefarious actor to circumvent the safety mechanisms (which usually entails breaking the law). Hm. Guess that explains why nobody got very far building Skynet. > are you really claiming that things like electrical generating stations cannot be taken over by breaking into the computers that control them even I don't think anyone said that, but given that humans have also been capable of this for the past 30 years it feels like a moot-point. You are describing a security issue, not an AI one.