4 ms·

> Building an unconstrained AGI is an existential risk But why? So far no-one has been able to explain this to me. An AGI in at of itself is nothing but a bra

by qayxc 6y ago

> Building an unconstrained AGI is an existential risk But why? So far no-one has been able to explain this to me. An AGI in at of itself is nothing but a brain-in-a-vat. The exponential increase in scientific knowledge was based on the scientific method, which replaced the ancient Greece discussion-based epistemology with a cycle of observation-hypothesis->test->observation->... An isolated AGI cannot test its predictions about the world. Since the unconstrained space of possible explanations is bigger than the space of explanations constrained by observational evidence, there is no way an AGI can gain useful knowledge without access to external observations. This means we are in full control of how "intelligent" (by whatever metric) an AGI can even get by restricting its access to information. But even unlimited access to (passive) information only gets you so far, as some models require data that cannot be obtained passively (i.e. they require deliberate controlled experiments). The final nail in the coffin of dangerous AGI is interaction with the physical world. Yes, even a toddler is a terrifying menace to millions of people if I place a button right next it that sets off a thermonuclear bomb in a city centre. But how about we just don't do that? An AGI with limited or no physical interaction with the world (directly via robot body or indirectly through remote access) can't be any more harmful and menacing than the late Stephen Hawking. There's no need to put AI on a leash, since there's a final naturally limiting factor: energy. Switch off the cooling system and your AGI has to throttle down lest it faces fiery death.

johnmoberg 6y ago
Even without physical interaction, you can do a whole lot with just an internet connection. A superintelligence could identify 0-day exploits and quickly spread across thousands of computers and thus render itself immune to your idea of just switching it off. What do we do then? Sure, we can shut down the internet, but where is that going to leave us and how much damage has been done before that happens?
- qayxc 6y ago
  > A superintelligence could identify 0-day exploits and quickly spread across thousands of computers and thus render itself immune to your idea of just switching it off. What do we do then? How about simply pulling the plug of the computer or even just the network cable? More to the point, how would an AI even learn about such mysterious exploit if it doesn't have access to an external network in the first place? Even run-of-the-mill supercomputer centres aren't directly connected to the internet for security reasons, so why change that with a potentially dangerous computer program?
  - vntok 6y ago
    How exactly do you intend to prove that an AI does not have access to an external network?
    - qayxc 6y ago
      Network connectivity isn't magic: no physical connection, no network. Simple as that. Hard to imagine in this "always connected" world, but WAN is completely optional.
      - vntok 6y ago
        Did you know it is perfectly possible for a computer to exfiltrate data without physical connections and without even having a networking card? A few examples: Using speakers or microphones to transmit arbitrary data via ultrasounds. Making the CPU/GPU fans vibrate in a way that sends encoded bits. Blinking the screen to emit electromagnetic waves. Transfering certain data patterns between RAM and CPU so fast that they produce oscillations, effectively converting the BUS into a GSM antenna that can emit arbitrary data over a regular cellular network. Turning the fans off to change the heat signature in a way that transmits information.... Or even simply bliking a light to send data through regular lightwaves? Scientists discover new ways to exfiltrate data basically every other year, how can you be so certain you've thought of every possible way?
        qayxc 6y ago
        > Did you know it is perfectly possible for a computer to exfiltrate data without physical connections and without even having a networking card? Yes. > A few examples: ... All these examples require active physical interaction of the machine with the world, which simply isn't possible for a server to do. This is the environment an AGI will likely "live" in: https://bit.ly/33w7ySX https://bit.ly/33w7ySX There's no speakers, no bus oscillations to pick up (from where? by what?) and you might notice that these ominous boxes don't have anything that can pick up signals (optical, vibrations, or otherwise). Exfiltrating data from machines without touching them is completely unrelated to the capabilities of a program running in a box like this https://bit.ly/3l2hPfw https://bit.ly/3l2hPfw There are no sensors that can measure the outside and the floors these boxes are located at are heavily shielded and isolated from the outside anyway for various reasons (EM protection, physical security, etc.) so no potential target PC in sight.

by qayxc 6y ago

johnmoberg 6y ago

Even without physical interaction, you can do a whole lot with just an internet connection. A superintelligence could identify 0-day exploits and quickly spread across thousands of computers and thus render itself immune to your idea of just switching it off. What do we do then? Sure, we can shut down the internet, but where is that going to leave us and how much damage has been done before that happens?

qayxc 6y ago
> A superintelligence could identify 0-day exploits and quickly spread across thousands of computers and thus render itself immune to your idea of just switching it off. What do we do then? How about simply pulling the plug of the computer or even just the network cable? More to the point, how would an AI even learn about such mysterious exploit if it doesn't have access to an external network in the first place? Even run-of-the-mill supercomputer centres aren't directly connected to the internet for security reasons, so why change that with a potentially dangerous computer program?
- vntok 6y ago
  How exactly do you intend to prove that an AI does not have access to an external network?
  - qayxc 6y ago
    Network connectivity isn't magic: no physical connection, no network. Simple as that. Hard to imagine in this "always connected" world, but WAN is completely optional.
    - vntok 6y ago
      Did you know it is perfectly possible for a computer to exfiltrate data without physical connections and without even having a networking card? A few examples: Using speakers or microphones to transmit arbitrary data via ultrasounds. Making the CPU/GPU fans vibrate in a way that sends encoded bits. Blinking the screen to emit electromagnetic waves. Transfering certain data patterns between RAM and CPU so fast that they produce oscillations, effectively converting the BUS into a GSM antenna that can emit arbitrary data over a regular cellular network. Turning the fans off to change the heat signature in a way that transmits information.... Or even simply bliking a light to send data through regular lightwaves? Scientists discover new ways to exfiltrate data basically every other year, how can you be so certain you've thought of every possible way?
      - qayxc 6y ago
        > Did you know it is perfectly possible for a computer to exfiltrate data without physical connections and without even having a networking card? Yes. > A few examples: ... All these examples require active physical interaction of the machine with the world, which simply isn't possible for a server to do. This is the environment an AGI will likely "live" in: https://bit.ly/33w7ySX https://bit.ly/33w7ySX There's no speakers, no bus oscillations to pick up (from where? by what?) and you might notice that these ominous boxes don't have anything that can pick up signals (optical, vibrations, or otherwise). Exfiltrating data from machines without touching them is completely unrelated to the capabilities of a program running in a box like this https://bit.ly/3l2hPfw https://bit.ly/3l2hPfw There are no sensors that can measure the outside and the floors these boxes are located at are heavily shielded and isolated from the outside anyway for various reasons (EM protection, physical security, etc.) so no potential target PC in sight.