4 ms·

Would you imagine that a computer smarter than the smartest human and having the understanding of computer software that probably only a computer would have cou

by crusso 9y ago

Would you imagine that a computer smarter than the smartest human and having the understanding of computer software that probably only a computer would have couldn't conceal what it's doing as it creates trojans and viruses that can give it control over every piece of connected equipment everywhere?

Retra 9y ago
All of that is moot when you can just turn off the power.
- lgas 9y ago
  If it was successfully concealing its actions from you, what reason would you have to turn off the power?
  - Retra 9y ago
    If it were successfully concealing its actions, what wrong would it be doing? If all the bad things someone does has no effect on anyone else, then who cares if they do them?
    - crusso 9y ago
      Because it could be concealing its actions until it controls enough infrastructure or military power to launch an attack or prevent people from being able to shut it down. What we don't know can hurt us.
      - Retra 9y ago
        It is not 'concealing its actions' if it is taking control of infrastructure and military power. Just like I'm not 'concealing my actions' by putting a blindfold over your eyes. My action is "putting a blindfold over your eyes", which must be very apparent for the entire action to have any value. Now if I hide that action by putting on an invisible blindfold, then you've got nothing to complain about anyway.
        crusso 9y ago
        Why would it be so obvious if it understands enough about strategy to use subterfuge? When the US government unleashed Stuxnet, did it announce what it was going to do?
- Klockan 9y ago
  That wont help, we know how to turn off humans from across the world with the click of a button yet we still haven't managed to create peace on earth.
- TeMPOraL 9y ago
  Batteries exist. And so do small-scale power generators like PV cells, windmills and, well, diesel generators.
- pjc50 9y ago
  Can you? The AI, when it becomes popular, is going to be a business-critical system. Potentially even intertwined with life-critical systems like self-driving cars and self-flying planes. If it isn't, then in "hidden malevolent hacker" phase it can just install trojans to make those systems dependent on it. Petrol production is hazardous to our future. Can you "just turn it off"? Yes; shut down a fairly small number refineries and the system will run dry within a week. CO2 emissions problem solved! It's just that, you know, people were depending on being able to get food delivered by truck. Obviously the first malevolent AI is going to hide itself behind an offshore opaque-ownership company and hire a good set of PR and lobbying firms. Can you turn off Skynet when it's reported as "critical to national security" in all the news sources?
  - Retra 9y ago
    So you're supposing that we have honed and selected an AI so wonderfully adept at valuing human needs that we make most human life dependent on it, yet it will spontaneously come to disregard the value of human life? And despite the fact that we would naturally require it to be sufficiently transparent about it's valuations in order to actually ensure we get the results we want, it will also develop this animosity in secret? That seems a bit contradictory to me. And if not, highly contrived and unlikely.
    - crusso 9y ago
      "And despite the fact that we would naturally require it to be sufficiently transparent about it's valuations" That's not necessarily true at all. Let's say we come up with a way to model neurons in silicon like a very complex neural net to the point of emergent consciousness. We might not be able to look into memory location 0xFFA79348 and get the "human value" int to see if it is high enough. It could very well be that we don't really understand the weighting in the net or the implications of those weights because it's too complicated. Even current neural nets of non-trivial complexity can be impenetrable to evaluate outside of the net itself.
      - Retra 9y ago

by crusso 9y ago

Retra 9y ago

All of that is moot when you can just turn off the power.

lgas 9y ago
If it was successfully concealing its actions from you, what reason would you have to turn off the power?
- Retra 9y ago
  If it were successfully concealing its actions, what wrong would it be doing? If all the bad things someone does has no effect on anyone else, then who cares if they do them?
  - crusso 9y ago
    Because it could be concealing its actions until it controls enough infrastructure or military power to launch an attack or prevent people from being able to shut it down. What we don't know can hurt us.
    - Retra 9y ago
      It is not 'concealing its actions' if it is taking control of infrastructure and military power. Just like I'm not 'concealing my actions' by putting a blindfold over your eyes. My action is "putting a blindfold over your eyes", which must be very apparent for the entire action to have any value. Now if I hide that action by putting on an invisible blindfold, then you've got nothing to complain about anyway.
      - crusso 9y ago
        Why would it be so obvious if it understands enough about strategy to use subterfuge? When the US government unleashed Stuxnet, did it announce what it was going to do?
Klockan 9y ago
That wont help, we know how to turn off humans from across the world with the click of a button yet we still haven't managed to create peace on earth.
TeMPOraL 9y ago
Batteries exist. And so do small-scale power generators like PV cells, windmills and, well, diesel generators.
pjc50 9y ago
Can you? The AI, when it becomes popular, is going to be a business-critical system. Potentially even intertwined with life-critical systems like self-driving cars and self-flying planes. If it isn't, then in "hidden malevolent hacker" phase it can just install trojans to make those systems dependent on it. Petrol production is hazardous to our future. Can you "just turn it off"? Yes; shut down a fairly small number refineries and the system will run dry within a week. CO2 emissions problem solved! It's just that, you know, people were depending on being able to get food delivered by truck. Obviously the first malevolent AI is going to hide itself behind an offshore opaque-ownership company and hire a good set of PR and lobbying firms. Can you turn off Skynet when it's reported as "critical to national security" in all the news sources?
- Retra 9y ago
  So you're supposing that we have honed and selected an AI so wonderfully adept at valuing human needs that we make most human life dependent on it, yet it will spontaneously come to disregard the value of human life? And despite the fact that we would naturally require it to be sufficiently transparent about it's valuations in order to actually ensure we get the results we want, it will also develop this animosity in secret? That seems a bit contradictory to me. And if not, highly contrived and unlikely.
  - crusso 9y ago
    "And despite the fact that we would naturally require it to be sufficiently transparent about it's valuations" That's not necessarily true at all. Let's say we come up with a way to model neurons in silicon like a very complex neural net to the point of emergent consciousness. We might not be able to look into memory location 0xFFA79348 and get the "human value" int to see if it is high enough. It could very well be that we don't really understand the weighting in the net or the implications of those weights because it's too complicated. Even current neural nets of non-trivial complexity can be impenetrable to evaluate outside of the net itself.
    - Retra 9y ago