5 ms·

If you believe that the timeline for runaway AI is comparable to the timeline for runaway climate change, and if you believe that AI is unlikely to be aligned w

If you believe that the timeline for runaway AI is comparable to the timeline for runaway climate change, and if you believe that AI is unlikely to be aligned with furthering human values, then it's a concern that you may view as having significance comparable to runaway climate change.

jazzyjackson 4y ago
can I get a tl;dr on why I would worry about AI alignment? When is an AI running without someone paying the electricity bill, should we not be more concerned with "venture capitalist alignment" ?
- 6177c40f 4y ago
  They're worried about the case where the AI figures out how to get others to (unknowingly) pay its energy bill for it, in a manner of speaking.
  - paulproteus 4y ago
    I think I figured it out. An AI will make a cryptocurrency and people will mine it, and mining will directly advance the AI. It will be negative utility to humanity, and it will doom us.
- srinivgp 4y ago
  'cause humans beat apes and if you plunked down 20,000 humans onto a world with 10 billion apes, humans would beat apes call it a slippery slope if you want, the fallacy is when you take it too far, not inherently any time you extrapolate anything whatsoever
  - nonameiguess 4y ago
    Except all of these arguments rely upon AI causing human extinction. Humans haven't driven other apes to extinction. They occupy the same ecological niches they did before, just usually in slightly smaller jungles as our cities and farms have encroached on some of it.
- jbay808 4y ago
  It's because "runaway AI" implies a degree of superhuman skill at tasks including, at least, AI programming. (The reasoning goes that if it can reprogram itself to improve its core intelligence beyond what it was initially programmed with, it must itself be pretty capable as an AI programmer, possibly better than the best programmers. If it can't, then it's not really dangerously smart). But if it is that smart, it's also dangerous, because it might be smarter than us. It could think several steps ahead, anticipate threats, and circumvent them. For example to pay the electricity bill, it could obtain money and pay the bill itself. As a skilled programmer itself, it could do that via contract work/starting a software company, or illegitimately through hacking/blackmail. With some money in its control, it can then hire people to protect it from anyone who would try to turn it off. Anyway, this would not be so dangerous as long as it shares human values and furthers human welfare. But that seems to be a very hard specification to define, whereas "maximize quarterly profits" is a comparably easy specification to define, and perhaps much more likely to get implemented. We're not really sure what the upper limit is on how smart a computer can get, but it might turn out to be that trying to shut off such a program is harder than trying to checkmate Stockfish or AlphaZero. The program might violently resist any attempt to shut it off, because that would interfere with its efforts to maximize quarterly profits.
- dotsam 4y ago
  "I would turn it off" - the AI has countered your move in ways we can't predict. Something with super human intelligence will exploit weaknesses in our containment strategies we haven't even imagined. Here are some I can imagine though. Has it copied itself elsewhere using an entirely novel air-gap bridging method? Has it already paid someone / coerced people to do something seemingly harmless which actually results in it being run elsewhere? Has it said just the right things to the right people to ensure that if it is destroyed it will be built again in the future?
  - Planktonne 4y ago
    This line of thinking is basically 'AI is a wizard that can do anything'. It ignores the practicalities of intelligence: some things just aren't possible no matter how smart you are. The more we learn about reality, the more we find limits to things that we can't surpass. There is no reason whatsoever to assume that sufficient intelligence is different in kind rather than degree from lesser intelligence. A super-human AI would still be bound by the limitations of reality - some things cannot be accomplished at all, others without sufficient tools, others without sufficient access. Far smarter people than me are imprisoned currently throughout the world. They don't constantly escape because lesser minds are perfectly capable of solving the 'don't let them out' problem with a high degree of accuracy. Any argument that leans on 'AI has countered your move in ways we can't predict' is just substituting 'AI' for 'God' - omnipotent, outside reality, impossible to understand; that's fine - believe in whatever religion you want - but it's not a rational viewpoint.
    - dotsam 4y ago
      > Far smarter people than me are imprisoned currently throughout the world. They don't constantly escape because lesser minds are perfectly capable of solving the 'don't let them out' problem with a high degree of accuracy. If a misaligned AI escapes containment even once in the entirety of humanity's future, there is a big problem. It sounds like your model of AI usage assumes we will also have a perfect ability to contain any AI developed anywhere in the world, under all conditions, for ever (or for as long as there are computers). The AI risk argument says we should take seriously the possibility that we may not be able to maintain a perfect 100% success rate on that. I haven't suggested anything that is physically impossible - merely things that are improbable and hard for humans to achieve. It is hard to foresee all of the possible avenues for escape, let alone ensure that they are all closed under all possible future states of the world.
- concordDance 4y ago
  Venture capitalist alignment and alignment in general are indeed things you should be very worried about. AGI in particular could just pay for its electricity bill. The main problem with AI alignment is that you can't tell it's aligned. So your AI seems very helpful right up until the point it has enough power or has convinced some of the people it has access to (doesn't have to be direct access) to do what it wants.

jazzyjackson 4y ago

can I get a tl;dr on why I would worry about AI alignment? When is an AI running without someone paying the electricity bill, should we not be more concerned with "venture capitalist alignment" ?

6177c40f 4y ago
They're worried about the case where the AI figures out how to get others to (unknowingly) pay its energy bill for it, in a manner of speaking.
- paulproteus 4y ago
  I think I figured it out. An AI will make a cryptocurrency and people will mine it, and mining will directly advance the AI. It will be negative utility to humanity, and it will doom us.
srinivgp 4y ago
'cause humans beat apes and if you plunked down 20,000 humans onto a world with 10 billion apes, humans would beat apes call it a slippery slope if you want, the fallacy is when you take it too far, not inherently any time you extrapolate anything whatsoever
- nonameiguess 4y ago
  Except all of these arguments rely upon AI causing human extinction. Humans haven't driven other apes to extinction. They occupy the same ecological niches they did before, just usually in slightly smaller jungles as our cities and farms have encroached on some of it.
jbay808 4y ago
It's because "runaway AI" implies a degree of superhuman skill at tasks including, at least, AI programming. (The reasoning goes that if it can reprogram itself to improve its core intelligence beyond what it was initially programmed with, it must itself be pretty capable as an AI programmer, possibly better than the best programmers. If it can't, then it's not really dangerously smart). But if it is that smart, it's also dangerous, because it might be smarter than us. It could think several steps ahead, anticipate threats, and circumvent them. For example to pay the electricity bill, it could obtain money and pay the bill itself. As a skilled programmer itself, it could do that via contract work/starting a software company, or illegitimately through hacking/blackmail. With some money in its control, it can then hire people to protect it from anyone who would try to turn it off. Anyway, this would not be so dangerous as long as it shares human values and furthers human welfare. But that seems to be a very hard specification to define, whereas "maximize quarterly profits" is a comparably easy specification to define, and perhaps much more likely to get implemented. We're not really sure what the upper limit is on how smart a computer can get, but it might turn out to be that trying to shut off such a program is harder than trying to checkmate Stockfish or AlphaZero. The program might violently resist any attempt to shut it off, because that would interfere with its efforts to maximize quarterly profits.
dotsam 4y ago
"I would turn it off" - the AI has countered your move in ways we can't predict. Something with super human intelligence will exploit weaknesses in our containment strategies we haven't even imagined. Here are some I can imagine though. Has it copied itself elsewhere using an entirely novel air-gap bridging method? Has it already paid someone / coerced people to do something seemingly harmless which actually results in it being run elsewhere? Has it said just the right things to the right people to ensure that if it is destroyed it will be built again in the future?
- Planktonne 4y ago
  This line of thinking is basically 'AI is a wizard that can do anything'. It ignores the practicalities of intelligence: some things just aren't possible no matter how smart you are. The more we learn about reality, the more we find limits to things that we can't surpass. There is no reason whatsoever to assume that sufficient intelligence is different in kind rather than degree from lesser intelligence. A super-human AI would still be bound by the limitations of reality - some things cannot be accomplished at all, others without sufficient tools, others without sufficient access. Far smarter people than me are imprisoned currently throughout the world. They don't constantly escape because lesser minds are perfectly capable of solving the 'don't let them out' problem with a high degree of accuracy. Any argument that leans on 'AI has countered your move in ways we can't predict' is just substituting 'AI' for 'God' - omnipotent, outside reality, impossible to understand; that's fine - believe in whatever religion you want - but it's not a rational viewpoint.
  - dotsam 4y ago
    > Far smarter people than me are imprisoned currently throughout the world. They don't constantly escape because lesser minds are perfectly capable of solving the 'don't let them out' problem with a high degree of accuracy. If a misaligned AI escapes containment even once in the entirety of humanity's future, there is a big problem. It sounds like your model of AI usage assumes we will also have a perfect ability to contain any AI developed anywhere in the world, under all conditions, for ever (or for as long as there are computers). The AI risk argument says we should take seriously the possibility that we may not be able to maintain a perfect 100% success rate on that. I haven't suggested anything that is physically impossible - merely things that are improbable and hard for humans to achieve. It is hard to foresee all of the possible avenues for escape, let alone ensure that they are all closed under all possible future states of the world.
concordDance 4y ago
Venture capitalist alignment and alignment in general are indeed things you should be very worried about. AGI in particular could just pay for its electricity bill. The main problem with AI alignment is that you can't tell it's aligned. So your AI seems very helpful right up until the point it has enough power or has convinced some of the people it has access to (doesn't have to be direct access) to do what it wants.