3 ms·

Interesting how much the people with boots on the ground and no PR departments to filter them actually do seem… pretty aware of how much we’re all going to die?

Interesting how much the people with boots on the ground and no PR departments to filter them actually do seem… pretty aware of how much we’re all going to die? Or at least they seem aware the probability of igniting the atmosphere is over 50% and not nicely contained in the distant future. Something of an update there for me.

simiones 5y ago
This is far-fetched sci-fi problem invention. The only real danger of AI in the next 1000 years is in things no one in the field is seriously addressing: use of AI in things like law enforcement, trained on bad data, to accelerate and justify existing systemic biases.
- geofft 5y ago
  Not that I disagree, but it's not just a danger of AI, it's a danger of complex self-optimizing systems in general. We've got a pretty robust complex self-optimizing system already, which goes by names like "society" and "the free market". You point it at bad data and it will reinforce that data. If demographic A is poorer than demographic B, market forces will determine that giving loans to demographic B will get you better risk-adjusted returns, causing demographic B to become even more relatively prosperous, justifying future decisions to prefer giving loans to demographic B. Which is to say that, while I think it's good for folks thinking about AI alignment to worry about this problem, all of us should worry about this problem.
- hollerith 5y ago
  >The only real danger of AI in the next 1000 years is . . . That's not true.
littlestymaar 5y ago
We're all going to die, like all our ancestors did before us, that's no surprise here ;). And about “igniting the atmosphere”, I'm much more worried about what we are currently doing (global warming) than with what may go wrong with AI…
- exdsq 5y ago
  Ahh, I've managed to not die every day for the last 27 years so probability seems on my side to never die
simurg 5y ago
Well, our teacher taught us that we live in the world "Beyond the Reach of God" so we'd better be aware of what the stakes are. ;) With that said, my optimism about alignment was slightly increased lately given the research about prosaic stuff. Also, all key members of EAI are very alignment-pilled and will probably focus on that part much more in the future especially if scaling continues unabated.

simiones 5y ago

This is far-fetched sci-fi problem invention. The only real danger of AI in the next 1000 years is in things no one in the field is seriously addressing: use of AI in things like law enforcement, trained on bad data, to accelerate and justify existing systemic biases.

geofft 5y ago
Not that I disagree, but it's not just a danger of AI, it's a danger of complex self-optimizing systems in general. We've got a pretty robust complex self-optimizing system already, which goes by names like "society" and "the free market". You point it at bad data and it will reinforce that data. If demographic A is poorer than demographic B, market forces will determine that giving loans to demographic B will get you better risk-adjusted returns, causing demographic B to become even more relatively prosperous, justifying future decisions to prefer giving loans to demographic B. Which is to say that, while I think it's good for folks thinking about AI alignment to worry about this problem, all of us should worry about this problem.
hollerith 5y ago
>The only real danger of AI in the next 1000 years is . . . That's not true.

littlestymaar 5y ago

We're all going to die, like all our ancestors did before us, that's no surprise here ;). And about “igniting the atmosphere”, I'm much more worried about what we are currently doing (global warming) than with what may go wrong with AI…

exdsq 5y ago
Ahh, I've managed to not die every day for the last 27 years so probability seems on my side to never die

simurg 5y ago

Well, our teacher taught us that we live in the world "Beyond the Reach of God" so we'd better be aware of what the stakes are. ;) With that said, my optimism about alignment was slightly increased lately given the research about prosaic stuff. Also, all key members of EAI are very alignment-pilled and will probably focus on that part much more in the future especially if scaling continues unabated.