4 ms·

Details are fun but the dilemma is: should the humanity seriously cripple itself (by avoiding AI) out of the fear it might hurt itself (with AI)? Are you gonna

Details are fun but the dilemma is: should the humanity seriously cripple itself (by avoiding AI) out of the fear it might hurt itself (with AI)? Are you gonna cut off your arm because you might hit yourself in the face with it in the future? The more useful the tool is, the more dangerous it is usually. Should we have killed all nuclear physicists before they figured out how to release nuclear energy? And even so.. would that prevent things or just delay things? Would it make us more or less prepared for the things to come?

pmarreck 2y ago
Exactly! We bravely trudge forward and see what overturning the next stone brings.
- ben_w 2y ago
  I'm enthusiastic about the potential rewards for AI. But I look back at our history of running towards new things without awareness of (or planning for) risks, and I see Bhopal accident happening at all despite that it should have been preventable, and I see Castle Bravo being larger than expected, and I see the stories about children crushed in industrial equipment because the Victorians had no workplace health and safety, and I see the way CO2 was known to have a greenhouse effect for around a century before we got the Kyoto Protocol and Paris Climate Accords. It's hard to tell where the real risks are, vs. things which are just Hollywood plot points — this is likely true in every field, it certainly is in cryptography: https://www.schneier.com/blog/archives/2015/04/the_eighth_movi.html https://www.schneier.com/blog/archives/2015/04/the_eighth_mo... So, for example: Rainbows End is fiction, but the exact same things that lead to real-life intelligence agencies wanting to break crypto also drive people to want to find a "you gotta believe me" McGuffin in real life — even if their main goal is simply to know it's possible before it happens, in order to catch people under its influence. Why does this matter? Because we've already had a chatbot accidentally encourage someone's delusional belief that their purpose in life was to assassinate Queen Elizabeth II (https://www.bbc.com/news/technology-67012224 https://www.bbc.com/news/technology-67012224) and "find lone agents willing to do crimes for you" is reportedly a thing IS already does manually — but is that even a big deal IRL, or just a decent plot device for a story?

pmarreck 2y ago

Exactly! We bravely trudge forward and see what overturning the next stone brings.

ben_w 2y ago
I'm enthusiastic about the potential rewards for AI. But I look back at our history of running towards new things without awareness of (or planning for) risks, and I see Bhopal accident happening at all despite that it should have been preventable, and I see Castle Bravo being larger than expected, and I see the stories about children crushed in industrial equipment because the Victorians had no workplace health and safety, and I see the way CO2 was known to have a greenhouse effect for around a century before we got the Kyoto Protocol and Paris Climate Accords. It's hard to tell where the real risks are, vs. things which are just Hollywood plot points — this is likely true in every field, it certainly is in cryptography: https://www.schneier.com/blog/archives/2015/04/the_eighth_movi.html https://www.schneier.com/blog/archives/2015/04/the_eighth_mo... So, for example: Rainbows End is fiction, but the exact same things that lead to real-life intelligence agencies wanting to break crypto also drive people to want to find a "you gotta believe me" McGuffin in real life — even if their main goal is simply to know it's possible before it happens, in order to catch people under its influence. Why does this matter? Because we've already had a chatbot accidentally encourage someone's delusional belief that their purpose in life was to assassinate Queen Elizabeth II (https://www.bbc.com/news/technology-67012224 https://www.bbc.com/news/technology-67012224) and "find lone agents willing to do crimes for you" is reportedly a thing IS already does manually — but is that even a big deal IRL, or just a decent plot device for a story?