3 ms·
To me, it seems like LeCun is missing the point of the (many and diverse) doom arguments. The is no need for consciousness, there is only a need for a bug. It
by ben_w 2y ago
To me, it seems like LeCun is missing the point of the (many and diverse) doom arguments.
The is no need for consciousness, there is only a need for a bug. It was purely luck that Nikita Khrushchev was in New York when Thule Site J mistook the moon for a soviet attack force.
There is no need for AI to seize power, humans will promote any given AI beyond the competency of that AI just as they already do with fellow humans ("the Peter principle").
The relative number of good and bad actors — even if we could agree on what that even meant, which we can't, especially with commons issues, iterated prisoners' dilemmas, and other similar Nash equilibria — doesn't help either way when the AI isn't aligned with the user.
(You may ask what I mean by "alignment", and in this case I mean vector cosine similarity "how closely will it do what the user really wants it to do, rather than what the creator of the AI wants, or what nobody at all wants because it's buggy?")
But even then, AI compute is proportional to how much money you have, so it's not a democratic battle, it's an oligarchic battle.
And even then, reality keeps demonstrating the incorrectness of the saying "the only way to stop a bad guy with a gun is a good guy with a gun", it's much easier to harm and destroy than to heal and build.
And that's without anyone needing to reach for "consciousness in the machines" (whichever of the 40-something definitions of "consciousness" you prefer).
Likewise it doesn't need plausible-future-but-not-yet-demonstrated things like "engineering a pandemic" or "those humanoid robots in the news right now, could we use them as the entire workforce in a factory to make more of them?"
- ziofill 2y agoI agree. Also, I’ve heard LeCun arguing that a super intelligent AI wouldn’t be so “dumb” as to decide to do something terrible for humanity. So it will be 100% resistant to adversarial attacks? And malicious actors won’t ever train their own? And even if we don’t go all the way to super intelligence, is it riskless to progressively yield control to AIs?
- scotty79 2y agoDetails are fun but the dilemma is: should the humanity seriously cripple itself (by avoiding AI) out of the fear it might hurt itself (with AI)? Are you gonna cut off your arm because you might hit yourself in the face with it in the future? The more useful the tool is, the more dangerous it is usually. Should we have killed all nuclear physicists before they figured out how to release nuclear energy? And even so.. would that prevent things or just delay things? Would it make us more or less prepared for the things to come?
- pmarreck 2y agoExactly! We bravely trudge forward and see what overturning the next stone brings.
- ben_w 2y agoI'm enthusiastic about the potential rewards for AI. But I look back at our history of running towards new things without awareness of (or planning for) risks, and I see Bhopal accident happening at all despite that it should have been preventable, and I see Castle Bravo being larger than expected, and I see the stories about children crushed in industrial equipment because the Victorians had no workplace health and safety, and I see the way CO2 was known to have a greenhouse effect for around a century before we got the Kyoto Protocol and Paris Climate Accords. It's hard to tell where the real risks are, vs. things which are just Hollywood plot points — this is likely true in every field, it certainly is in cryptography: https://www.schneier.com/blog/archives/2015/04/the_eighth_movi.html https://www.schneier.com/blog/archives/2015/04/the_eighth_mo... So, for example: Rainbows End is fiction, but the exact same things that lead to real-life intelligence agencies wanting to break crypto also drive people to want to find a "you gotta believe me" McGuffin in real life — even if their main goal is simply to know it's possible before it happens, in order to catch people under its influence. Why does this matter? Because we've already had a chatbot accidentally encourage someone's delusional belief that their purpose in life was to assassinate Queen Elizabeth II (https://www.bbc.com/news/technology-67012224 https://www.bbc.com/news/technology-67012224) and "find lone agents willing to do crimes for you" is reportedly a thing IS already does manually — but is that even a big deal IRL, or just a decent plot device for a story?
- xpe 2y agoMissing the point is a nice way of putting it. LeCun’s interests and position him to miss the point. Personally, I view his takes on AI as unserious — in the sense that I have a hard time believing he really engages in a serious manner. The flaws of motivated reasoning and “early-stopping” are painfully obvious.
- xpe 2y agoTypo fix: LeCun’s commercial interests and incentives position him to miss the point.
- pmarreck 2y agoGood points, and I prefer this version of the "AI Doomer" argument to the more FUD-infused ones. One note: "It was purely luck that Nikita Khrushchev was in New York when Thule Site J mistook the moon for a soviet attack force." I cannot verify this story (ironically, I not only googled but consulted two different AI's, the brand-new "Reflection" model (which is quite impressive) as well as OpenAI's GPT4o... They both say that the Thule moon false alarm occurred a year after Khrushchev's visit to New York) Point noted though.