4 ms·

it’s hard to disagree with him with any empirical basis when all of his statements seem empirically sound and all of his opponent’s AI Doomer statements seem li

it’s hard to disagree with him with any empirical basis when all of his statements seem empirically sound and all of his opponent’s AI Doomer statements seem like evidenceless FUD I couldn’t help noticing that all the AI Doomer folks are pure materialists who think that consciousness and will can be completely encoded in cause-and-effect atomic relationships. The real problem is that that belief is BS until proven true. And as long as there are more good actors than bad, and AI remains just a sophisticated tool, the good effects will always outweigh the bad effects.

MattHeard 2y ago
Hello, thank you for sharing your thoughts on this topic. I'm currently writing a blog post where the thesis is that the root disagreement between "AI doomers" and others is actually primarily a disagreement about materialism, and I've been looking for evidence of this disagreement in the wild. Thanks for sharing your opinion.
- xpe 2y ago
  Really? You sound serious. I would recommend rethinking such a claim. There are simpler and more plausible explanations for why people view existential risk differently.
  - pmarreck 2y ago
    What are those? Because the risk is far higher if you believe that "will" is fundamentally materialist in nature. Those of us who do not (for whatever reason), do not evaluate this risk remotely as highly. It is difficult to prove an irrational thing with rationality. How do we know that you and I see the same color orange (this is the concept of https://en.wikipedia.org/wiki/Qualia https://en.wikipedia.org/wiki/Qualia)? Measuring the wavelength entering our eyes is insufficient. This is going to end up being an infinitely long HN discussion because it's 1) unsolvable without more data 2) infinitely interesting to any intellectual /shrug
- pmarreck 2y ago
  If you look at the backgrounds of the list of people who have signed the "AI Doomer" manifesto (the one urging what I'd call an overly extreme level of caution), such at Geoffrey Hinton, Eliezer Yudkowsky, Elon Musk etc... you will find that they're all rational materialists. I don't think the correlation is accidental. So you're on to something, here. And I've felt the exact same way as you, here. I'd love to see your blog post when it's done.
ben_w 2y ago
To me, it seems like LeCun is missing the point of the (many and diverse) doom arguments. The is no need for consciousness, there is only a need for a bug. It was purely luck that Nikita Khrushchev was in New York when Thule Site J mistook the moon for a soviet attack force. There is no need for AI to seize power, humans will promote any given AI beyond the competency of that AI just as they already do with fellow humans ("the Peter principle"). The relative number of good and bad actors — even if we could agree on what that even meant, which we can't, especially with commons issues, iterated prisoners' dilemmas, and other similar Nash equilibria — doesn't help either way when the AI isn't aligned with the user. (You may ask what I mean by "alignment", and in this case I mean vector cosine similarity "how closely will it do what the user really wants it to do, rather than what the creator of the AI wants, or what nobody at all wants because it's buggy?") But even then, AI compute is proportional to how much money you have, so it's not a democratic battle, it's an oligarchic battle. And even then, reality keeps demonstrating the incorrectness of the saying "the only way to stop a bad guy with a gun is a good guy with a gun", it's much easier to harm and destroy than to heal and build. And that's without anyone needing to reach for "consciousness in the machines" (whichever of the 40-something definitions of "consciousness" you prefer). Likewise it doesn't need plausible-future-but-not-yet-demonstrated things like "engineering a pandemic" or "those humanoid robots in the news right now, could we use them as the entire workforce in a factory to make more of them?"
- ziofill 2y ago
  I agree. Also, I’ve heard LeCun arguing that a super intelligent AI wouldn’t be so “dumb” as to decide to do something terrible for humanity. So it will be 100% resistant to adversarial attacks? And malicious actors won’t ever train their own? And even if we don’t go all the way to super intelligence, is it riskless to progressively yield control to AIs?
- scotty79 2y ago
  Details are fun but the dilemma is: should the humanity seriously cripple itself (by avoiding AI) out of the fear it might hurt itself (with AI)? Are you gonna cut off your arm because you might hit yourself in the face with it in the future? The more useful the tool is, the more dangerous it is usually. Should we have killed all nuclear physicists before they figured out how to release nuclear energy? And even so.. would that prevent things or just delay things? Would it make us more or less prepared for the things to come?
TeMPOraL 2y ago
> consciousness and will can be completely encoded in cause-and-effect atomic relationships. The real problem is that that belief is BS until proven true. Wait. Isn't literally the exactly other way around? Materialism is the null hypothesis here, backed by all empirical evidence to date; it's all the other hypotheses presenting some kind of magic that are BS until proven.
- h_tbob 2y ago
  A wise philosopher once said this. You know your experience is real. But you do not know if the material world you see is the result of a great delusion by a master programmer. Thus the only thing you truly know has no mass at all. Thus a wise person takes the immaterial as immediate apparent, but the physical as questionable. You can always prove the immaterial “I think therefore I am”. But due to the uncertainty of matter, nothing physical can be truly known. In other words you could always be wrong in your perception. So in sum, your experience has no mass, volume, or width. There are no physical properties at all to consciousness. Yet it is the only thing that we can know exists. Weird, huh?
  - ziofill 2y ago
    Reminds me of Donald Hoffman’s perspective on consciousness
  - scotty79 2y ago
    > You can always prove the immaterial “I think therefore I am”. But due to the uncertainty of matter, nothing physical can be truly known. But the brain that does the proving of immaterial is itself material so if matter is uncertain then the reasoning of the proof of immaterial can also be flawed thus you can't prove anything. The only provable thing is that philosophers ask themselves useless questions, think about them long and hard building up convoluted narratives they claim to be proofs, but on the way they assume something stupid to move forward, which eventually leads to bogus "insights".
  - Chance-Device 2y ago
    Descartes. And it’s pretty clear that consciousness is the Noumenon, just the part of it that is us. So if you want to know what the ontology of matter is, congratulations, you’re it.
ImHereToVote 2y ago
It's a food thing our fate won't be sealed by a difference in metaphysical beliefs.
nkrisc 2y ago
> I couldn’t help noticing that all the AI Doomer folks are pure materialists who think that consciousness and will can be completely encoded in cause-and-effect atomic relationships. The real problem is that that belief is BS until proven true. It’s no less BS than the other beliefs which can be summed up as “magic”.
- pmarreck 2y ago
  > It’s no less BS than the other beliefs which can be summed up as “magic”. So basically I have to choose between a non-dualist pure-materialist worldview in which every single thing I care about, feel or experience is fundamentally a meaningless illusion (and to what end? why have a universe with increasing entropy except for life which takes this weird diversion, at least temporarily, into lower entropy?), which I'll sarcastically call the "gaslighting theory of existence", and a universe that might be "materialism PLUS <undiscovered elements>" which you arrogantly dismiss as "magic" by conveniently grouping it together with arguably-objectively-ridiculous arbitrary religious beliefs? Sounds like a false-dichotomy fallacy to me
xpe 2y ago
Many people disagree with LeCun for reasoning having nothing to do with materialism. It is about logical reasoning over possible future scenarios.
grantcas 2y ago
[dead]

MattHeard 2y ago

Hello, thank you for sharing your thoughts on this topic. I'm currently writing a blog post where the thesis is that the root disagreement between "AI doomers" and others is actually primarily a disagreement about materialism, and I've been looking for evidence of this disagreement in the wild. Thanks for sharing your opinion.

xpe 2y ago
Really? You sound serious. I would recommend rethinking such a claim. There are simpler and more plausible explanations for why people view existential risk differently.
- pmarreck 2y ago
  What are those? Because the risk is far higher if you believe that "will" is fundamentally materialist in nature. Those of us who do not (for whatever reason), do not evaluate this risk remotely as highly. It is difficult to prove an irrational thing with rationality. How do we know that you and I see the same color orange (this is the concept of https://en.wikipedia.org/wiki/Qualia https://en.wikipedia.org/wiki/Qualia)? Measuring the wavelength entering our eyes is insufficient. This is going to end up being an infinitely long HN discussion because it's 1) unsolvable without more data 2) infinitely interesting to any intellectual /shrug
pmarreck 2y ago
If you look at the backgrounds of the list of people who have signed the "AI Doomer" manifesto (the one urging what I'd call an overly extreme level of caution), such at Geoffrey Hinton, Eliezer Yudkowsky, Elon Musk etc... you will find that they're all rational materialists. I don't think the correlation is accidental. So you're on to something, here. And I've felt the exact same way as you, here. I'd love to see your blog post when it's done.

ben_w 2y ago

To me, it seems like LeCun is missing the point of the (many and diverse) doom arguments. The is no need for consciousness, there is only a need for a bug. It was purely luck that Nikita Khrushchev was in New York when Thule Site J mistook the moon for a soviet attack force. There is no need for AI to seize power, humans will promote any given AI beyond the competency of that AI just as they already do with fellow humans ("the Peter principle"). The relative number of good and bad actors — even if we could agree on what that even meant, which we can't, especially with commons issues, iterated prisoners' dilemmas, and other similar Nash equilibria — doesn't help either way when the AI isn't aligned with the user. (You may ask what I mean by "alignment", and in this case I mean vector cosine similarity "how closely will it do what the user really wants it to do, rather than what the creator of the AI wants, or what nobody at all wants because it's buggy?") But even then, AI compute is proportional to how much money you have, so it's not a democratic battle, it's an oligarchic battle. And even then, reality keeps demonstrating the incorrectness of the saying "the only way to stop a bad guy with a gun is a good guy with a gun", it's much easier to harm and destroy than to heal and build. And that's without anyone needing to reach for "consciousness in the machines" (whichever of the 40-something definitions of "consciousness" you prefer). Likewise it doesn't need plausible-future-but-not-yet-demonstrated things like "engineering a pandemic" or "those humanoid robots in the news right now, could we use them as the entire workforce in a factory to make more of them?"

ziofill 2y ago
I agree. Also, I’ve heard LeCun arguing that a super intelligent AI wouldn’t be so “dumb” as to decide to do something terrible for humanity. So it will be 100% resistant to adversarial attacks? And malicious actors won’t ever train their own? And even if we don’t go all the way to super intelligence, is it riskless to progressively yield control to AIs?
scotty79 2y ago
Details are fun but the dilemma is: should the humanity seriously cripple itself (by avoiding AI) out of the fear it might hurt itself (with AI)? Are you gonna cut off your arm because you might hit yourself in the face with it in the future? The more useful the tool is, the more dangerous it is usually. Should we have killed all nuclear physicists before they figured out how to release nuclear energy? And even so.. would that prevent things or just delay things? Would it make us more or less prepared for the things to come?

TeMPOraL 2y ago

> consciousness and will can be completely encoded in cause-and-effect atomic relationships. The real problem is that that belief is BS until proven true. Wait. Isn't literally the exactly other way around? Materialism is the null hypothesis here, backed by all empirical evidence to date; it's all the other hypotheses presenting some kind of magic that are BS until proven.

h_tbob 2y ago
A wise philosopher once said this. You know your experience is real. But you do not know if the material world you see is the result of a great delusion by a master programmer. Thus the only thing you truly know has no mass at all. Thus a wise person takes the immaterial as immediate apparent, but the physical as questionable. You can always prove the immaterial “I think therefore I am”. But due to the uncertainty of matter, nothing physical can be truly known. In other words you could always be wrong in your perception. So in sum, your experience has no mass, volume, or width. There are no physical properties at all to consciousness. Yet it is the only thing that we can know exists. Weird, huh?
- ziofill 2y ago
  Reminds me of Donald Hoffman’s perspective on consciousness
- scotty79 2y ago
  > You can always prove the immaterial “I think therefore I am”. But due to the uncertainty of matter, nothing physical can be truly known. But the brain that does the proving of immaterial is itself material so if matter is uncertain then the reasoning of the proof of immaterial can also be flawed thus you can't prove anything. The only provable thing is that philosophers ask themselves useless questions, think about them long and hard building up convoluted narratives they claim to be proofs, but on the way they assume something stupid to move forward, which eventually leads to bogus "insights".
- Chance-Device 2y ago
  Descartes. And it’s pretty clear that consciousness is the Noumenon, just the part of it that is us. So if you want to know what the ontology of matter is, congratulations, you’re it.

ImHereToVote 2y ago

It's a food thing our fate won't be sealed by a difference in metaphysical beliefs.

nkrisc 2y ago

> I couldn’t help noticing that all the AI Doomer folks are pure materialists who think that consciousness and will can be completely encoded in cause-and-effect atomic relationships. The real problem is that that belief is BS until proven true. It’s no less BS than the other beliefs which can be summed up as “magic”.

pmarreck 2y ago
> It’s no less BS than the other beliefs which can be summed up as “magic”. So basically I have to choose between a non-dualist pure-materialist worldview in which every single thing I care about, feel or experience is fundamentally a meaningless illusion (and to what end? why have a universe with increasing entropy except for life which takes this weird diversion, at least temporarily, into lower entropy?), which I'll sarcastically call the "gaslighting theory of existence", and a universe that might be "materialism PLUS <undiscovered elements>" which you arrogantly dismiss as "magic" by conveniently grouping it together with arguably-objectively-ridiculous arbitrary religious beliefs? Sounds like a false-dichotomy fallacy to me

xpe 2y ago

Many people disagree with LeCun for reasoning having nothing to do with materialism. It is about logical reasoning over possible future scenarios.

grantcas 2y ago

[dead]