4 ms·

He's actually suggested it may happen within 10 years. Which makes it hard to take him seriously in general. (I do personally take his overall concerns serious

He's actually suggested it may happen within 10 years. Which makes it hard to take him seriously in general. (I do personally take his overall concerns seriously, but I don't trust his timeline predictions at all, and many of his other statements.)

kylebenzle 3y ago
Can anyone enumerate what exactly your concerns are cause I don't get it. AI I'd a great spell checker, a REALLY good spell checker, and people are losing their minds over it, I don't get it.
- ahamm 3y ago
  I used to think that this fear was driven by rational minds until I read Michael Lewis' "Going Infinite" and learned more about effective altruism [EA]. Michael Lewis writes that what is remarkable about these guys [i.e. EAs] is that they're willing to follow their beliefs to their logical conclusion (paraphrasing) without regard to social cost/consequences or just inconvenience of it all. In fact, in my estimation this is just the definition of religious fundamentalism, and gives us a new lens with which to understand EA and the well funded brain children of the movement. Every effective religion needs a doomsday scenario, or some 'second coming' apocalyptic like scenario (not sure why, just seems to be a pattern). I think all this fear mongering around AI is just that - it's based on irrational belief at the end of the day - its humanism rewashed into tech 'rationalism' (which was originally just washed from Christianity et. al.)
  - hm-nah 3y ago
    My comment above, which I think you’re replying to, is not based in fear. It’s based on first-hand experience experimenting with the likes of AutoExpert, AutoGen and ChatDev. The tools underlying these projects are quite close to doing, in a very short amount of time and cost, what it takes human knowledge workers a long time (and hence cost) to do. I think it’s as close as Summer 24’ that we get simplified GenAI grounding. Once hallucinations are grounded and there are some cleaner ways to implement GenAI workflows and pipelines…it won’t be long until you see droves of knowledge workers looking for jobs. Or if not, they’ll be creating the workflows that replace the bulk of their work, hence we’ll get that “I only want to work 20hrs a week” reality.
    - ahamm 3y ago
      I was responding to the initial question actually (https://news.ycombinator.com/item?id=38113521 https://news.ycombinator.com/item?id=38113521). I appreciate the insight though - looking forward to checking out the Gibson book. Still, I'm not sure that I see the 'AI jumping out of the box scenario' any time soon (or ever). Chat apps finding missile codes, or convincing your smart toaster to jump in your bubble bath while you're inside seem only as real as a man in a box with intentions is (and unfortunately humans can kill from inside and out of boxes). I'm definitely concerned about the implications to social welfare, the social bias involved in systems that make big decisions about an individuals freedom, etc, but these leaps from super impressive automated text analysis to humanity doomsday scenario seem like fear mongering to me mostly because these are scenarios that already exist today (along with massive problems in general social welfare). The scenarios that don't exist (like Nick Bostrom's "An objective function that turns the world into a paper clip factory in "Superintelligence: Paths, Dangers, Strategies") strike me as easy to fix by just standing by the power outlet so things don't get out of hand. There are a lot of risks that don't exist yet. Alien contact is one of them - never happened, but it could happen, and if it does it could wipe us out - so be afraid and do something noble for the cause. This to me feels like a very rational response to what is essentially a 'magical' prior. We're scared now because we 'feel' close to General AI, but we really have no way of quantifying how close we are to it, and how dangerous (if at all) it would actually be. I'm definitely open to being wrong, but its hard not to agree with LeCun on a level.
- marvin 3y ago
  Humans are likely far less intelligent/capable than systems that will soon be practically feasible to engineer, and we will lose to a generally superhuman AI that has sufficiently different objectives than we do. So an accident with a runaway a system like that would be catastrophic. Then Yudkowsky spins a gauntlet of no-evidence hypotheses for why such an accident is inevitable and leads to the death of literally all humans in the same instant. But the first part of the argument is something that will be the critical piece of the engineering of such systems.
  - kylebenzle 3y ago
    But can ANYONE bridge the gap from LLMs to an AGI? I've been programming since highschool, 5 years doing data science and I am at a total loss as to what people that the threat it. Like, ACTUALLY, what do you think is the fear?
- berniedurfee 3y ago
  I don’t think we’re close, not even within 50 years of real AGI. Which may or may not decide to wipe us out. However, even given the current state of “AI”, I think there are countless dangers and risks. The ability to fabricate audio and video that’s indistinguishable from the real thing can absolutely wreck havoc on society. Even just the transition of spam and phishing from poorly worded blasts with grammatical errors to very eloquent and specific messages will exponentially increase the effectiveness of attacks. LLM generated content which is a mix of fact and fantasy will soon far outweigh all the written and spoken content created by humans in all of history. That’s going to put humans in a place we’ve never been. Current “AI” is a tool that can allow a layman to very quickly build all sorts of weapons to be used to meet their ends. It’s scary because “AI” is a very powerful tool that individuals and states will weaponize and turn against their enemies. It will be nice to finally have a good spellchecker though.
  - recursive 3y ago
    > The ability to fabricate audio and video that’s indistinguishable from the real thing can absolutely wreck havoc on society. Only for as long people hold on to the idea that videos are reliable. Video is a medium. It will become as reliable as text. The existence of text, and specifically lies, has not wrecked society.
    - ben_w 3y ago
      Lots of people have fought and died in wars over the belief that a book was the literal Word of God, and some of those wars lasted a lifetime.
      - recursive 3y ago
        In the future we can do that with videos. But it won't be anything new.
        ben_w 3y ago
        I'm not disputing that, I'm saying there's precedent to expect it will wreck societies.
- hackinthebochs 3y ago
  To be clear, no one is afraid of these text-completion models. We're afraid of what might be around the corner. The issue is that small misalignments in an optimizer's objectives can have outsized real-world effects. General intelligence allows an optimizer to find efficient solutions to a wide range of computational problems, thus maximizing the utility of available computational resources. The rules constrain its behavior such that on net it ideally provides sufficient value to us above what it destroys (all action destroys value e.g. energy). But misalignment in objectives provides an avenue by which the AGI can on net destroy value despite our best efforts. Can we be sure we can provide loophole-free objectives that ensures only value-producing behavior from the human perspective? Can we prove that the ratio of value created to value lost due to misalignment is always above some suitable threshold? Can we prove that the range of value destruction is bounded so that if it does go off the rails, its damage is limited? Until we do, x-risk should be the default assumption.
- hm-nah 3y ago
  AI is, or will very soon be, superior to all human knowledge workers. In the spirit of capitalism, lowest cost/highest profit trumps all. With the US geriatric governance and speed of change, a lot of people will be hit by under/unemployment. All this termult happening, meanwhile the AI arms race is going on. Nation states hacking eachother using AI. Then you have the AI itself trying to escape the box…want a preview of this? Listen to “Agency” audiobook on Audible. I think it’s by Stephenson or Gibson. It’s surely going to be a wild decade before us.
- bbor 3y ago
  A lot of very long responses already that are missing the point IMO; the answer to your question is that a really good spell checker might be the last missing piece we needed to build larger systems that display autonomous and intelligent behavior on roughly human levels. The chat web app is fun, but people shouldn’t be worried about that at all - people should be worried about a hierarchy of 100,000 chat web apps using each other to, idk, socially engineer their way into nuclear missile codes or something —- pick your favorite doomsday ;).
boringg 3y ago
10 years that sounds like a good old fashion doomsday prophecy? Those people have been around since the dawn of time.

kylebenzle 3y ago

Can anyone enumerate what exactly your concerns are cause I don't get it. AI I'd a great spell checker, a REALLY good spell checker, and people are losing their minds over it, I don't get it.

ahamm 3y ago
I used to think that this fear was driven by rational minds until I read Michael Lewis' "Going Infinite" and learned more about effective altruism [EA]. Michael Lewis writes that what is remarkable about these guys [i.e. EAs] is that they're willing to follow their beliefs to their logical conclusion (paraphrasing) without regard to social cost/consequences or just inconvenience of it all. In fact, in my estimation this is just the definition of religious fundamentalism, and gives us a new lens with which to understand EA and the well funded brain children of the movement. Every effective religion needs a doomsday scenario, or some 'second coming' apocalyptic like scenario (not sure why, just seems to be a pattern). I think all this fear mongering around AI is just that - it's based on irrational belief at the end of the day - its humanism rewashed into tech 'rationalism' (which was originally just washed from Christianity et. al.)
- hm-nah 3y ago
  My comment above, which I think you’re replying to, is not based in fear. It’s based on first-hand experience experimenting with the likes of AutoExpert, AutoGen and ChatDev. The tools underlying these projects are quite close to doing, in a very short amount of time and cost, what it takes human knowledge workers a long time (and hence cost) to do. I think it’s as close as Summer 24’ that we get simplified GenAI grounding. Once hallucinations are grounded and there are some cleaner ways to implement GenAI workflows and pipelines…it won’t be long until you see droves of knowledge workers looking for jobs. Or if not, they’ll be creating the workflows that replace the bulk of their work, hence we’ll get that “I only want to work 20hrs a week” reality.
  - ahamm 3y ago
    I was responding to the initial question actually (https://news.ycombinator.com/item?id=38113521 https://news.ycombinator.com/item?id=38113521). I appreciate the insight though - looking forward to checking out the Gibson book. Still, I'm not sure that I see the 'AI jumping out of the box scenario' any time soon (or ever). Chat apps finding missile codes, or convincing your smart toaster to jump in your bubble bath while you're inside seem only as real as a man in a box with intentions is (and unfortunately humans can kill from inside and out of boxes). I'm definitely concerned about the implications to social welfare, the social bias involved in systems that make big decisions about an individuals freedom, etc, but these leaps from super impressive automated text analysis to humanity doomsday scenario seem like fear mongering to me mostly because these are scenarios that already exist today (along with massive problems in general social welfare). The scenarios that don't exist (like Nick Bostrom's "An objective function that turns the world into a paper clip factory in "Superintelligence: Paths, Dangers, Strategies") strike me as easy to fix by just standing by the power outlet so things don't get out of hand. There are a lot of risks that don't exist yet. Alien contact is one of them - never happened, but it could happen, and if it does it could wipe us out - so be afraid and do something noble for the cause. This to me feels like a very rational response to what is essentially a 'magical' prior. We're scared now because we 'feel' close to General AI, but we really have no way of quantifying how close we are to it, and how dangerous (if at all) it would actually be. I'm definitely open to being wrong, but its hard not to agree with LeCun on a level.
marvin 3y ago
Humans are likely far less intelligent/capable than systems that will soon be practically feasible to engineer, and we will lose to a generally superhuman AI that has sufficiently different objectives than we do. So an accident with a runaway a system like that would be catastrophic. Then Yudkowsky spins a gauntlet of no-evidence hypotheses for why such an accident is inevitable and leads to the death of literally all humans in the same instant. But the first part of the argument is something that will be the critical piece of the engineering of such systems.
- kylebenzle 3y ago
  But can ANYONE bridge the gap from LLMs to an AGI? I've been programming since highschool, 5 years doing data science and I am at a total loss as to what people that the threat it. Like, ACTUALLY, what do you think is the fear?
berniedurfee 3y ago
I don’t think we’re close, not even within 50 years of real AGI. Which may or may not decide to wipe us out. However, even given the current state of “AI”, I think there are countless dangers and risks. The ability to fabricate audio and video that’s indistinguishable from the real thing can absolutely wreck havoc on society. Even just the transition of spam and phishing from poorly worded blasts with grammatical errors to very eloquent and specific messages will exponentially increase the effectiveness of attacks. LLM generated content which is a mix of fact and fantasy will soon far outweigh all the written and spoken content created by humans in all of history. That’s going to put humans in a place we’ve never been. Current “AI” is a tool that can allow a layman to very quickly build all sorts of weapons to be used to meet their ends. It’s scary because “AI” is a very powerful tool that individuals and states will weaponize and turn against their enemies. It will be nice to finally have a good spellchecker though.
- recursive 3y ago
  > The ability to fabricate audio and video that’s indistinguishable from the real thing can absolutely wreck havoc on society. Only for as long people hold on to the idea that videos are reliable. Video is a medium. It will become as reliable as text. The existence of text, and specifically lies, has not wrecked society.
  - ben_w 3y ago
    Lots of people have fought and died in wars over the belief that a book was the literal Word of God, and some of those wars lasted a lifetime.
    - recursive 3y ago
      In the future we can do that with videos. But it won't be anything new.
      - ben_w 3y ago
        I'm not disputing that, I'm saying there's precedent to expect it will wreck societies.
hackinthebochs 3y ago
To be clear, no one is afraid of these text-completion models. We're afraid of what might be around the corner. The issue is that small misalignments in an optimizer's objectives can have outsized real-world effects. General intelligence allows an optimizer to find efficient solutions to a wide range of computational problems, thus maximizing the utility of available computational resources. The rules constrain its behavior such that on net it ideally provides sufficient value to us above what it destroys (all action destroys value e.g. energy). But misalignment in objectives provides an avenue by which the AGI can on net destroy value despite our best efforts. Can we be sure we can provide loophole-free objectives that ensures only value-producing behavior from the human perspective? Can we prove that the ratio of value created to value lost due to misalignment is always above some suitable threshold? Can we prove that the range of value destruction is bounded so that if it does go off the rails, its damage is limited? Until we do, x-risk should be the default assumption.
hm-nah 3y ago
AI is, or will very soon be, superior to all human knowledge workers. In the spirit of capitalism, lowest cost/highest profit trumps all. With the US geriatric governance and speed of change, a lot of people will be hit by under/unemployment. All this termult happening, meanwhile the AI arms race is going on. Nation states hacking eachother using AI. Then you have the AI itself trying to escape the box…want a preview of this? Listen to “Agency” audiobook on Audible. I think it’s by Stephenson or Gibson. It’s surely going to be a wild decade before us.
bbor 3y ago
A lot of very long responses already that are missing the point IMO; the answer to your question is that a really good spell checker might be the last missing piece we needed to build larger systems that display autonomous and intelligent behavior on roughly human levels. The chat web app is fun, but people shouldn’t be worried about that at all - people should be worried about a hierarchy of 100,000 chat web apps using each other to, idk, socially engineer their way into nuclear missile codes or something —- pick your favorite doomsday ;).

boringg 3y ago

10 years that sounds like a good old fashion doomsday prophecy? Those people have been around since the dawn of time.