6 ms·

> You won't find the likes of Yudkowsky talking about this because they'd rather go on about their fan fiction than actually work to prevent real problems we ma

> You won't find the likes of Yudkowsky talking about this because they'd rather go on about their fan fiction than actually work to prevent real problems we may face soon. The fact that they've tricked some really smart people into going along with their shenanigans is extremely disappointing. If they've tricked smart people into going along with their shenanigans, it was by making clear technical arguments for why AGI is an existential risk. The core argument is just 1. We might create ML models smarter than ourselves soon 2. We don't really understand how ML models work or what they might do 3. That seems dangerous There's more to it than that of course, but most of the "more to it" is justifications for each of those steps (looking at the historical rate of progress, guessing what might go wrong during the training process, etc.). The people who dismiss that AI is an existential risk might have really good counterarguments, but I've never heard one. The only counterarguments people seem to make are "people are scared of technology all the time, but usually they're wrong to be", "that seems like sci-fi nonsense", etc. If you want people to stop being "tricked" by Yudkowsky and co, the best way to do that would probably be to come up with some counterarguments and communicate them.

baxtr 3y ago
Counter arguments to what exactly? Your line of thought is: some advanced technology is potentially dangerous. This is so vague, how can anyone counter argue? The sun is dangerous, water can be very dangerous, even food! I’m not sure I can follow.
- lxnn 3y ago
  [flagged]
- digging 3y ago
  The sun is dangerous, but we're not pushing the Earth closer to it every year.
deleted 3y ago
[deleted]
nradov 3y ago
None of those arguments for existential risk are actually "clear" or "technical". Just a lot of hand waving which only impresses those who don't understand the technology.
- digging 3y ago
  In what way does the technology disprove them? They're pretty general statements (I agree they're not really technical arguments).
  - nradov 3y ago
    You're not even asking the right question. No one can prove a negative. Extraordinary claims require extraordinary evidence. So far no one has produced real evidence that the latest AI developments represent any sort of existential threat. The proponents are essentially making religious arguments, not scientific ones.
    - digging 3y ago
      No, that's not right at all. Proof of impossibility does exist in logic: https://en.wikipedia.org/wiki/Proof_of_impossibility https://en.wikipedia.org/wiki/Proof_of_impossibility It's also demonstrable that we have other physical ideas such as FTL travel that are indicated to be impossible by current theories of physics. If we didn't have the math saying otherwise, it would be an open question of whether we can travel faster than light; but we have pretty solid math math saying we cannot. And what we're talking about is logical in nature. Is it possible to create artificial intelligence? Unarguably yes. Is it possible for human-level intelligence to exist? Unarguably yes. Therefore, how is it reasonable to say that creating human-level artificial intelligence should be assumed impossible until proven? Just because we don't have the technology for it yet doesn't mean we should assume it's impossible. Logically, it follows from the first two assertions that it is physically possible to create artificial human-like intelligence. Once again, and I don't know if I've said it in this thread directly but I've had to post it over and over again, nobody is claiming that the latest AI developments represent an existential threat. That's a complete mischaracterization of the debate and reeks of bad-faith argument but I will give you the benefit of the doubt and assume you've misunderstood. In fact, that statement conflates two things into one nonsense argument. What's scary about current AI is how quickly it is moving, indicating a short timeline to (future) dangerous AGI. Also, it is possible that future AGI will be dangerous. These are two separate, rather simple assertions. If you disagree with either, that's fair, but you have to address them separately.
wilsonnb3 3y ago
> that seems like sci-fi nonsense Your burden of proof is backwards. It is on the AI-doomers to explain why sci-fi concepts like “AI improving itself into a super intelligence” or “AGI smart enough to kill everyone to make paper clips while simultaneously stupid enough to not realize that no one will need paper clips if they are all dead” have any relevance in the real world. The entire AI-doomer worldview is built off of unproven assumptions about the nature of intelligence and what computers are capable of, largely because thought leaders in the movement are incapable of separating sci-fi from reality.
- throwway120385 3y ago
  Paperclip maximizer is the wrong word. Let's call it the "engagement maximizer" and you have a pretty accurate statement about how our civilization will be ended by use of algorithms and eventually AI to blow up our shared sense of culture. If you watched the conversation around Meta with Facebook and Instagram, and then later the conversation around TikTok, almost nobody cut to the heart of the issue, which is that "algorithms" are being used to make decisions about what to show people, and that the algorithms themselves have been changed subtly hundreds or thousands of times over the years to maximize engagement, until the type of engagement being maximized has made people basically crazy. The same engagement-maximizing work will proceed with AI, and it will allow the software engineers and managers responsible for developing the "algorithm" abdicate even more responsibility for genocides and for mass hysteria because they can pretend like they have no control over the algorithm. The same irresponsible people will pocket a bunch of money for all this work and they will maximize engagement until it blows up our entire culture. And because nobody understands it except the techno-cultists, nobody will hold them accountable.
  - akomtu 3y ago
    "Engagement maximizer" also aligns with descriptions of the anti-religion of the future I've seen. What does engage people the most? Romans knew it's bread and circuses, where circuses are bloody cruel shows on public stadiums. Today the most engaging activity is onlyfans and the like: the other dark side of human nature. Dial both to 11, add an all seeing AI with merciless thought police and you'll get an accurate picture of 2450 AD. The only variable is how long that grim stage of society will last before the tech breaking down.
Avshalom 3y ago
You tell them that this is the greatest danger to humanity of all time and that they are uniquely suited to averting that danger and they don't have to change or sacrifice or risk anything in their life while fighting this danger. It's a very compelling combination of ego and convenience.
mbgerring 3y ago
By “Clear technical arguments” you’re referring to tens of thousands of words of unreadable fan fiction
ramblenode 3y ago
> If they've tricked smart people into going along with their shenanigans, it was by making clear technical arguments for why AGI is an existential risk. To me it doesn't feel technical at all--just superficial use of some domain verbiage with lots of degrees of freedom to duct tape it all together into a story. He very much reminds me of Eric Drexler and the nanotech doomerism of the 80s and 90s. Guy also had all the right verbiage and a small following of fairly educated people. But where is the grey goo? If we need a counterargument to Yudkowsky do we also need one to Drexler?
- TeMPOraL 3y ago
  > But where is the grey goo? It's called life. Drexler may have been off with the approach to take, but he isn't wrong about the fundamentals.

baxtr 3y ago

Counter arguments to what exactly? Your line of thought is: some advanced technology is potentially dangerous. This is so vague, how can anyone counter argue? The sun is dangerous, water can be very dangerous, even food! I’m not sure I can follow.

lxnn 3y ago
[flagged]
digging 3y ago
The sun is dangerous, but we're not pushing the Earth closer to it every year.

deleted 3y ago

[deleted]

nradov 3y ago

None of those arguments for existential risk are actually "clear" or "technical". Just a lot of hand waving which only impresses those who don't understand the technology.

digging 3y ago
In what way does the technology disprove them? They're pretty general statements (I agree they're not really technical arguments).
- nradov 3y ago
  You're not even asking the right question. No one can prove a negative. Extraordinary claims require extraordinary evidence. So far no one has produced real evidence that the latest AI developments represent any sort of existential threat. The proponents are essentially making religious arguments, not scientific ones.
  - digging 3y ago
    No, that's not right at all. Proof of impossibility does exist in logic: https://en.wikipedia.org/wiki/Proof_of_impossibility https://en.wikipedia.org/wiki/Proof_of_impossibility It's also demonstrable that we have other physical ideas such as FTL travel that are indicated to be impossible by current theories of physics. If we didn't have the math saying otherwise, it would be an open question of whether we can travel faster than light; but we have pretty solid math math saying we cannot. And what we're talking about is logical in nature. Is it possible to create artificial intelligence? Unarguably yes. Is it possible for human-level intelligence to exist? Unarguably yes. Therefore, how is it reasonable to say that creating human-level artificial intelligence should be assumed impossible until proven? Just because we don't have the technology for it yet doesn't mean we should assume it's impossible. Logically, it follows from the first two assertions that it is physically possible to create artificial human-like intelligence. Once again, and I don't know if I've said it in this thread directly but I've had to post it over and over again, nobody is claiming that the latest AI developments represent an existential threat. That's a complete mischaracterization of the debate and reeks of bad-faith argument but I will give you the benefit of the doubt and assume you've misunderstood. In fact, that statement conflates two things into one nonsense argument. What's scary about current AI is how quickly it is moving, indicating a short timeline to (future) dangerous AGI. Also, it is possible that future AGI will be dangerous. These are two separate, rather simple assertions. If you disagree with either, that's fair, but you have to address them separately.

wilsonnb3 3y ago

> that seems like sci-fi nonsense Your burden of proof is backwards. It is on the AI-doomers to explain why sci-fi concepts like “AI improving itself into a super intelligence” or “AGI smart enough to kill everyone to make paper clips while simultaneously stupid enough to not realize that no one will need paper clips if they are all dead” have any relevance in the real world. The entire AI-doomer worldview is built off of unproven assumptions about the nature of intelligence and what computers are capable of, largely because thought leaders in the movement are incapable of separating sci-fi from reality.

throwway120385 3y ago
Paperclip maximizer is the wrong word. Let's call it the "engagement maximizer" and you have a pretty accurate statement about how our civilization will be ended by use of algorithms and eventually AI to blow up our shared sense of culture. If you watched the conversation around Meta with Facebook and Instagram, and then later the conversation around TikTok, almost nobody cut to the heart of the issue, which is that "algorithms" are being used to make decisions about what to show people, and that the algorithms themselves have been changed subtly hundreds or thousands of times over the years to maximize engagement, until the type of engagement being maximized has made people basically crazy. The same engagement-maximizing work will proceed with AI, and it will allow the software engineers and managers responsible for developing the "algorithm" abdicate even more responsibility for genocides and for mass hysteria because they can pretend like they have no control over the algorithm. The same irresponsible people will pocket a bunch of money for all this work and they will maximize engagement until it blows up our entire culture. And because nobody understands it except the techno-cultists, nobody will hold them accountable.
- akomtu 3y ago
  "Engagement maximizer" also aligns with descriptions of the anti-religion of the future I've seen. What does engage people the most? Romans knew it's bread and circuses, where circuses are bloody cruel shows on public stadiums. Today the most engaging activity is onlyfans and the like: the other dark side of human nature. Dial both to 11, add an all seeing AI with merciless thought police and you'll get an accurate picture of 2450 AD. The only variable is how long that grim stage of society will last before the tech breaking down.

Avshalom 3y ago

You tell them that this is the greatest danger to humanity of all time and that they are uniquely suited to averting that danger and they don't have to change or sacrifice or risk anything in their life while fighting this danger. It's a very compelling combination of ego and convenience.

mbgerring 3y ago

By “Clear technical arguments” you’re referring to tens of thousands of words of unreadable fan fiction

ramblenode 3y ago

> If they've tricked smart people into going along with their shenanigans, it was by making clear technical arguments for why AGI is an existential risk. To me it doesn't feel technical at all--just superficial use of some domain verbiage with lots of degrees of freedom to duct tape it all together into a story. He very much reminds me of Eric Drexler and the nanotech doomerism of the 80s and 90s. Guy also had all the right verbiage and a small following of fairly educated people. But where is the grey goo? If we need a counterargument to Yudkowsky do we also need one to Drexler?

TeMPOraL 3y ago
> But where is the grey goo? It's called life. Drexler may have been off with the approach to take, but he isn't wrong about the fundamentals.