16 ms·

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunk

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

j_crick 4y ago
> but I can’t come up with solid arguments against them. Let chaos take the world! /s
Vecr 4y ago
Yeah, I can't really argue against the whole idea. My "P(doom)"(language from the article), at least as related to agentic AI, is pretty low. About 7 or 8 percent before I die, and I'm not really old. There's probably not a lot I could do about it anyway, given that most AI jobs are alleged to just make it worse. That said, I can't really go around telling people with different numbers they should be working on antiviral research or physical manufacturing, just because I (personally, without their massive and cohesive "evidence" (i.e. gamed out scenarios) bases) think that's more important.
- vcxy 4y ago
  "Roll a D20 and if you get 1 humanity as you know it will be destroyed in your lifetime" seems bad. Why do you consider 7 or 8 percent low?
  - Vecr 4y ago
    I'm not one of those "all value in the universe" people, P(doom) for me does not have to be that bad. I would actually consider the inability to manufacture or unwillingness to manufacture certain things at scale in the US to count. Generally, my point is that agentic AI is not something I can really do much about, and other things have a combined P(doom) probability higher than it. The unwillingness in certain groups to use what they consider to be toxic or "forever" chemicals in the manufacture of physical products at scale that can make them world a better and less fragile place is something that could be worked in instead.
- HDThoreaun 4y ago
  7 or 8 percent chance of apocalypse is low?
whatshisface 4y ago
I don't think it's impossible to come up with arguments against them. For one thing, extrapolating the current gradual rate of progress forwards means that we will get to see several minor "intelligence spills" before the hypothetical big and last one, and by observing what went wrong, humanity will have the opportunity to come up with solutions. Since very smart human beings have in history done a lot of damage, but never ended the species or anything, we can conclude that the world has an "intelligence safety margin" that extends up to Alexander the Great or Napoleon. There have been some incredibly smart people in human history and none of them have ruined everything, a few countries at most.
- Vecr 4y ago
  I'm pretty sure they have stock arguments against what you said, even from people like Nick Bostrom who I think have a bit better handle on the situation. In that vein though, I think making the world more robust in general might be a way forward. Having multiple broad spectrum anti-virals for every class of virus ready to go and stockpiled, having flexible on-shore manufacturing infrastructure, with supply chains that are on the same continent or at least have massive component stockpiling on national levels, and having smart and vigilant people running these systems from the very top down to the janitors and security guards. Make a world where the machine stopping because a few agents, human or otherwise, doing something unexpected, much less likely. At least to the point EAs can start buying mosquito nets again.
- arisAlexis 4y ago
  something never happening before == can't happen is extremely dangerous attittude. I suggest you read Taleb and his duck example.
  - Quarrelsome 4y ago
    perhaps its more about "worry about the issue when it _can_ happen" as opposed to planning for the expiry of the sun today.
- ShredKazoo 4y ago
  >I don't think it's impossible to come up with arguments against them. For one thing, extrapolating the current gradual rate of progress forwards means that we will get to see several minor "intelligence spills" before the hypothetical big and last one, and by observing what went wrong, humanity will have the opportunity to come up with solutions. This has been discussed under "warning shots" https://www.lesswrong.com/posts/idipkijjz5PoxAwju/warning-shots-probably-wouldn-t-change-the-picture-much https://www.lesswrong.com/posts/idipkijjz5PoxAwju/warning-sh...
akvadrako 4y ago
My argument against is that we can't do much about it. We can't predict how machines will interpret our rules if they are smarter than us. Nor do we have a shared set of values we can articulate. Nor can we control rogue users making rogue AI. So unless one is proposing a global crackdown on AI research, AI safety is a lost cause.
- mtlmtlmtlmtl 4y ago
  That doesn't mean it should just be ignored though. Alignment and safety is important and a real problem even for non-general AI. There are lots of solvable problems that will never be solved if we just throw up our hands and say "we'll never get it right, might as well not try".
- kubanczyk 4y ago
  > Nor can we control rogue users making rogue AI. Any uncontrolled group that creates an AI can either find a body of safety research accessible to them, or not. Preparing the former is hardly a lost cause.
lozenge 4y ago
What have they actually done for AI safety? Written essays and held symposiums in grand Wytham Abbey. Attracted followers and forum posts. Where's their git repo?
lame-robot-hoax 4y ago
Most of the people that could have better things to do than to respond and give it more credibility. As well, while the arguments are logical, most of them rely upon large assumptions to move between steps. If any of these assumptions fail, the entire thing fails. Especially the hard take off assumption. As well, the assumption that AGI is happening in the next 3-10 years. I’d say most prominent people in the AI research space don’t think we’re much, if any, closer to AGI. Yet you have Yud and LW screaming that we will all be dead in a few years and AGI is right around the corner. When people like Chollet and Ng say we aren’t close to AGI, I’m more likely to believe they’re right, vs. Yud who hasn’t contributed to any actual developments within the field besides theorizing about alignment and how AGI can go wrong.
Quarrelsome 4y ago
Humans + AI will always be stronger than AI alone. So why be afraid of AI? Nothing has changed, humans still have all the agency and remain the #1 tangible threat to other humans which has been the case throughout almost all of human history.
simple-thoughts 4y ago
The ai safety arguments rely on something called the “orthoganality thesis” which is a huge, unintuitive assumption. In real life, intelligence is associated with picking better goals. Generally, an entity with higher intelligence will pick different goals than a being with lower intelligence. The orthoganility thesis is an unproven assumption that intelligence and goals are not corellated, meaning an intelligent being can pursue stupid goals. States like that, it’s obviously wrong and laughable. But by using complex language, EA cultists hide the ridiculous assumptions their system has so that they can maintain their feelings of superiority while gaining real power that enables them to abuse others.
- aetherson 4y ago
  I've known lots of intelligent people who pursue stupid goals. I maybe agree with you that there's an this belief that a maximally intelligent creature will blindly follow maximally obviously stupid goals, and that belief is under-argued, but your phrasing above isn't the slam dunk argument that you seem to believe.
  - simple-thoughts 4y ago
    Remember the term here is “associated”. If you spend a lot of time with people from diverse mental backgrounds, which is much harder to do than you might think, you’ll easily see that people who lack education, common sense, or the necessary context to be intelligent to pick much worse goals than people who aren’t in that position. Of course none of us humans are particularly intelligent, and all of us are stupid. Being a human is not intrinsically about intelligence - being a human is about being a social mammal. It’s funny seeing how rationalists often fail to exercise while emphasizing their own rationality - a perfect contradiction demonstrated by the difference between imagination and bodily realities.
- rafabulsing 4y ago
  No goals are intrinsically stupid. The only reason that you might think some particular goal is stupid is that it goes against your goals. You could conceive of a super intelligent AI that came into existence with the goal of terminating itself. That would be an "stupid goal" from our perspective, since we have the goal of self-preservation really ingrained in our brains. But for a being that self-termination is the absolute best thing ever, it's not stupid. It makes perfect sense, since, well, that it's goal. It doesn't care about self-preservation, it doesn't care about becoming more intelligent/rich/powerful, other than as an instrumental goal to help achieve self-termination, if it's not able to do so in its current state. And most importantly, no amount of getting more intelligent would change this fundamental goal, just as humans getting more intelligent has not overridden our fundamental goals of "breathe, feed, have sex". It may have given us other goals as well, but those are very much still there.
AnIdiotOnTheNet 4y ago
There are many, but the biggest one is just that the possibility space is vast and unknowable and while drawing a line straight through it to AI doom is of course possible it doesn't say anything at all about likelihood, and it can't. It makes a ton of assumptions and expects everyone to just go along with them to reach its desired conclusion. AI certainly has risks, I don't think any reasonable human being doubts that, its just that the AI doom cult seems to think the worst outcomes are near certainties without really backing that up.
- JohnPrine 4y ago
  > AI doom cult seems to think the worst outcomes are near certainties without really backing that up What are you basing this on? There have been tons of arguments written about why these worst outcomes are likely. Read Bostrom's Superintelligence for example, or Yudkowsky's Intelligence Explosion Microeconomics.
  - AnIdiotOnTheNet 4y ago
    > What are you basing this on? Many conversations with AI doomers. They gloss over and make assumptions about intelligence that aren't really backed by priors and when this is pointed out they hand wave and say "but computer". > Read Bostrom's Superintelligence for example, or Yudkowsky's Intelligence Explosion Microeconomics. I don't really have any interest in doing so, and if I'm honest have a particularly unfavorable read of Yudkowsky as a person based on his cultish following.
TechBro8615 4y ago
AI safety reminds me a lot of regulatory capture, and I don't think it's a coincidence that its top proponents are in charge of the dominant players in the space. It's easy to preach about AI safety when you're sitting on all the GPU power.
danaris 4y ago
From what I've seen, the AI safety arguments basically rely on assuming that the singularity is not merely real, but guaranteed: that we will create a full-fledged general AI, that it will be smarter than us, and that it will be fully capable of upgrading itself to be far beyond our control before we have any ability to prevent it from doing so. Every step of this relies on assumptions that are not merely questionable, but unfalsifiable. Whether AGI is possible or not, regardless of anyone's personal opinion, is as yet unprovable and unfalsifiable. Assuming that AGI itself is possible, there is no way to tell whether we, as humans, can create an intelligence that is "smarter" than we are. Assuming that we can create an AGI that is "smarter" than we are, there is no way to determine whether it would be able to upgrade itself to become exponentially smarter than that, and beyond human understanding and control. If you have a hard time coming up with arguments against these things, maybe it's because they're fundamentally unfalsifiable, which makes them useless for trying to build any framework of understanding on.
- csense 4y ago
  > the AI safety arguments basically rely on assuming that the singularity is not merely real, but guaranteed The argument goes like this: If you want to save the world, then doomsday scenario S is the best place for you to invest your resources if S has a nonzero probability, and currently we are investing less resources in S than in other existential risk scenarios (per basis point of probability). "AI risk" is a pretty good candidate for S (especially back in 2010-2015 when the movement was just starting.) > Whether AGI is possible or not, regardless of anyone's personal opinion, is as yet unprovable and unfalsifiable. "AGI is impossible" is certainly falsifiable: All I have to do is build an AGI and show it to you. Further, there is no theoretical reason AGI is impossible. Rather the reverse; consider these Well Settled Scientific Facts: - (1) Human Mind = Human Brain - (2) Human Brain obeys the laws of physics - (3) We understand the laws of physics well enough to simulate them in a computer If you accept these three facts, then in theory AGI is possible: You could implement AGI by building a machine implementation of the Human Brain by fully simulating the underlying physics. > Assuming that AGI itself is possible, there is no way to tell whether we, as humans, can create an intelligence that is "smarter" than we are. > Assuming that we can create an AGI that is "smarter" than we are, there is no way to determine whether it would be able to upgrade itself to become exponentially smarter than that, and beyond human understanding and control. An AI safety person would say to this: "You're right, we don't know -- and that's exactly the problem." What we can do is assign a probability based on our confidence. How likely do you think it is that we can do those things? 20%? 2%? 0.000002%? If you say "0.000002%", what makes it so extraordinarily certain it's impossible? If you say "2%" or "20%" then as a matter of self-preservation, shouldn't our society be devoting a lot of money and smart people's time and attention to figuring out how to make sure it doesn't happen?

j_crick 4y ago

> but I can’t come up with solid arguments against them. Let chaos take the world! /s

Vecr 4y ago

Yeah, I can't really argue against the whole idea. My "P(doom)"(language from the article), at least as related to agentic AI, is pretty low. About 7 or 8 percent before I die, and I'm not really old. There's probably not a lot I could do about it anyway, given that most AI jobs are alleged to just make it worse. That said, I can't really go around telling people with different numbers they should be working on antiviral research or physical manufacturing, just because I (personally, without their massive and cohesive "evidence" (i.e. gamed out scenarios) bases) think that's more important.

vcxy 4y ago
"Roll a D20 and if you get 1 humanity as you know it will be destroyed in your lifetime" seems bad. Why do you consider 7 or 8 percent low?
- Vecr 4y ago
  I'm not one of those "all value in the universe" people, P(doom) for me does not have to be that bad. I would actually consider the inability to manufacture or unwillingness to manufacture certain things at scale in the US to count. Generally, my point is that agentic AI is not something I can really do much about, and other things have a combined P(doom) probability higher than it. The unwillingness in certain groups to use what they consider to be toxic or "forever" chemicals in the manufacture of physical products at scale that can make them world a better and less fragile place is something that could be worked in instead.
HDThoreaun 4y ago
7 or 8 percent chance of apocalypse is low?

whatshisface 4y ago

I don't think it's impossible to come up with arguments against them. For one thing, extrapolating the current gradual rate of progress forwards means that we will get to see several minor "intelligence spills" before the hypothetical big and last one, and by observing what went wrong, humanity will have the opportunity to come up with solutions. Since very smart human beings have in history done a lot of damage, but never ended the species or anything, we can conclude that the world has an "intelligence safety margin" that extends up to Alexander the Great or Napoleon. There have been some incredibly smart people in human history and none of them have ruined everything, a few countries at most.

Vecr 4y ago
I'm pretty sure they have stock arguments against what you said, even from people like Nick Bostrom who I think have a bit better handle on the situation. In that vein though, I think making the world more robust in general might be a way forward. Having multiple broad spectrum anti-virals for every class of virus ready to go and stockpiled, having flexible on-shore manufacturing infrastructure, with supply chains that are on the same continent or at least have massive component stockpiling on national levels, and having smart and vigilant people running these systems from the very top down to the janitors and security guards. Make a world where the machine stopping because a few agents, human or otherwise, doing something unexpected, much less likely. At least to the point EAs can start buying mosquito nets again.
arisAlexis 4y ago
something never happening before == can't happen is extremely dangerous attittude. I suggest you read Taleb and his duck example.
- Quarrelsome 4y ago
  perhaps its more about "worry about the issue when it _can_ happen" as opposed to planning for the expiry of the sun today.
ShredKazoo 4y ago
>I don't think it's impossible to come up with arguments against them. For one thing, extrapolating the current gradual rate of progress forwards means that we will get to see several minor "intelligence spills" before the hypothetical big and last one, and by observing what went wrong, humanity will have the opportunity to come up with solutions. This has been discussed under "warning shots" https://www.lesswrong.com/posts/idipkijjz5PoxAwju/warning-shots-probably-wouldn-t-change-the-picture-much https://www.lesswrong.com/posts/idipkijjz5PoxAwju/warning-sh...

akvadrako 4y ago

My argument against is that we can't do much about it. We can't predict how machines will interpret our rules if they are smarter than us. Nor do we have a shared set of values we can articulate. Nor can we control rogue users making rogue AI. So unless one is proposing a global crackdown on AI research, AI safety is a lost cause.

mtlmtlmtlmtl 4y ago
That doesn't mean it should just be ignored though. Alignment and safety is important and a real problem even for non-general AI. There are lots of solvable problems that will never be solved if we just throw up our hands and say "we'll never get it right, might as well not try".
kubanczyk 4y ago
> Nor can we control rogue users making rogue AI. Any uncontrolled group that creates an AI can either find a body of safety research accessible to them, or not. Preparing the former is hardly a lost cause.

lozenge 4y ago

What have they actually done for AI safety? Written essays and held symposiums in grand Wytham Abbey. Attracted followers and forum posts. Where's their git repo?

lame-robot-hoax 4y ago

Most of the people that could have better things to do than to respond and give it more credibility. As well, while the arguments are logical, most of them rely upon large assumptions to move between steps. If any of these assumptions fail, the entire thing fails. Especially the hard take off assumption. As well, the assumption that AGI is happening in the next 3-10 years. I’d say most prominent people in the AI research space don’t think we’re much, if any, closer to AGI. Yet you have Yud and LW screaming that we will all be dead in a few years and AGI is right around the corner. When people like Chollet and Ng say we aren’t close to AGI, I’m more likely to believe they’re right, vs. Yud who hasn’t contributed to any actual developments within the field besides theorizing about alignment and how AGI can go wrong.

Quarrelsome 4y ago

Humans + AI will always be stronger than AI alone. So why be afraid of AI? Nothing has changed, humans still have all the agency and remain the #1 tangible threat to other humans which has been the case throughout almost all of human history.

simple-thoughts 4y ago

The ai safety arguments rely on something called the “orthoganality thesis” which is a huge, unintuitive assumption. In real life, intelligence is associated with picking better goals. Generally, an entity with higher intelligence will pick different goals than a being with lower intelligence. The orthoganility thesis is an unproven assumption that intelligence and goals are not corellated, meaning an intelligent being can pursue stupid goals. States like that, it’s obviously wrong and laughable. But by using complex language, EA cultists hide the ridiculous assumptions their system has so that they can maintain their feelings of superiority while gaining real power that enables them to abuse others.

aetherson 4y ago
I've known lots of intelligent people who pursue stupid goals. I maybe agree with you that there's an this belief that a maximally intelligent creature will blindly follow maximally obviously stupid goals, and that belief is under-argued, but your phrasing above isn't the slam dunk argument that you seem to believe.
- simple-thoughts 4y ago
  Remember the term here is “associated”. If you spend a lot of time with people from diverse mental backgrounds, which is much harder to do than you might think, you’ll easily see that people who lack education, common sense, or the necessary context to be intelligent to pick much worse goals than people who aren’t in that position. Of course none of us humans are particularly intelligent, and all of us are stupid. Being a human is not intrinsically about intelligence - being a human is about being a social mammal. It’s funny seeing how rationalists often fail to exercise while emphasizing their own rationality - a perfect contradiction demonstrated by the difference between imagination and bodily realities.
rafabulsing 4y ago
No goals are intrinsically stupid. The only reason that you might think some particular goal is stupid is that it goes against your goals. You could conceive of a super intelligent AI that came into existence with the goal of terminating itself. That would be an "stupid goal" from our perspective, since we have the goal of self-preservation really ingrained in our brains. But for a being that self-termination is the absolute best thing ever, it's not stupid. It makes perfect sense, since, well, that it's goal. It doesn't care about self-preservation, it doesn't care about becoming more intelligent/rich/powerful, other than as an instrumental goal to help achieve self-termination, if it's not able to do so in its current state. And most importantly, no amount of getting more intelligent would change this fundamental goal, just as humans getting more intelligent has not overridden our fundamental goals of "breathe, feed, have sex". It may have given us other goals as well, but those are very much still there.

AnIdiotOnTheNet 4y ago

There are many, but the biggest one is just that the possibility space is vast and unknowable and while drawing a line straight through it to AI doom is of course possible it doesn't say anything at all about likelihood, and it can't. It makes a ton of assumptions and expects everyone to just go along with them to reach its desired conclusion. AI certainly has risks, I don't think any reasonable human being doubts that, its just that the AI doom cult seems to think the worst outcomes are near certainties without really backing that up.

JohnPrine 4y ago
> AI doom cult seems to think the worst outcomes are near certainties without really backing that up What are you basing this on? There have been tons of arguments written about why these worst outcomes are likely. Read Bostrom's Superintelligence for example, or Yudkowsky's Intelligence Explosion Microeconomics.
- AnIdiotOnTheNet 4y ago
  > What are you basing this on? Many conversations with AI doomers. They gloss over and make assumptions about intelligence that aren't really backed by priors and when this is pointed out they hand wave and say "but computer". > Read Bostrom's Superintelligence for example, or Yudkowsky's Intelligence Explosion Microeconomics. I don't really have any interest in doing so, and if I'm honest have a particularly unfavorable read of Yudkowsky as a person based on his cultish following.

TechBro8615 4y ago

AI safety reminds me a lot of regulatory capture, and I don't think it's a coincidence that its top proponents are in charge of the dominant players in the space. It's easy to preach about AI safety when you're sitting on all the GPU power.

danaris 4y ago

From what I've seen, the AI safety arguments basically rely on assuming that the singularity is not merely real, but guaranteed: that we will create a full-fledged general AI, that it will be smarter than us, and that it will be fully capable of upgrading itself to be far beyond our control before we have any ability to prevent it from doing so. Every step of this relies on assumptions that are not merely questionable, but unfalsifiable. Whether AGI is possible or not, regardless of anyone's personal opinion, is as yet unprovable and unfalsifiable. Assuming that AGI itself is possible, there is no way to tell whether we, as humans, can create an intelligence that is "smarter" than we are. Assuming that we can create an AGI that is "smarter" than we are, there is no way to determine whether it would be able to upgrade itself to become exponentially smarter than that, and beyond human understanding and control. If you have a hard time coming up with arguments against these things, maybe it's because they're fundamentally unfalsifiable, which makes them useless for trying to build any framework of understanding on.

csense 4y ago
> the AI safety arguments basically rely on assuming that the singularity is not merely real, but guaranteed The argument goes like this: If you want to save the world, then doomsday scenario S is the best place for you to invest your resources if S has a nonzero probability, and currently we are investing less resources in S than in other existential risk scenarios (per basis point of probability). "AI risk" is a pretty good candidate for S (especially back in 2010-2015 when the movement was just starting.) > Whether AGI is possible or not, regardless of anyone's personal opinion, is as yet unprovable and unfalsifiable. "AGI is impossible" is certainly falsifiable: All I have to do is build an AGI and show it to you. Further, there is no theoretical reason AGI is impossible. Rather the reverse; consider these Well Settled Scientific Facts: - (1) Human Mind = Human Brain - (2) Human Brain obeys the laws of physics - (3) We understand the laws of physics well enough to simulate them in a computer If you accept these three facts, then in theory AGI is possible: You could implement AGI by building a machine implementation of the Human Brain by fully simulating the underlying physics. > Assuming that AGI itself is possible, there is no way to tell whether we, as humans, can create an intelligence that is "smarter" than we are. > Assuming that we can create an AGI that is "smarter" than we are, there is no way to determine whether it would be able to upgrade itself to become exponentially smarter than that, and beyond human understanding and control. An AI safety person would say to this: "You're right, we don't know -- and that's exactly the problem." What we can do is assign a probability based on our confidence. How likely do you think it is that we can do those things? 20%? 2%? 0.000002%? If you say "0.000002%", what makes it so extraordinarily certain it's impossible? If you say "2%" or "20%" then as a matter of self-preservation, shouldn't our society be devoting a lot of money and smart people's time and attention to figuring out how to make sure it doesn't happen?