19 ms·

The issue I take with these kind of "AI safety" organizations is that they focus on the wrong aspects of AI safety. Specifically, they run this narrative that A

by shafyy 3y ago

The issue I take with these kind of "AI safety" organizations is that they focus on the wrong aspects of AI safety. Specifically, they run this narrative that AI will make us humans go extinct. This is not a real risk today. Real risks are more in the category of systemic racism and sexism, deep fakes, over reliance on AI etc. But of course, "AI will humans extinct" is much sexier and collects clicks. Therefore, the real AI risks that are present today are underrepresented in mainstream media. But these people don't care about AI safety, they do whatever required to push their profile and companies. A good person to follow on real AI safety is Emily M. Bender (professor of computer linguistics at University of Washington): https://mstdn.social/@emilymbender@dair-community.social https://mstdn.social/@emilymbender@dair-community.social

tourgen 3y ago
The elite class in your country views AI as a risk to their status as elites, not an actual existential threat to humanity. They are just lying to you, as usual. That is what our current crop of globalist, free-trade, open-borders elites do. Imagine if you had an AI companion that instantly identified pilpul in every piece of media you consumed: voice, text, whatever. It highlighted it for you. What if you had an AI companion that identified instantly when you are being lied to or emotionally manipulated? What if this AI companion could also recommend economic and social policies that would actually improve the lives of people within your nation and not simply enrich a criminal cabal of globalist elites that treat you like cattle?
- pixl97 3y ago
  The Elite class is just as apt to consolidate power with AI and rule the entire world with it. If you have a super duper AI in your pocket looking at the data around you, then they a super super super duper duper duper AI looking at every bit of data from every corner of the world they can feed the thing giving themselves power and control you couldn't even begin to imagine. Falling into conspiratorial thinking on a single dimension without even considering all the different factors that could change belies ignorance. Yes, AI is set up to upend the elites status, but is just as apt to upset your status of being able to afford food and a house and meaningful work. > not simply enrich a criminal cabal of globalist elites that treat you like cattle? There is a different problem here... And that is humankind has made tools capable of concentrating massive amounts of power well before we solved human greed. Any system you make that's powerful has to overcome greedy power seeking hyper-optimizers. If I could somehow hit a button and Thanos away the current elites, then another group of powerseekers would just claim that status. It is an inane human behavior.
deleted 3y ago
[deleted]
acjohnson55 3y ago
I would argue that all of the above are serious concerns.
wongarsu 3y ago
A good way to categorize risk is look at both likelihood and severity of consequences. The most visible issues today (racism, deep fakes, over reliance) are almost certain to occur, but also for the most part have relatively minor consequences (mostly making things that are already happening worse). "Advanced AI will make humans extinct" is much less likely but has catastrophic consequences. Focusing on the catastrophic risks isn't unreasonable, especially since society at large seem to already handle the more frequently occurring risks (the EU's AI Act addresses many of them). And of course research into one of them benefits the other, so the categories aren't mutually exclusive.
- pixl97 3y ago
  I would put consolidating and increasing corporate and or government power on that list of potential visible very short term issues. As AI becomes more incorporated in military applications, such as individual weapon systems, or large fleets of autonomous drones then the catastrophic consequence meter clicks up a notch in the sense that attack/defense paradigms change, much like they did in WWI with the machine gun and tanks, and in WWII with high speed military operations and airplanes. Our predictive ability on when/what will start a war lowers increasing uncertainty and potential proliferation. An in a world with nukes, higher uncertainty isn't a good thing. Anyone that says AI can't/won't cause problems at this scale just ignores that individuals/corporations/governments are power seeking entities. Ones that are very greedy and unaligned with the well being of the individual can present huge risks. How we control these risks without creating other systems that are just as risky is going to be an interesting problem.
- silverlake 3y ago
  This doesn’t work either. The consequence of extinction is infinity (to humans). Likelihood * infinity = infinity. So by hand-waving at a catastrophic sci-fi scenario they can demand we heed their demands, whatever that is.
  - wongarsu 3y ago
    At the extremes you get into the territory of Pascal's Mugging [1]. Which is a delightfully simple example of how our simple methods of stating goals quickly goes wrong 1: https://en.wikipedia.org/wiki/Pascal%27s_mugging https://en.wikipedia.org/wiki/Pascal%27s_mugging
ChatGTP 3y ago
If you were to take a look at the list of signatories on safe.ai, that's basically everyone who is everyone that works on building AI, what could Emily B Bender a professor of computer linguistics possibly add to the conversation and how would she be able to talk more about the "real AI safety" than any of those people? Edit: Sorry if it sounds arrogant, I don't mean Emily wouldn't have anything to add, but not sure how the parent can just write off basically that whole list and claim someone who isn't a leader in the field would be the "real voice"?
- sebzim4500 3y ago
  I think we need to be realistic and accept that people are going to pick the expert that agrees with them, even if on paper they are far less qualified.
- deleted 3y ago
  [deleted]
  - deleted 3y ago
    [deleted]
- Planktonne 3y ago
  She's a professor of computer linguistics; it's literally her field that's being discussed. The list of signatories includes people with far less relevant qualifications, and significantly greater profit motive. She's an informed party who doesn't stand to profit; we should listen to her a lot more readily than others.
  - brookst 3y ago
    How are fame, speaking engagements, and book deals not a form of profit? She's intelligent and worth listening to, but she has just as much personal bias and motivation as anyone else.
    - Planktonne 3y ago
TristanDaCunha 3y ago
You have it totally backwards. It's a much bigger catastrophe if we over-focus on "safety" as avoiding sexism and so on, and then everyone dies.
- cubefox 3y ago
  Exactly. Biased LLMs are incredibly unimportant compared to the quite possible extinction of humanity.
olalonde 3y ago
> A good person to follow on real AI safety is Emily M. Bender (professor of computer linguistics at University of Washington): https://mstdn.social/@emilymbender@dair-community.social https://mstdn.social/@emilymbender@dair-community.social - Pronouns - "AI bros" - "mansplaining" - "extinction from capitalism" - "white supremacy" - "one old white guy" (referring to Geoffrey Hinton) Yeah... I think I will pass.
- rcpt 3y ago
  I think "pronouns" is ok but, yeah, "AI bros" is enough to get a pass from me to. Holier-than-thou name calling is still name calling.
  - Al0neStar 3y ago
    There are a lot of AI-bros on twitter.
    - deleted 3y ago
      [deleted]
    - mitthrowaway2 3y ago
      Do they self-identify as AI bros? Or are you just choosing to call them that?
- brookst 3y ago
  Odd that you lump a person's choice of their own pronouns into a legitimate complaint that all she seems to have is prejudice and ad hominems.
  - deleted 3y ago
    [deleted]
  - olalonde 3y ago
    I have no issue with her choice of pronouns. I just find it odd that she states them when ~100% of the population would infer them correctly from her name Emily (and picture). My guess is she put them there for ideological signaling.
riku_iki 3y ago
> organizations is that they focus on the wrong aspects of AI safety. Specifically, they run this narrative that AI will make us humans go extinct. their goals are to get funding, so FUD is very good focus for it..
blazespin 3y ago
the issue with hacker news comments these days is people don't actually do any due diligence before posting. center for ai safety is 90% about present AI risks and this ai statement is just a one off thing.
- adamsmith143 3y ago
  Particularly ironic given this isn't actually what the focus on... https://www.safe.ai/ai-risk https://www.safe.ai/ai-risk
deltaninenine 3y ago
Don't characterize the public as that stupid. The current risks of AI are startling clear to a layman. The extinction level even is more far fetched to a layman. You are the public and your viewpoint is aligned with the public. Nobody is thinking extinction level event.
- hollerith 3y ago
  Extinction is exactly what this submission is about. Here is the full text of the statement: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." By "extinction", the signatories mean extinction of the human species.
  - deltaninenine 3y ago
    I'm referring to the public. Not the submission.
mitthrowaway2 3y ago
> This is not a real risk today. Yes, clearly. But it is a risk for tomorrow. We do still care about the future, right?
- Filligree 3y ago
  I’m sure we can start talking about AI regulation once the existential risks are already happening. I, for one, will be saying “told you so”. That’s talking, right?
- adamsmith143 3y ago
  No man, the future of trillions of humans is obviously much less important than 1 person getting insulted on the internet.
  - CatWChainsaw 3y ago
    Especially ironic considering the "longtermism" mental illness that's in vogue with SV types.
gfodor 3y ago
the real immediate risk isn't either of these imo. it's agentic AI leveraging some of that to act on the wishes of bad actors.
kypro 3y ago
You hear similar arguments from those who believe climate change is happening but disagree with current efforts to counter-act it. The logic being that right now climate change is not causing any major harm and that we can't really predict the future so there's no point in worrying about what might happen in a decade or two. I don't think anyone is arguing that right now climate change or AI is threat to human civilisation. The point is that there are clear trends in place and that those trends are concerning. On AI specifically, it's fairly easy to see how a slightly more advanced LLM could be a destructive force if it was given an unaligned goal by a malicious actor. For example, a slightly more advanced LLM could hack into critical infrastructure killing or injuring many thousands of people. In the near-future AI may help us advance biotech research and it could aid in the creation of bioweapons and other destructive capabilities. Longer-term risks (those maybe a couple of decades out) become much greater and also much harder to predict, but they're worth thinking about and planning for today. For example, what happens when humanity becomes dependant on AI for its labour, or when AI is controlling the majority of our infrastructure? I disagree but can understand the position that AI safety isn't humanities number one risk or priority right now, however I don't understand the dismissive attitude towards what seems like a clear existential risk when you project a decade or two out.
- cmilton 3y ago
  >that those trends are concerning. Which trends would you be referring to? >it's fairly easy to see how a slightly more advanced LLM could be a destructive force if it was given an unaligned goal by a malicious actor. For example, a slightly more advanced LLM could hack into critical infrastructure killing or injuring many thousands of people. How are you building this progression? Is there any evidence to back up this claim? I am having a hard time discerning this from fear-mongering.
  - adamsmith143 3y ago
    If AI improves by 0.0001% per year on your favorite intelligence metric there will eventually be a point where it surpasses human performance and another point where it surpasses all humans combined on that metric. There is danger in that scenario.
boringuser2 3y ago
What is "systemic racism"? How is it a risk? Don't bother explaining, we already know it's unfalfisiable.
adamsmith143 3y ago
>Real risks are more in the category of systemic racism and sexism, deep fakes, over reliance on AI etc. This is a really bad take and risks missing the forest for the trees in a major way. The risks of today pale in comparison to the risks of tomorrow in this case. It's like being worried about birds dying in wind turbines while the world ecosystem collapses due to climate change. The larger risk is further away in time but far more important. Theres a real risk that people get fooled by this idea that LLMs saying bad words is more important than human extinction. Though it seems like the public is already moving on and correctly focusing on the real issues.
alasdair_ 3y ago
>This is not a real risk today. Many experts believe it is a real risk within the next decade (a “hard takeoff” scenario) That is a short enough timeframe that it’s worth caring about.
TaylorAlexander 3y ago
Yep agree. They talk a big game about how their product is so powerful it will take over the world if we're not careful, but they don't talk about how they are complicit in relatively more mundane harms (compared to AI taking over the world) that are real and happening today thanks to their system design. They want to promote the idea that their product is all-powerful, but they don't want to take responsibility for dealing with bad assumptions built in to their design.

by shafyy 3y ago

tourgen 3y ago

The elite class in your country views AI as a risk to their status as elites, not an actual existential threat to humanity. They are just lying to you, as usual. That is what our current crop of globalist, free-trade, open-borders elites do. Imagine if you had an AI companion that instantly identified pilpul in every piece of media you consumed: voice, text, whatever. It highlighted it for you. What if you had an AI companion that identified instantly when you are being lied to or emotionally manipulated? What if this AI companion could also recommend economic and social policies that would actually improve the lives of people within your nation and not simply enrich a criminal cabal of globalist elites that treat you like cattle?

pixl97 3y ago
The Elite class is just as apt to consolidate power with AI and rule the entire world with it. If you have a super duper AI in your pocket looking at the data around you, then they a super super super duper duper duper AI looking at every bit of data from every corner of the world they can feed the thing giving themselves power and control you couldn't even begin to imagine. Falling into conspiratorial thinking on a single dimension without even considering all the different factors that could change belies ignorance. Yes, AI is set up to upend the elites status, but is just as apt to upset your status of being able to afford food and a house and meaningful work. > not simply enrich a criminal cabal of globalist elites that treat you like cattle? There is a different problem here... And that is humankind has made tools capable of concentrating massive amounts of power well before we solved human greed. Any system you make that's powerful has to overcome greedy power seeking hyper-optimizers. If I could somehow hit a button and Thanos away the current elites, then another group of powerseekers would just claim that status. It is an inane human behavior.

deleted 3y ago

[deleted]

acjohnson55 3y ago

I would argue that all of the above are serious concerns.

wongarsu 3y ago

A good way to categorize risk is look at both likelihood and severity of consequences. The most visible issues today (racism, deep fakes, over reliance) are almost certain to occur, but also for the most part have relatively minor consequences (mostly making things that are already happening worse). "Advanced AI will make humans extinct" is much less likely but has catastrophic consequences. Focusing on the catastrophic risks isn't unreasonable, especially since society at large seem to already handle the more frequently occurring risks (the EU's AI Act addresses many of them). And of course research into one of them benefits the other, so the categories aren't mutually exclusive.

pixl97 3y ago
I would put consolidating and increasing corporate and or government power on that list of potential visible very short term issues. As AI becomes more incorporated in military applications, such as individual weapon systems, or large fleets of autonomous drones then the catastrophic consequence meter clicks up a notch in the sense that attack/defense paradigms change, much like they did in WWI with the machine gun and tanks, and in WWII with high speed military operations and airplanes. Our predictive ability on when/what will start a war lowers increasing uncertainty and potential proliferation. An in a world with nukes, higher uncertainty isn't a good thing. Anyone that says AI can't/won't cause problems at this scale just ignores that individuals/corporations/governments are power seeking entities. Ones that are very greedy and unaligned with the well being of the individual can present huge risks. How we control these risks without creating other systems that are just as risky is going to be an interesting problem.
silverlake 3y ago
This doesn’t work either. The consequence of extinction is infinity (to humans). Likelihood * infinity = infinity. So by hand-waving at a catastrophic sci-fi scenario they can demand we heed their demands, whatever that is.
- wongarsu 3y ago
  At the extremes you get into the territory of Pascal's Mugging [1]. Which is a delightfully simple example of how our simple methods of stating goals quickly goes wrong 1: https://en.wikipedia.org/wiki/Pascal%27s_mugging https://en.wikipedia.org/wiki/Pascal%27s_mugging

ChatGTP 3y ago

If you were to take a look at the list of signatories on safe.ai, that's basically everyone who is everyone that works on building AI, what could Emily B Bender a professor of computer linguistics possibly add to the conversation and how would she be able to talk more about the "real AI safety" than any of those people? Edit: Sorry if it sounds arrogant, I don't mean Emily wouldn't have anything to add, but not sure how the parent can just write off basically that whole list and claim someone who isn't a leader in the field would be the "real voice"?

sebzim4500 3y ago
I think we need to be realistic and accept that people are going to pick the expert that agrees with them, even if on paper they are far less qualified.
deleted 3y ago
[deleted]
- deleted 3y ago
  [deleted]
Planktonne 3y ago
She's a professor of computer linguistics; it's literally her field that's being discussed. The list of signatories includes people with far less relevant qualifications, and significantly greater profit motive. She's an informed party who doesn't stand to profit; we should listen to her a lot more readily than others.
- brookst 3y ago
  How are fame, speaking engagements, and book deals not a form of profit? She's intelligent and worth listening to, but she has just as much personal bias and motivation as anyone else.
  - Planktonne 3y ago

TristanDaCunha 3y ago

You have it totally backwards. It's a much bigger catastrophe if we over-focus on "safety" as avoiding sexism and so on, and then everyone dies.

cubefox 3y ago
Exactly. Biased LLMs are incredibly unimportant compared to the quite possible extinction of humanity.

olalonde 3y ago

> A good person to follow on real AI safety is Emily M. Bender (professor of computer linguistics at University of Washington): https://mstdn.social/@emilymbender@dair-community.social https://mstdn.social/@emilymbender@dair-community.social - Pronouns - "AI bros" - "mansplaining" - "extinction from capitalism" - "white supremacy" - "one old white guy" (referring to Geoffrey Hinton) Yeah... I think I will pass.

rcpt 3y ago
I think "pronouns" is ok but, yeah, "AI bros" is enough to get a pass from me to. Holier-than-thou name calling is still name calling.
- Al0neStar 3y ago
  There are a lot of AI-bros on twitter.
  - deleted 3y ago
    [deleted]
  - mitthrowaway2 3y ago
    Do they self-identify as AI bros? Or are you just choosing to call them that?
brookst 3y ago
Odd that you lump a person's choice of their own pronouns into a legitimate complaint that all she seems to have is prejudice and ad hominems.
- deleted 3y ago
  [deleted]
- olalonde 3y ago
  I have no issue with her choice of pronouns. I just find it odd that she states them when ~100% of the population would infer them correctly from her name Emily (and picture). My guess is she put them there for ideological signaling.

riku_iki 3y ago

> organizations is that they focus on the wrong aspects of AI safety. Specifically, they run this narrative that AI will make us humans go extinct. their goals are to get funding, so FUD is very good focus for it..

blazespin 3y ago

the issue with hacker news comments these days is people don't actually do any due diligence before posting. center for ai safety is 90% about present AI risks and this ai statement is just a one off thing.

adamsmith143 3y ago
Particularly ironic given this isn't actually what the focus on... https://www.safe.ai/ai-risk https://www.safe.ai/ai-risk

deltaninenine 3y ago

Don't characterize the public as that stupid. The current risks of AI are startling clear to a layman. The extinction level even is more far fetched to a layman. You are the public and your viewpoint is aligned with the public. Nobody is thinking extinction level event.

hollerith 3y ago
Extinction is exactly what this submission is about. Here is the full text of the statement: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." By "extinction", the signatories mean extinction of the human species.
- deltaninenine 3y ago
  I'm referring to the public. Not the submission.

mitthrowaway2 3y ago

> This is not a real risk today. Yes, clearly. But it is a risk for tomorrow. We do still care about the future, right?

Filligree 3y ago
I’m sure we can start talking about AI regulation once the existential risks are already happening. I, for one, will be saying “told you so”. That’s talking, right?
adamsmith143 3y ago
No man, the future of trillions of humans is obviously much less important than 1 person getting insulted on the internet.
- CatWChainsaw 3y ago
  Especially ironic considering the "longtermism" mental illness that's in vogue with SV types.

gfodor 3y ago

the real immediate risk isn't either of these imo. it's agentic AI leveraging some of that to act on the wishes of bad actors.

kypro 3y ago

You hear similar arguments from those who believe climate change is happening but disagree with current efforts to counter-act it. The logic being that right now climate change is not causing any major harm and that we can't really predict the future so there's no point in worrying about what might happen in a decade or two. I don't think anyone is arguing that right now climate change or AI is threat to human civilisation. The point is that there are clear trends in place and that those trends are concerning. On AI specifically, it's fairly easy to see how a slightly more advanced LLM could be a destructive force if it was given an unaligned goal by a malicious actor. For example, a slightly more advanced LLM could hack into critical infrastructure killing or injuring many thousands of people. In the near-future AI may help us advance biotech research and it could aid in the creation of bioweapons and other destructive capabilities. Longer-term risks (those maybe a couple of decades out) become much greater and also much harder to predict, but they're worth thinking about and planning for today. For example, what happens when humanity becomes dependant on AI for its labour, or when AI is controlling the majority of our infrastructure? I disagree but can understand the position that AI safety isn't humanities number one risk or priority right now, however I don't understand the dismissive attitude towards what seems like a clear existential risk when you project a decade or two out.

cmilton 3y ago
>that those trends are concerning. Which trends would you be referring to? >it's fairly easy to see how a slightly more advanced LLM could be a destructive force if it was given an unaligned goal by a malicious actor. For example, a slightly more advanced LLM could hack into critical infrastructure killing or injuring many thousands of people. How are you building this progression? Is there any evidence to back up this claim? I am having a hard time discerning this from fear-mongering.
- adamsmith143 3y ago
  If AI improves by 0.0001% per year on your favorite intelligence metric there will eventually be a point where it surpasses human performance and another point where it surpasses all humans combined on that metric. There is danger in that scenario.

boringuser2 3y ago

What is "systemic racism"? How is it a risk? Don't bother explaining, we already know it's unfalfisiable.

adamsmith143 3y ago

>Real risks are more in the category of systemic racism and sexism, deep fakes, over reliance on AI etc. This is a really bad take and risks missing the forest for the trees in a major way. The risks of today pale in comparison to the risks of tomorrow in this case. It's like being worried about birds dying in wind turbines while the world ecosystem collapses due to climate change. The larger risk is further away in time but far more important. Theres a real risk that people get fooled by this idea that LLMs saying bad words is more important than human extinction. Though it seems like the public is already moving on and correctly focusing on the real issues.

alasdair_ 3y ago

>This is not a real risk today. Many experts believe it is a real risk within the next decade (a “hard takeoff” scenario) That is a short enough timeframe that it’s worth caring about.

TaylorAlexander 3y ago

Yep agree. They talk a big game about how their product is so powerful it will take over the world if we're not careful, but they don't talk about how they are complicit in relatively more mundane harms (compared to AI taking over the world) that are real and happening today thanks to their system design. They want to promote the idea that their product is all-powerful, but they don't want to take responsibility for dealing with bad assumptions built in to their design.