5 ms·

This is an interesting comment because lately it feels like its very cool to be an alarmist! Lots of positive press for people warning about the dangers of AI,

This is an interesting comment because lately it feels like its very cool to be an alarmist! Lots of positive press for people warning about the dangers of AI, Altman and others being taken very seriously, VC and other funders obviously leaning into the space in part because of the related hype And in other fields, being alarmist has paid off too with little recourse for bad predictions -- how many times have we heard that there will be huge climate disasters ending humanity, the extinction of bees, mass starvation, etc. (not to diminish the dangers of climate change which is obviously very real)? I think alarmism is generally rewarded, at least in media.

gonehome 3y ago
Some types of alarm yeah, if within the window of things it's statusy to be alarmed about. Most of the AI concern that's high status to believe has been the bias, misinformation, safety, stuff. Until very recently talk about e-risk was dismissed and mocked without really engaging with the underlying arguments. That may be changing now, but on net I still mostly see people mocked and dismissed for it. The set of people alarmed by AGI e-risk are also pretty different than the set alarmed about a lot of these other issues that aren't really e-risks (though still might have bad outcomes). At least EY, Bostrom, Toby Ord are not also as worried about about all these other things to nearly the same extent - the extinction risk of unaligned AGI is different in severity.
theptip 3y ago
Important to pay attention to the content of the alarm though. Altman went in front of congress and a Senator said “when you say things could go badly, I assume you are talking about jobs”. Many people are alarmed about disinformation, job destruction, bias, etc. Actually holding an x-risk belief is still a fringe position, most people still laugh it off. That said, the Overton Window is moving. The Time piece from Yudkowsky was something of a milestone (even if it was widely ridiculed).
- miohtama 3y ago
  Altman also has very selfish motivation, because when there is now AI regulation, only Google, OpenAI (Microsoft) and maybe Meta are allowed to build “compliant” AI. It’s called regulatory capture. * EU passed its AI regulation directive recently and it has been bashed already here on HackerNews
  - matt_holden 3y ago
    Sam doesn't have much financial upside from OpenAI (reportedly, he doesn't have any equity). And he wrote about the risk in 2015 months before OpenAI was founded: https://blog.samaltman.com/machine-intelligence-part-1 https://blog.samaltman.com/machine-intelligence-part-1 https://blog.samaltman.com/machine-intelligence-part-2 https://blog.samaltman.com/machine-intelligence-part-2 Fine if you disagree with his arguments, but why assume you know what his motivation is?
    - kortilla 3y ago
      I find it highly unlikely that he has less upside than the employees who also don’t have equity, but do have profit participation units.
- trashtester 3y ago
  > Actually holding an x-risk belief is still a fringe position Beliving it is an x-risk is not fringe. It's pretty mainstream now that there is a _risk_ of an existential level event. The fringe is more like Yudkowsky or Leahy insisting that there is a near certainty of such an event if we continue down the current path. With Hinton, Bengio, Sutskever and Hassabis and Altman all agreeing that there exists a non-trivial existential risk (even if their opinions vary with respect to the magnitude), it seems more like this represents the mainstream.
  - theptip 3y ago
    I think it’s not fringe amongst experts and those in the field. It’s absolutely still fringe among the general public, and I think it’s outside the Overton Window (ie politicians aren’t talking about it).
    - trashtester 3y ago
      The Overton Window applies to the general public, and maybe particular the press. And this is all over the press and other media now, both the old and new, left leaning and right leaning. I would say it's pretty well within the Overton Window. Politicians in the US are a bit behind. They probably just need to run the topic with some polls and voter study groups to decide what opinions are most popular with their voter bases.
distortionfield 3y ago
The extinction of bees, mass starvation, and the ozone hole (bonus) are all examples of alarmist takes that were course corrected. It’s sort of a weird spot to argue that they were overblown when the reason they are not a problem now is because they were addressed.
- climatologist 3y ago
  What does it mean to address the risk of superintelligence? There is no way to stop technological progress and AI development is just part of the same process. Moreover, the alarmism doesn't make much sense because we already have misaligned agents at odds with human values, those agents are called profit seeking corporations but I never hear the alarmists talk about putting a stop to for-profit business ventures. Do you know anyone that considers the pursuit of profits and constant exploitation of natural resources as a problem that needs to be addressed because I don't. Everyone seems very happy with the status quo and AI development is just more of the same status quo development, just corporations seeking ways to exploit and profit from digital resources. OpenAI being a perfect example of this.
  - flagrant_taco 3y ago
    > There is no way to stop technological progress What makes you say this is impossible? We could simply not go down this road, there are only so many people knowledgeable enough and with access to the right hardware to make progress towards AI. They could all agree, or be compelled, to stop. We seem to have successfully halted research into cloning, though that wasn't a given and could have fallen into the same trap of having to develop it before one's enemy does.
    - climatologist 3y ago
      There are no enemies. The biosphere is a singular organism and right now people are doing their best to basically destroy all of it. The only way to prevent further damage is to reduce the human population but that's another non-starter so as long as the human population is increasing it will compel the people in charge to continue pushing for more technological "innovation" because technology is the best way to control 8B+ people[1]. Very few people are actually alarmed about the right issues (in no particular order): population size, industrial pollution, military-industrial complex, for-profit multi-national corporations, digital surveillance, factory farming, global warming, &etc. This is why the alarmism from the AI crowd seems disingenuous because AI progress is simply an extension of for-profit corporatism and exploitation applied to digital resources and to properly address the risk from AI would require addressing the actual root causes of why technological progress is misaligned with human values. 1: https://www.theguardian.com/world/2015/jul/24/france-big-brother-surveillance-powers https://www.theguardian.com/world/2015/jul/24/france-big-bro...

gonehome 3y ago

Some types of alarm yeah, if within the window of things it's statusy to be alarmed about. Most of the AI concern that's high status to believe has been the bias, misinformation, safety, stuff. Until very recently talk about e-risk was dismissed and mocked without really engaging with the underlying arguments. That may be changing now, but on net I still mostly see people mocked and dismissed for it. The set of people alarmed by AGI e-risk are also pretty different than the set alarmed about a lot of these other issues that aren't really e-risks (though still might have bad outcomes). At least EY, Bostrom, Toby Ord are not also as worried about about all these other things to nearly the same extent - the extinction risk of unaligned AGI is different in severity.

theptip 3y ago

Important to pay attention to the content of the alarm though. Altman went in front of congress and a Senator said “when you say things could go badly, I assume you are talking about jobs”. Many people are alarmed about disinformation, job destruction, bias, etc. Actually holding an x-risk belief is still a fringe position, most people still laugh it off. That said, the Overton Window is moving. The Time piece from Yudkowsky was something of a milestone (even if it was widely ridiculed).

miohtama 3y ago
Altman also has very selfish motivation, because when there is now AI regulation, only Google, OpenAI (Microsoft) and maybe Meta are allowed to build “compliant” AI. It’s called regulatory capture. * EU passed its AI regulation directive recently and it has been bashed already here on HackerNews
- matt_holden 3y ago
  Sam doesn't have much financial upside from OpenAI (reportedly, he doesn't have any equity). And he wrote about the risk in 2015 months before OpenAI was founded: https://blog.samaltman.com/machine-intelligence-part-1 https://blog.samaltman.com/machine-intelligence-part-1 https://blog.samaltman.com/machine-intelligence-part-2 https://blog.samaltman.com/machine-intelligence-part-2 Fine if you disagree with his arguments, but why assume you know what his motivation is?
  - kortilla 3y ago
    I find it highly unlikely that he has less upside than the employees who also don’t have equity, but do have profit participation units.
trashtester 3y ago
> Actually holding an x-risk belief is still a fringe position Beliving it is an x-risk is not fringe. It's pretty mainstream now that there is a _risk_ of an existential level event. The fringe is more like Yudkowsky or Leahy insisting that there is a near certainty of such an event if we continue down the current path. With Hinton, Bengio, Sutskever and Hassabis and Altman all agreeing that there exists a non-trivial existential risk (even if their opinions vary with respect to the magnitude), it seems more like this represents the mainstream.
- theptip 3y ago
  I think it’s not fringe amongst experts and those in the field. It’s absolutely still fringe among the general public, and I think it’s outside the Overton Window (ie politicians aren’t talking about it).
  - trashtester 3y ago
    The Overton Window applies to the general public, and maybe particular the press. And this is all over the press and other media now, both the old and new, left leaning and right leaning. I would say it's pretty well within the Overton Window. Politicians in the US are a bit behind. They probably just need to run the topic with some polls and voter study groups to decide what opinions are most popular with their voter bases.

distortionfield 3y ago

The extinction of bees, mass starvation, and the ozone hole (bonus) are all examples of alarmist takes that were course corrected. It’s sort of a weird spot to argue that they were overblown when the reason they are not a problem now is because they were addressed.

climatologist 3y ago
What does it mean to address the risk of superintelligence? There is no way to stop technological progress and AI development is just part of the same process. Moreover, the alarmism doesn't make much sense because we already have misaligned agents at odds with human values, those agents are called profit seeking corporations but I never hear the alarmists talk about putting a stop to for-profit business ventures. Do you know anyone that considers the pursuit of profits and constant exploitation of natural resources as a problem that needs to be addressed because I don't. Everyone seems very happy with the status quo and AI development is just more of the same status quo development, just corporations seeking ways to exploit and profit from digital resources. OpenAI being a perfect example of this.
- flagrant_taco 3y ago
  > There is no way to stop technological progress What makes you say this is impossible? We could simply not go down this road, there are only so many people knowledgeable enough and with access to the right hardware to make progress towards AI. They could all agree, or be compelled, to stop. We seem to have successfully halted research into cloning, though that wasn't a given and could have fallen into the same trap of having to develop it before one's enemy does.
  - climatologist 3y ago
    There are no enemies. The biosphere is a singular organism and right now people are doing their best to basically destroy all of it. The only way to prevent further damage is to reduce the human population but that's another non-starter so as long as the human population is increasing it will compel the people in charge to continue pushing for more technological "innovation" because technology is the best way to control 8B+ people[1]. Very few people are actually alarmed about the right issues (in no particular order): population size, industrial pollution, military-industrial complex, for-profit multi-national corporations, digital surveillance, factory farming, global warming, &etc. This is why the alarmism from the AI crowd seems disingenuous because AI progress is simply an extension of for-profit corporatism and exploitation applied to digital resources and to properly address the risk from AI would require addressing the actual root causes of why technological progress is misaligned with human values. 1: https://www.theguardian.com/world/2015/jul/24/france-big-brother-surveillance-powers https://www.theguardian.com/world/2015/jul/24/france-big-bro...