4 ms·

The kind of reasoning he describes has been driving me crazy when listening to various tech podcasts that touch on AI risk and social media harms. The moral pa

The kind of reasoning he describes has been driving me crazy when listening to various tech podcasts that touch on AI risk and social media harms. The moral panic argument comes up repeatedly: “people said the same thing about the {printing press, comic books, D&D…}”. But it always frustrates me because at best, these are examples that highlight our tendency to panic about things and that we’re sometimes wrong. And I do think it’s good to think about this, but it’s the weight that people attach to the argument that I don’t understand. In the face of some truly worrying facts about these new tech developments, the past panics are being held up as if they tell us everything we need to know. It’s baffling and worrisome.

lupusreal 3y ago
To the "People said X about Y and were wrong so people saying X about Z must also be wrong" form of argument, I always think of "They laughed at the Wright Brothers, but they also laughed at Bozo the Clown." Basically, the fact that people were concerned about technology we now consider normal and more or less harmless doesn't prove that people presently concerned about some other new technology are also wrong.
- dcow 3y ago
  Right. There is no proof on either side—either that AI is capable of organically morphing into a super-intelligence or that it inherently isn’t. It’s not formal. It’s people burning cycles telling each other about their intrinsic risk tolerance.
  - ryanschneider 3y ago
    Aha! This post finally made it click for me that that is exactly what’s going on, thanks for putting it so succinctly. I think one of the big differences between AI and most other previous technologies is that the potential impact different people envision is very high variance, anywhere from extremely negative to positive with almost all points in between as well. So it’s not just different risk tolerances its also that different people see the risks and rewards very differently.
  - naasking 3y ago
    > Right. There is no proof on either side—either that AI is capable of organically morphing into a super-intelligence or that it inherently isn’t. It’s not formal. You're right that there is no formal proof, but the conclusion that humans have not reached peak intelligence is pretty solid. Based on evolutionary arguments, we're broadly only as intelligent as we needed to be to reproduce. If humans are not at peak intelligence, it then follows that something more intelligent is possible. It's similar to how birds are necessarily not the fastest objects that can fly, and jet planes and rockets show that artificial systems can easily surpass the inherent limitations of evolved biological systems. The only wishy-washy part of the AI deduction is whether "intelligence" is a similar kind of property that evolved. If it is then the evolutionary argument applies, but even if it's not, and it's just a broad, mixed set of capabilities, AI is advancing on all of those fronts too, so I'm not sure you can escape the conclusion so easily.
    - dcow 3y ago
      So we agree higher intelligence is capable. But, “Is that a bad doomsday scenario?” is the next question. And then, “Is it so bad that society should fear these intelligence tools?” And to argue that, you have to get into a real abstract realm where there is zero conclusive anything (outside of philosophical thought experiments) indicating whether we should welcome or fear our new super intelligent systems.
      - naasking 3y ago
        You don't have to get that abstract. Absent some guarantee that intelligence implies inherent moral superiority, the precautionary principle would apply, and so we should plan for there being no association between the two. Some reasonable level of caution is thus warranted. People can disagree on what's "reasonable", but we should at least all agree that a danger could exist and that some caution should apply. We can't generalize from any association between intelligence and morality from humans because human morality and intelligence evolved, possibly even co-evolved, and these constraints simply don't apply to artificial systems any more than a bird's feathers apply to artificial flight.
- jimbokun 3y ago
  > "They laughed at the Wright Brothers, but they also laughed at Bozo the Clown." This is a very good compression of the article, losing very little information.
squigz 3y ago
> In the face of some truly worrying facts about these new tech developments, the past panics are being held up as if they tell us everything we need to know. I think it's more like, "In the face of some truly worrying panicking..." I think most people realize the risks, but also realize that panicking like this is the end of {art, culture, etc} isn't very helpful either, as evidenced by various similar events in human history
- ImHereToVote 3y ago
  I think your point is that panic is not effective. We should instead prepare for the externalities of a new technological development, and work to solve those. Is that more or less your position?
  - squigz 3y ago
    Yes, that pretty much sums it up.
- haswell 3y ago
  > as evidenced by various similar events in human history The issue is that people are jumping to the conclusion that the present conditions are analogous to the “false” panics of the past, while discounting “real” panics that turned out to be justified. Global warming, thalidomide, lead, the growing surveillance state, sex abuse in the Catholic church, etc. were all dismissed as panics until they turned out to be real issues. The problem is that we look back and no longer categorize these as “moral panics”, because in the end they turned out to be real and few people remember them that way. Only the “false” panics remain, which leads to some fallacious conclusions IMO. I don’t think panic is ever a good default stance on anything. But this is an issue orthogonal to the true underlying danger of whatever people are panicking about.
  - squigz 3y ago
    > The issue is that people are jumping to the conclusion that the present conditions are analogous to the “false” panics of the past, while discounting “real” panics that turned out to be justified. Again, I'm not really sure this is fair, at least in my experience. I think when most people reference these past events, it's to point out that - despite panic about those events - we managed to solve the issues involved with them, and move on.
deleted 3y ago
[deleted]
mike_hearn 3y ago
> these are examples that highlight our tendency to panic about things No, they are examples that highlight the tendency of certain sorts of people to panic about things. You're over-focusing on the objects and under-focusing on the subjects. There are two heuristics at work here: 1. People have been talking smack about AI for a long time. Remember that GPT-2 was advertised as too dangerous to release. GPT 2. Google claimed Imagen was too dangerous to release. Others claimed open source models were dangerous. Or that foundation models were too dangerous. This has been a pattern for a while. A bunch of people believed them and became AI safety extremists. Then ChatGPT made a much better version available to all, including image generation and ... it was fine. The world didn't get more dangerous. Data point: the people who said AI safety wasn't a real problem were correct. The people who said everyone should panic were wrong. It makes sense to generalize from this experience. 2. There is a certain class of person, let's call them intellectuals, whose status and income depends on their production of attention-grabbing claims. If these people stop being interesting then their careers shrivel and die. This leads such people to prioritize making claims that are optimized for virality, but not optimized for being true. Claiming the world will end if they aren't listened to is the natural evolutionary end-game here because it's impossible to be more attention-grabbing or viral than that. Data point: intellectuals specifically have made a long stream of claims about supposedly world-ending risks, which were later falsified. Data point: Since leaving psychiatry, Scott Alexander is primarily a public intellectual. it is important for him to make several interesting claims a week, which is not very easy. So it's expected for him to be attracted to claims like AI X-Risk regardless of genuine risk, and he's a very intelligent guy so can easily convince himself of things he'd like to be true (not in this case that X-Risk is high, but that X-Risk should be considered as if it might be high). Add it up and what you see is that the people in society who ignore intellectualist catastrophizing are repeatedly the winners. Sam Altman got his researcher's fantasies under control, got AI launched and are now the big winner. Other AI groups (Google) stayed in fantasy-land for far too long and were dragged kicking and screaming back to reality, and they are not the big winners.
- haswell 3y ago
  > Add it up and what you see is that the people in society who ignore intellectualist catastrophizing are repeatedly the winners. Sam Altman got his researcher's fantasies under control, got AI launched and are now the big winner. Other AI groups (Google) stayed in fantasy-land for far too long and were dragged kicking and screaming back to reality, and they are not the big winners. Entire industries are currently threatened by this tech, and large numbers of normal people are set to lose their jobs. This kind of disruption is often the cost of progress, but historically, revolutions that reshaped the workforce happened over decades or centuries, not years or months. It's unclear what happens next, but it's probably going to be bumpy. It seems your definition of "winning" is "Company X dominated the market and made a lot of money". Why should the average person impacted by this tech see this as a positive thing? In your view, does Sam Altman "winning" supersede the interests of society at large? And what would "winning" look like if the definition was broadened to include the interests of the market in which Sam operates OpenAI?

lupusreal 3y ago

To the "People said X about Y and were wrong so people saying X about Z must also be wrong" form of argument, I always think of "They laughed at the Wright Brothers, but they also laughed at Bozo the Clown." Basically, the fact that people were concerned about technology we now consider normal and more or less harmless doesn't prove that people presently concerned about some other new technology are also wrong.

dcow 3y ago
Right. There is no proof on either side—either that AI is capable of organically morphing into a super-intelligence or that it inherently isn’t. It’s not formal. It’s people burning cycles telling each other about their intrinsic risk tolerance.
- ryanschneider 3y ago
  Aha! This post finally made it click for me that that is exactly what’s going on, thanks for putting it so succinctly. I think one of the big differences between AI and most other previous technologies is that the potential impact different people envision is very high variance, anywhere from extremely negative to positive with almost all points in between as well. So it’s not just different risk tolerances its also that different people see the risks and rewards very differently.
- naasking 3y ago
  > Right. There is no proof on either side—either that AI is capable of organically morphing into a super-intelligence or that it inherently isn’t. It’s not formal. You're right that there is no formal proof, but the conclusion that humans have not reached peak intelligence is pretty solid. Based on evolutionary arguments, we're broadly only as intelligent as we needed to be to reproduce. If humans are not at peak intelligence, it then follows that something more intelligent is possible. It's similar to how birds are necessarily not the fastest objects that can fly, and jet planes and rockets show that artificial systems can easily surpass the inherent limitations of evolved biological systems. The only wishy-washy part of the AI deduction is whether "intelligence" is a similar kind of property that evolved. If it is then the evolutionary argument applies, but even if it's not, and it's just a broad, mixed set of capabilities, AI is advancing on all of those fronts too, so I'm not sure you can escape the conclusion so easily.
  - dcow 3y ago
    So we agree higher intelligence is capable. But, “Is that a bad doomsday scenario?” is the next question. And then, “Is it so bad that society should fear these intelligence tools?” And to argue that, you have to get into a real abstract realm where there is zero conclusive anything (outside of philosophical thought experiments) indicating whether we should welcome or fear our new super intelligent systems.
    - naasking 3y ago
      You don't have to get that abstract. Absent some guarantee that intelligence implies inherent moral superiority, the precautionary principle would apply, and so we should plan for there being no association between the two. Some reasonable level of caution is thus warranted. People can disagree on what's "reasonable", but we should at least all agree that a danger could exist and that some caution should apply. We can't generalize from any association between intelligence and morality from humans because human morality and intelligence evolved, possibly even co-evolved, and these constraints simply don't apply to artificial systems any more than a bird's feathers apply to artificial flight.
jimbokun 3y ago
> "They laughed at the Wright Brothers, but they also laughed at Bozo the Clown." This is a very good compression of the article, losing very little information.

squigz 3y ago

> In the face of some truly worrying facts about these new tech developments, the past panics are being held up as if they tell us everything we need to know. I think it's more like, "In the face of some truly worrying panicking..." I think most people realize the risks, but also realize that panicking like this is the end of {art, culture, etc} isn't very helpful either, as evidenced by various similar events in human history

ImHereToVote 3y ago
I think your point is that panic is not effective. We should instead prepare for the externalities of a new technological development, and work to solve those. Is that more or less your position?
- squigz 3y ago
  Yes, that pretty much sums it up.
haswell 3y ago
> as evidenced by various similar events in human history The issue is that people are jumping to the conclusion that the present conditions are analogous to the “false” panics of the past, while discounting “real” panics that turned out to be justified. Global warming, thalidomide, lead, the growing surveillance state, sex abuse in the Catholic church, etc. were all dismissed as panics until they turned out to be real issues. The problem is that we look back and no longer categorize these as “moral panics”, because in the end they turned out to be real and few people remember them that way. Only the “false” panics remain, which leads to some fallacious conclusions IMO. I don’t think panic is ever a good default stance on anything. But this is an issue orthogonal to the true underlying danger of whatever people are panicking about.
- squigz 3y ago
  > The issue is that people are jumping to the conclusion that the present conditions are analogous to the “false” panics of the past, while discounting “real” panics that turned out to be justified. Again, I'm not really sure this is fair, at least in my experience. I think when most people reference these past events, it's to point out that - despite panic about those events - we managed to solve the issues involved with them, and move on.

deleted 3y ago

[deleted]

mike_hearn 3y ago

> these are examples that highlight our tendency to panic about things No, they are examples that highlight the tendency of certain sorts of people to panic about things. You're over-focusing on the objects and under-focusing on the subjects. There are two heuristics at work here: 1. People have been talking smack about AI for a long time. Remember that GPT-2 was advertised as too dangerous to release. GPT 2. Google claimed Imagen was too dangerous to release. Others claimed open source models were dangerous. Or that foundation models were too dangerous. This has been a pattern for a while. A bunch of people believed them and became AI safety extremists. Then ChatGPT made a much better version available to all, including image generation and ... it was fine. The world didn't get more dangerous. Data point: the people who said AI safety wasn't a real problem were correct. The people who said everyone should panic were wrong. It makes sense to generalize from this experience. 2. There is a certain class of person, let's call them intellectuals, whose status and income depends on their production of attention-grabbing claims. If these people stop being interesting then their careers shrivel and die. This leads such people to prioritize making claims that are optimized for virality, but not optimized for being true. Claiming the world will end if they aren't listened to is the natural evolutionary end-game here because it's impossible to be more attention-grabbing or viral than that. Data point: intellectuals specifically have made a long stream of claims about supposedly world-ending risks, which were later falsified. Data point: Since leaving psychiatry, Scott Alexander is primarily a public intellectual. it is important for him to make several interesting claims a week, which is not very easy. So it's expected for him to be attracted to claims like AI X-Risk regardless of genuine risk, and he's a very intelligent guy so can easily convince himself of things he'd like to be true (not in this case that X-Risk is high, but that X-Risk should be considered as if it might be high). Add it up and what you see is that the people in society who ignore intellectualist catastrophizing are repeatedly the winners. Sam Altman got his researcher's fantasies under control, got AI launched and are now the big winner. Other AI groups (Google) stayed in fantasy-land for far too long and were dragged kicking and screaming back to reality, and they are not the big winners.

haswell 3y ago
> Add it up and what you see is that the people in society who ignore intellectualist catastrophizing are repeatedly the winners. Sam Altman got his researcher's fantasies under control, got AI launched and are now the big winner. Other AI groups (Google) stayed in fantasy-land for far too long and were dragged kicking and screaming back to reality, and they are not the big winners. Entire industries are currently threatened by this tech, and large numbers of normal people are set to lose their jobs. This kind of disruption is often the cost of progress, but historically, revolutions that reshaped the workforce happened over decades or centuries, not years or months. It's unclear what happens next, but it's probably going to be bumpy. It seems your definition of "winning" is "Company X dominated the market and made a lot of money". Why should the average person impacted by this tech see this as a positive thing? In your view, does Sam Altman "winning" supersede the interests of society at large? And what would "winning" look like if the definition was broadened to include the interests of the market in which Sam operates OpenAI?