6 ms·

I didn’t see a single mention of economic risk, obsolescence, or jobs. It seems like 99.9% of humans being useless mouths to feed is not a concern for x-risk fo

I didn’t see a single mention of economic risk, obsolescence, or jobs. It seems like 99.9% of humans being useless mouths to feed is not a concern for x-risk folks so long as the 0.1% are still around. Rather than being a vague, hard to imagine threat like an omniscient robot which for some reason is smart enough to build planet-to-paperclip factories but not smart enough to understand that’s not what humans would want, economic obsolescence isn’t just a known threat, it’s all but certain. Maybe it’s less exciting for the Bostrom class to think about, but it’s far more practical.

TheBlight 3y ago
While I mostly agree with your sentiment, the subject is "existential risk" and the species won't necessarily end if most humans are no longer employable. Of course, the impacts could very well be devastating.
- sigmoid10 3y ago
  Depends on whether you're talking about civilisation or the species. Killing modern civilisation is comparatively simple and wouldn't even need a superintelligent actor. But it still amounts to an existential risk for billions of people who can no longer be sustained.
  - janalsncm 3y ago
    Exactly. It doesn’t make much difference to a guy (whose family is starving to death because his potato crop on an unincorporated 5x5 plot of land behind Walmart failed) if a group of trillionaires is able to survive on Mars or not.
- cratermoon 3y ago
  Except, according to the TESCREAL[1] line, "existential risk" does mean the end of species. In their argument, the species -- human abilities -- is the important thing to preserve, and it doesn't matter if 99% of all humans die. In this view, whatever is necessary to preserve the species is OK, individuals don't matter. Of course, they all believe that THEY will be in the few that survive. 1 https://www.truthdig.com/articles/the-acronym-behind-our-wildest-ai-dreams-and-nightmares/ https://www.truthdig.com/articles/the-acronym-behind-our-wil...
mitthrowaway2 3y ago
I think your point is a good one, but this bit: > not smart enough to understand that’s not what humans would want is a misrepresentation of "the Bostrom class" viewpoint, which is that the AI's programming would not, by default, bind it to doing what humans want, even if it is capable of understanding what we want.
- goatlover 3y ago
  But why would it want to paperclip the world if we asked it to make some paperclips? Of course it's meant to be an analogy for misalignment with technology we lose control over, but it's a poor analogy. There's simply no reason for it to paperclip the world.
  - mitthrowaway2 3y ago
    Because computer programs do what we said, not what we meant. If you accidentally write an infinite loop somewhere, your program will get stuck there. If you ask for "some" paperclips, you'll possibly be fine; on the other hand if you carelessly asked it to maximize something, such as paperclips-per-month or paperclips-per-dollar, then it might go far beyond where human common sense would stop.
    - Tokumei-no-hito 3y ago
      > do whatever is necessary to guarantee consistently increasing quarterly profits Seems like a likely request
    - goatlover 3y ago
      We're talking about a Super AI, not normal computer programs. Ask ChatGPT to maximize paperclip making and see what it says.
      - mitthrowaway2 3y ago
        ChatGPT is not a general intelligence, let alone a superintelligence. Ask for paperclips and it will try to provide a plausible, satisfying, and entertaining answer. To put it another way, my parents wanted me to become a Jewish doctor but I became an atheist engineer. Was I simply not smart enough to understand what they wanted?
Obscurity4340 3y ago
Its such hubris for them to assume they can put 99% of humans out of a job, not implement a basic income, and all will be well in Rome. Kinda hard to subjugate with bread and circuses when you take away all the dough. All you are left with are circuses, and boy are they setting the stage for one.
gensym 3y ago
> It seems like 99.9% of humans being useless mouths to feed is not a concern for x-risk folks so long as the 0.1% are still around. This is one of those observations, that once you make it, you see stamped _all over_ the x-risker writings. Those folks are absolutely convinced they they and their loved ones will be part of the remaining 0.1%. (And some are pretty bad at hiding that they're actually looking forward to not having to deal with the other 99.9).
- hollerith 3y ago
  Not everything is society is about class warfare or about elites betraying non-elites. If an asteroid was hurtling towards Earth, we'd genuinely all be in the same boat: either we are all going to die as the Earth's atmosphere reaches 400 degree Fahrenheit worldwide (or is simply flung into space) or we are all going to live after we successfully deflect the asteroid. Yes, AI research has important class-warfare-type consequences, but it also has quite important Earth-killing-asteroid-type consequences, and it annoys me how often a conversation about the latter consequences on this site gets derailed into a conversation about the former consequences. Why can't you let us have a conversation about AI extinction risk on this site once in a while?
  - gensym 3y ago
    > Not everything is society is about class warfare or about elites betraying non-elites. No, but this is. Sam Altman has publicly stated that he has "guns, gold, potassium iodide, antibiotics, batteries, water, gas masks from the Israeli Defense Force, and a big patch of land in Big Sur I can fly to.” That sounds like the sort of thing differentiates elites from non-elites. (And the purpose of the guns? I'm pretty sure it's not to shoot robots or viruses). > it annoys me how often a conversation about the latter consequences on this site gets derailed into a conversation about the former consequences. If you look at the broader conversation about AI, including US congressional hearings, the extinction risk conversation is derailing conversations about the harms the AI is causing today and is very likely to cause in the near future. > Why can't you let us have a conversation about AI extinction risk on this site once in a while? No one is stopping you.
logicchains 3y ago
> It seems like 99.9% of humans being useless mouths to feed is not a concern for x-risk folks so long as the 0.1% are still around Humanity's already been through something like this and come out fine: slave states (like in Ancient Greece and later Rome) where the vast majority of the labor was done by slaves and the slave-owning populations lived relatively care-free lives. There's a reason ancient Greece produced so much philosophy: because they had so much free time on their hands due to slaves doing all the work.
- moralestapia 3y ago
  I don't think you got GPs point, but the complete opposite. The slaves on your example were useful, GP (and others) are anticipating a world where most of the non-ruling class is useless. What happens then? Will good samaritanism and humanitarianism kick in? I doubt it.
  - logicchains 3y ago
    The slaves were useful, the slave-owners were "useless" (because why pay one to do the work when a slave could do it for free?). How is a society in which 99% of people don't need to work because slaves do all the work for them different from one in which 99% of people don't need to work because robots do the work for them?
    - moralestapia 3y ago
      There has never been a situation in history where 99% of people live off a much smaller 1% of slaves. That's insane.
      - logicchains 3y ago
        I didn't mean 99% living off 1%; I mean 100% of the society living off slave labor and only around 1% doing any real work. The slaves didn't count as people back then, similar to the robots in these discussions. In ancient Sparta, for instance, it's estimated that there were 5-10 slaves for every Spartan. That means no Spartan had to work.
logicchains 3y ago
For an AI to be capable of doing 99.9% of human jobs, it's going to need to be as capable of critical thinking and decision making and dexterity as a human. Why do you think such a being would be willing to work for free (or at least for much cheaper than a human)? You're making the pretty big assumption that it's possible to make an AI that's equal to a human in every way but completely subservient, why are you so sure that's possible?
- janalsncm 3y ago
  You don’t need to make robots that are equivalent to humans. What’s important is that robots can equal and surpass the relevant economic outputs of humans. That is, it doesn’t matter that a self driving car doesn’t have a personality. By the way this is also why GDP (and gdp/capita) is a terrible measurement for a society. You can double GDP growth by firing your factory workers and replacing them with faster machines, but that doesn’t necessarily benefit those workers. And all of the gains can go to the top.
austinjp 3y ago
It's a shame that "Marxism" is a trigger-word for some folks, because there's been a lot of writing on this problem for decades. https://en.m.wikipedia.org/wiki/Means_of_production https://en.m.wikipedia.org/wiki/Means_of_production AI and associated tech are currently implemented as an accelerated form of industrialisation. The rich (the overwhelming majority anyway) have always sought to consolidate the security of their advantage. The multipliers in this equation are growing exponentially, and as 'the means of production' grow more powerful and all-encompassing, there is no reason to believe that today's tech CEOs have any incentive to relinquish their advantages.

TheBlight 3y ago

While I mostly agree with your sentiment, the subject is "existential risk" and the species won't necessarily end if most humans are no longer employable. Of course, the impacts could very well be devastating.

sigmoid10 3y ago
Depends on whether you're talking about civilisation or the species. Killing modern civilisation is comparatively simple and wouldn't even need a superintelligent actor. But it still amounts to an existential risk for billions of people who can no longer be sustained.
- janalsncm 3y ago
  Exactly. It doesn’t make much difference to a guy (whose family is starving to death because his potato crop on an unincorporated 5x5 plot of land behind Walmart failed) if a group of trillionaires is able to survive on Mars or not.
cratermoon 3y ago
Except, according to the TESCREAL[1] line, "existential risk" does mean the end of species. In their argument, the species -- human abilities -- is the important thing to preserve, and it doesn't matter if 99% of all humans die. In this view, whatever is necessary to preserve the species is OK, individuals don't matter. Of course, they all believe that THEY will be in the few that survive. 1 https://www.truthdig.com/articles/the-acronym-behind-our-wildest-ai-dreams-and-nightmares/ https://www.truthdig.com/articles/the-acronym-behind-our-wil...

mitthrowaway2 3y ago

I think your point is a good one, but this bit: > not smart enough to understand that’s not what humans would want is a misrepresentation of "the Bostrom class" viewpoint, which is that the AI's programming would not, by default, bind it to doing what humans want, even if it is capable of understanding what we want.

goatlover 3y ago
But why would it want to paperclip the world if we asked it to make some paperclips? Of course it's meant to be an analogy for misalignment with technology we lose control over, but it's a poor analogy. There's simply no reason for it to paperclip the world.
- mitthrowaway2 3y ago
  Because computer programs do what we said, not what we meant. If you accidentally write an infinite loop somewhere, your program will get stuck there. If you ask for "some" paperclips, you'll possibly be fine; on the other hand if you carelessly asked it to maximize something, such as paperclips-per-month or paperclips-per-dollar, then it might go far beyond where human common sense would stop.
  - Tokumei-no-hito 3y ago
    > do whatever is necessary to guarantee consistently increasing quarterly profits Seems like a likely request
  - goatlover 3y ago
    We're talking about a Super AI, not normal computer programs. Ask ChatGPT to maximize paperclip making and see what it says.
    - mitthrowaway2 3y ago
      ChatGPT is not a general intelligence, let alone a superintelligence. Ask for paperclips and it will try to provide a plausible, satisfying, and entertaining answer. To put it another way, my parents wanted me to become a Jewish doctor but I became an atheist engineer. Was I simply not smart enough to understand what they wanted?

Obscurity4340 3y ago

Its such hubris for them to assume they can put 99% of humans out of a job, not implement a basic income, and all will be well in Rome. Kinda hard to subjugate with bread and circuses when you take away all the dough. All you are left with are circuses, and boy are they setting the stage for one.

gensym 3y ago

> It seems like 99.9% of humans being useless mouths to feed is not a concern for x-risk folks so long as the 0.1% are still around. This is one of those observations, that once you make it, you see stamped _all over_ the x-risker writings. Those folks are absolutely convinced they they and their loved ones will be part of the remaining 0.1%. (And some are pretty bad at hiding that they're actually looking forward to not having to deal with the other 99.9).

hollerith 3y ago
Not everything is society is about class warfare or about elites betraying non-elites. If an asteroid was hurtling towards Earth, we'd genuinely all be in the same boat: either we are all going to die as the Earth's atmosphere reaches 400 degree Fahrenheit worldwide (or is simply flung into space) or we are all going to live after we successfully deflect the asteroid. Yes, AI research has important class-warfare-type consequences, but it also has quite important Earth-killing-asteroid-type consequences, and it annoys me how often a conversation about the latter consequences on this site gets derailed into a conversation about the former consequences. Why can't you let us have a conversation about AI extinction risk on this site once in a while?
- gensym 3y ago
  > Not everything is society is about class warfare or about elites betraying non-elites. No, but this is. Sam Altman has publicly stated that he has "guns, gold, potassium iodide, antibiotics, batteries, water, gas masks from the Israeli Defense Force, and a big patch of land in Big Sur I can fly to.” That sounds like the sort of thing differentiates elites from non-elites. (And the purpose of the guns? I'm pretty sure it's not to shoot robots or viruses). > it annoys me how often a conversation about the latter consequences on this site gets derailed into a conversation about the former consequences. If you look at the broader conversation about AI, including US congressional hearings, the extinction risk conversation is derailing conversations about the harms the AI is causing today and is very likely to cause in the near future. > Why can't you let us have a conversation about AI extinction risk on this site once in a while? No one is stopping you.

logicchains 3y ago

> It seems like 99.9% of humans being useless mouths to feed is not a concern for x-risk folks so long as the 0.1% are still around Humanity's already been through something like this and come out fine: slave states (like in Ancient Greece and later Rome) where the vast majority of the labor was done by slaves and the slave-owning populations lived relatively care-free lives. There's a reason ancient Greece produced so much philosophy: because they had so much free time on their hands due to slaves doing all the work.

moralestapia 3y ago
I don't think you got GPs point, but the complete opposite. The slaves on your example were useful, GP (and others) are anticipating a world where most of the non-ruling class is useless. What happens then? Will good samaritanism and humanitarianism kick in? I doubt it.
- logicchains 3y ago
  The slaves were useful, the slave-owners were "useless" (because why pay one to do the work when a slave could do it for free?). How is a society in which 99% of people don't need to work because slaves do all the work for them different from one in which 99% of people don't need to work because robots do the work for them?
  - moralestapia 3y ago
    There has never been a situation in history where 99% of people live off a much smaller 1% of slaves. That's insane.
    - logicchains 3y ago
      I didn't mean 99% living off 1%; I mean 100% of the society living off slave labor and only around 1% doing any real work. The slaves didn't count as people back then, similar to the robots in these discussions. In ancient Sparta, for instance, it's estimated that there were 5-10 slaves for every Spartan. That means no Spartan had to work.

logicchains 3y ago

For an AI to be capable of doing 99.9% of human jobs, it's going to need to be as capable of critical thinking and decision making and dexterity as a human. Why do you think such a being would be willing to work for free (or at least for much cheaper than a human)? You're making the pretty big assumption that it's possible to make an AI that's equal to a human in every way but completely subservient, why are you so sure that's possible?

janalsncm 3y ago
You don’t need to make robots that are equivalent to humans. What’s important is that robots can equal and surpass the relevant economic outputs of humans. That is, it doesn’t matter that a self driving car doesn’t have a personality. By the way this is also why GDP (and gdp/capita) is a terrible measurement for a society. You can double GDP growth by firing your factory workers and replacing them with faster machines, but that doesn’t necessarily benefit those workers. And all of the gains can go to the top.

austinjp 3y ago

It's a shame that "Marxism" is a trigger-word for some folks, because there's been a lot of writing on this problem for decades. https://en.m.wikipedia.org/wiki/Means_of_production https://en.m.wikipedia.org/wiki/Means_of_production AI and associated tech are currently implemented as an accelerated form of industrialisation. The rich (the overwhelming majority anyway) have always sought to consolidate the security of their advantage. The multipliers in this equation are growing exponentially, and as 'the means of production' grow more powerful and all-encompassing, there is no reason to believe that today's tech CEOs have any incentive to relinquish their advantages.