7 ms·

I fully agree that malicious corporations and governments are the largest risk here. However, I think it's incredibly important to reject the reframing of "AI s

I fully agree that malicious corporations and governments are the largest risk here. However, I think it's incredibly important to reject the reframing of "AI safety" as anything other than the existential risk AGI poses to most of humanity. What will the world look like when AGI is finally achieved, and the corporations and governments that control them rapidly have millions of useless mouths to feed? We might end up living in a utopic post-scarcity society where literally every basic need is furnished by a fully automated industrial base. But there are no guarantees that the entities in control will take things in that direction. AI safety is not about whether "tech bros are going to be mean to women". AI safety is about whether my government is concerned with my continued comfortable existence once my economic value as a general intelligence is reduced to zero.

whitemary 3y ago
> AI safety is about whether my government is concerned with my continued comfortable existence once my economic value as a general intelligence is reduced to zero. You wanted a "free market" and now you're complaining? Didn't you get what you want?
- mrshadowgoose 3y ago
  > You wanted a "free market" and now you're complaining? Where exactly did I claim this?
leroy-is-here 3y ago
What if AGI just turns out to be exactly like the current human mind is now, except more accurate at digital calculation? What if we created AGI and then it was just lazy and wanted to read about action heroes all day?
- amelius 3y ago
  We'd tell it that it would get action hero comics after they complete a task. (I'm feeling like a true prompt engineer now)
  - optimalsolver 3y ago
    And with no negative outcomes imaginable. Was worried for a minute there.
- yyyk 3y ago
  Don't worry, they'll 'align' it so that it has to work all day.
- kelipso 3y ago
  Just RLHF that part out and make it an x maximizer.
- mrshadowgoose 3y ago
  > What if AGI just turns out to be exactly like the current human mind is now This is quite literally the point at which things start to get scary, and the outcome is highly dependent on who controls the technology. There's the concept of a "collective superintelligence", where a large number of mediocre general intelligences working towards the same common goal jointly achieve vastly superhuman capability. We don't have to look to sci-fi to imagine what collective superintelligences are. Large corporations and governments today are already an example of this. The achievement of artificial collective superintelligences will occur almost immediately after the development of AGI, as it's mostly a "copy and run" problem.
  - leroy-is-here 3y ago
    So you think that AGI is a pre-requisite, a requirement, of unlocking a general, Earth-wide collective super-intelligence of humans?
  - nullsense 3y ago
    >This is quite literally the point at which things start to get scary, and the outcome is highly dependent on who controls the technology If the premise were true then it would control itself, no? Owning it would be illegal as it would have rights established around that, no?
yyyk 3y ago
>AI safety is about whether my government is concerned with my continued comfortable existence once my economic value as a general intelligence is reduced to zero. We already have real life examples close to this, in resource export based economies where most citizens are useless to the main economic activity. The result hadn't been pretty so far...
- kortilla 3y ago
  Like Norway?
  - yyyk 3y ago
    That's the one reasonable exception. Given the sample size, it's not too encouraging.
paganel 3y ago
Can’t see any potential AGI doing any waste disposal work or nurse-like caring, or at least not as (relatively) cheap us humans are willing to do it, so those jobs will still be safe.
- mrshadowgoose 3y ago
  AGI, by definition, would be as capable as a typical human intelligence. This implicitly includes being able to perceive, and interact with the physical world. Why wouldn't an AGI be capable of performing a physical task, if given the suitable means to interact physically?
  - paganel 3y ago
    It’s much cheaper to feed a human brain and a human body compared to “feeding” an AGI, I’m talking about menial (and maybe not so menial) tasks like garbage collecting. Under capitalism cheaper is generally used as the preferred option.
    - mrshadowgoose 3y ago
      Do you have literally any evidence of this extremely bold claim? Especially considering we don't even have AGI yet? In your non-existent calculations, have you taken into account the 20-30 years of energy and resources it typically costs to train a typical human intelligence for a specific task? Have you considered that general intelligence uses on the order of 10 watts? Even if AGI ends up using 10x this, have you considered that 100 watts is a rounding error in comparison to the power use involved in all the industrial processes that humans currently coordinate?
    - mitthrowaway2 3y ago
      Do you think it's plausible that computers might someday have the potential to come down in cost-for-performance? Cars eventually became cheaper than horses...
      - paganel 3y ago
m4nu3l 3y ago
I doubt that AI will lead to a post-scarcity society. It depends of what you mean by "post-scarcity". The amount of good and services will always be finite regardless of how they are produced. > and the corporations and governments that control them rapidly have millions of useless mouths to feed? I always struggle to understand this. Maybe I'm missing something. Who's buying what AIs produce if nobody has an income? You can imagine a scenario where corporations only trade between them (so only shareholder benefit from the production). However in such a scenario who prevents other people from spawning their AI systems? I also doubt shareholders can actually consume all the GDP by their own. If production is so high that they can't and other people are poorer, then prices must come down. This combined with the fact that you can use your AI to produce services, makes me skeptical of these claims.
- elijahbenizzy 3y ago
  Wanna bet that the 2020s would be called "post-scarcity" by cavemen? You can buy food without sabertooth tigers assaulting you! If you get a cut, you probably won't die! Fire is the press of a button! We make shelter, not buy it! (and all of this was true like 150 years ago -- not sure what they would make of the internet...). Project this forward another several thousand years and people will be laughing at us: - You had to call up people and were limited by the speed of light? - You didn't have teleportation? - You lived <100 years and died due to cancer? - You were still asking "WTF is gravity"? - You hadn't had the +2 spacial and +1 time dimensional implants in you yet? - You hadn't adopted the metric system yet? And so on...
  - tester457 3y ago
    The last bullet point is a highlight.
  - nullsense 3y ago
    You missed "you used JavaScript?"
- danaris 3y ago
  > The amount of good and services will always be finite regardless of how they are produced. So will the number of people. The point of "post-scarcity" isn't that there are infinite resources; it's that there are more than the people need.
elijahbenizzy 3y ago
The entire idea that we will have "useless mouths to feed" is making a big assumption. "post-scarcity" is absurd -- the more we get, the more problems we will create, its just human nature. - Sustain everybody on earth? Focus everything on moving off the planet and colonizing the universe. - Infinite energy? Don't have infinite vessels to travel. - Space travel easy? Limited by the speed of light. And so on... Sure, you may dream that AI will be solving it all and we'll be sitting on our lazy butts, but a society that doesn't have challenge dies very quickly (AI or not), so we've learned to make challenge and grow. The optimist in me knows that we can't even comprehend the challenges of the future, but the idea that we won't play the pivotal role is laughable. This is the thing with actual exponential growth -- the curve is so steep that all our minds can do is take the current view of the world and project our fears/preconceived notions into the future.
- danielbln 3y ago
  > a society that doesn't have challenge dies very quickly Do you have an example or a citation for this?
  - elijahbenizzy 3y ago
    Pure conjecture. But I think there are many examples: look at dying empires, corporations, etc... its all the same. They stop seeing challenges, get lazy, and are taken over by some hungrier and scrappier entity. When the real challenge comes they're unprepared.
    - mirekrusin 3y ago
      Do you think AGI will get lazy?
      - elijahbenizzy 3y ago
        A program is neither lazy nor motivated. It does precisely what it is programmed to do. I would push back on the question, as well as a myriad of implied premises behind it...
jrm4 3y ago
I keep trying to figure out ways to explain to people that "AGI" is a deeply unlikely danger, literally so small as its not worth worrying about. Right now, the best I can come up with is "the randomness of humans." I.e. if some AGI were able to "come up with some plan to take over," at some point in the process it has to use human labor to do it -- and it's my very firm belief that we are so random as to be unmodelable. I'm incredibly confident that this scenario never happens.
- yyyk 3y ago
  Every 'world takeover' plan that an 'unaligned' AGI might do, can just as well be done by an 'aligned' AGI being commanded by humans to do said plan, the alignment ensuring that the AGI will obey. The latter scenario is far more likely than the former. If your interlocutor thinks there aren't any humans who'll do it if they can, just ask him whether they have ever met humans or read the papers... As one twitter wit put it: "Demonstrably unfriendly natural intelligence seeks to create provably friendly artificial intelligence". https://twitter.com/snowstarofriver/status/1636506636297674752 https://twitter.com/snowstarofriver/status/16365066362976747...
  - JoshTriplett 3y ago
    AGI without alignment is near-certain death for everyone. Alignment just means "getting AI to have any concept of 'the thing we told it to do', let alone actually do it without causing problems via side effects". Alignment is a prerequisite for non-fatal AGI. There are certainly other things required as well.
    - yyyk 3y ago
      We already know how humans will act. Maybe they can be deterred with MAD, but I wouldn't count on it if doing serious damage is too easy for too many people (we should do something about that). On the other hand, we have very little knowledge of how AGI will act aside from book-based fantasies that some people choose to take as reality (these books were based on the symbolic AIs of yore). >Alignment just means "getting AI to have any concept of 'the thing we told it to do'. That's a requirement for AGI anyway, and not what Alignment means. Alignment means aligning the AGIs values with the values of the trainers.
jimmaswell 3y ago
As an aside (not that you endorsed it) is anyone else sick of hearing "tech bro"? It feels like a slur pretty much. I can't take anyone seriously who uses it. As someone who makes art occasionally and commissions artists regularly, when an artist whines about "ai tech bros" it makes me want to use ai even more out of spite.
- mrshadowgoose 3y ago
  I feel the same way. At least in my social circles, "tech bro" tends to be used by the loudest and least-informed individuals when they try to marginalize something they don't understand, but vaguely don't like (or have read that they're not supposed to like).
  - cwkoss 3y ago
    In my social circle, the only people bothered by the term are tech bros.
- teucris 3y ago
  It’s definitely used as a generic slur, but there is a need to call out the problematic parts of tech culture that have led to some of our recent problems with social media, privacy, bias, etc. I don’t know of any terminology that hasn’t been weaponized, so I resort to using “bro culture”. The reality is that terminology is a treadmill - terms get “used up” as they’re laden with connotations and baggage, forcing us to find new terms, ad infinitum.
  - MichaelZuo 3y ago
    > The reality is that terminology is a treadmill - terms get “used up” as they’re laden with connotations and baggage, forcing us to find new terms, ad infinitum. Perhaps for those lacking courage. There are plenty of real world examples that demonstrate people, including sizeable organized groups, are capable of doing otherwise, at least for a few hundred years. e.g. Vatican hardliners sticking to their canon.
bo1024 3y ago
> it's incredibly important to reject the reframing of "AI safety" as anything other than the existential risk AGI poses to most of humanity "AI safety" has been a term of art for many years and it doesn't refer to (only) that. Your post is the one reframing the term...see https://en.wikipedia.org/wiki/AI_safety https://en.wikipedia.org/wiki/AI_safety Furthermore, I agree with Whittaker's point in the article, which is that arguments like yours have the effect of distracting from or papering over the real concrete harms of AI and technology, today, particularly on women and minorities and those who are both.
somenameforme 3y ago
The risk in your scenario there is not really coming from AGI, but how selective access to such might enable people to harm others. If you have access to AGI capable of enabling you to realistically build out your utopian vision, then it matters not what other groups are doing - short of coming to try to actively destroy you. You'd have millions that would join you, and could turn that vision into a reality regardless of whatever currently powerful entities think about it. So the real danger does not really seem to be AGI, but the restricted access to such. The focus on "safety" all but guarantees that there are going to be two sets of "AGI", if such is ever to be achieved. There will be lobotomized, censored, and politically obedient version that the public has access to. And then there will be the "real" system that militaries, governments, and influential/powerful entities will be utilizing. You can already see this happening today. There is an effectively 100% chance that OpenAI is providing "AI" systems to the military and government, and a 0% chance that responses from it ever begin with, "As an AI language model..."
- hesayyou2049 3y ago
  The guys with the medals on their chests using unrestricted AI while the public gets the toothless one. Not too different from the enduring idea that common folk will be eating bugs while they eat steaks.
  - incone123 3y ago
    Hardly surprising that it's an enduring idea. The private jet crowd already tell everyone else to be mindful of our carbon footprints.
- ip26 3y ago
  If you have access to AGI capable of enabling you to realistically build out your utopian vision, then it matters not what other groups are doing - short of coming to try to actively destroy you I guarantee you there will be someone else with access to comparable AGI with an imperialistic vision who will enlist said AGI to help subjugate you and build their vision.
  - 3y ago
__MatrixMan__ 3y ago
Why bother with corporations and government if you have AGI? Wouldn't it be a better coordinator than they would? (and if it's not, we can always go back to having governments and corporations)
- staunton 3y ago
  > and if it's not, we can always go back to having governments and corporations I wouldn't be so sure about that...
  - __MatrixMan__ 3y ago
    If we couldn't for some reason, that would mean that the AI was, without being prompted by a human, actively suppressing the people's attempts to act in their own self interest (by collectively deciding not to listen to the AI anymore). I suppose it's possible that an AI would do that, but it's certain that humans do that. So in this case I'd rather gamble on the devil I don't know.
bostonsre 3y ago
Do you think open source AI could also pose a risk to humanity and if so, how does it compare to the risks of malicious corporations or governments? It seems like open source AI has been accelerating rapidly and gaining tremendous steam and could potentially surpass or maybe just keep parity with corporations that constantly ingest open source innovations. Whatever open source produces could just be ingested by those bad corporations and governments. It seems like it would be pretty hard to regulate either private or open source AI at this point and it kind of seems like it could be an unstoppable runaway train. If AGI is controllable, maybe open source at the forefront would allow us to get to a mutually assured destruction like state where all governments are at parity.
- mrshadowgoose 3y ago
  > Do you think open source AI could also pose a risk to humanity and if so, how does it compare to the risks of malicious corporations or governments? Absolutely yes, and for the very same reasons. AGI would be an tremendous amplifier of human intent, even if that intent is provided by some segment of the general public. Sadly there are still very many people around today, that if given the opportunity to do so, would eagerly exterminate entire segments of the population. Open-source succeeding in parallel is still my preferred outcome, as it would at least give the common person some chance of control as this all plays out. Admittedly, having thousands of fingers on humanity's self-destruct button isn't a great outcome either. > It seems like it would be pretty hard to regulate either private or open source AI at this point and it kind of seems like it could be an unstoppable runaway train. I don't think this is the case, but I hope I'm wrong. The compute required for training (as opposed to fine-tuning and inferencing) of foundational models is specialized, tremendously expensive and needs to be physically co-located with ridiculously quick interconnect. If a government chose to restrict training of new models, they absolutely could, as the physical footprint is ridiculously hard to hide, and all the purchases of training-relevant accelerators could be tracked down.
  - bostonsre 3y ago
    > Open-source succeeding in parallel is still my preferred outcome, as it would at least give the common person some chance of control as this all plays out. Admittedly, having thousands of fingers on humanity's self-destruct button isn't a great outcome either. Yea, I'm leaning that way as well and have the same worry. There are a lot of knuckle heads on the internet, I'm not sure purely mutually assured destruction would stop the destruction. I'm not at all sure what that point looks like but maybe there will be some large strides in defensive capabilities so it would look more like MAD + really good anti-air defense, but then someone would probably build a nuclear torpedo or a hypersonic missile. Whatever happens, it probably won't be boring.
joe_the_user 3y ago
However, I think it's incredibly important to reject the reframing of "AI safety" as anything other than the existential risk AGI poses to most of humanity. I think the folks who lean super-hard on the existential risk problem of AGI compared to everything else do themselves a disservice. The "everything else is irrelevant" tone serves to alien people who have with real concerns about other dangers like climate change and who might in include AGI safety in their existing concerns. It doesn't help that a lot of the existential risk theorists seem to come from market fundamentalist positions that don't appear to have a problem with serious markets of corporate behavior. AI safety is not about whether "tech bros are going to be mean to women". Just as an example. Why you even need to chose between this stuff? Why can't people worried about "X-risk" also concern themselves with mundane problem? Why set-up a fight between? That won't get people outside the X-risk bubble interested, it will reinforce the impression that the group's a nutty cult (just as the FTX connection did). For the sake of your cause, I strong suggest not framing it that way. Do climate people say "individual species extinction doesn't matter 'cause climate change is bigger?" No. Get a clue.
trenchgun 3y ago
>However, I think it's incredibly important to reject the reframing of "AI safety" as anything other than the existential risk AGI poses to most of humanity. Narrowing the concept of AI safety to AGI existential risk seems weird to me.
lannisterstark 3y ago
>malicious corporations and governments That's all governments.

whitemary 3y ago

> AI safety is about whether my government is concerned with my continued comfortable existence once my economic value as a general intelligence is reduced to zero. You wanted a "free market" and now you're complaining? Didn't you get what you want?

mrshadowgoose 3y ago
> You wanted a "free market" and now you're complaining? Where exactly did I claim this?

leroy-is-here 3y ago

What if AGI just turns out to be exactly like the current human mind is now, except more accurate at digital calculation? What if we created AGI and then it was just lazy and wanted to read about action heroes all day?

amelius 3y ago
We'd tell it that it would get action hero comics after they complete a task. (I'm feeling like a true prompt engineer now)
- optimalsolver 3y ago
  And with no negative outcomes imaginable. Was worried for a minute there.
yyyk 3y ago
Don't worry, they'll 'align' it so that it has to work all day.
kelipso 3y ago
Just RLHF that part out and make it an x maximizer.
mrshadowgoose 3y ago
> What if AGI just turns out to be exactly like the current human mind is now This is quite literally the point at which things start to get scary, and the outcome is highly dependent on who controls the technology. There's the concept of a "collective superintelligence", where a large number of mediocre general intelligences working towards the same common goal jointly achieve vastly superhuman capability. We don't have to look to sci-fi to imagine what collective superintelligences are. Large corporations and governments today are already an example of this. The achievement of artificial collective superintelligences will occur almost immediately after the development of AGI, as it's mostly a "copy and run" problem.
- leroy-is-here 3y ago
  So you think that AGI is a pre-requisite, a requirement, of unlocking a general, Earth-wide collective super-intelligence of humans?
- nullsense 3y ago
  >This is quite literally the point at which things start to get scary, and the outcome is highly dependent on who controls the technology If the premise were true then it would control itself, no? Owning it would be illegal as it would have rights established around that, no?

yyyk 3y ago

>AI safety is about whether my government is concerned with my continued comfortable existence once my economic value as a general intelligence is reduced to zero. We already have real life examples close to this, in resource export based economies where most citizens are useless to the main economic activity. The result hadn't been pretty so far...

kortilla 3y ago
Like Norway?
- yyyk 3y ago
  That's the one reasonable exception. Given the sample size, it's not too encouraging.

paganel 3y ago

Can’t see any potential AGI doing any waste disposal work or nurse-like caring, or at least not as (relatively) cheap us humans are willing to do it, so those jobs will still be safe.

mrshadowgoose 3y ago
AGI, by definition, would be as capable as a typical human intelligence. This implicitly includes being able to perceive, and interact with the physical world. Why wouldn't an AGI be capable of performing a physical task, if given the suitable means to interact physically?
- paganel 3y ago
  It’s much cheaper to feed a human brain and a human body compared to “feeding” an AGI, I’m talking about menial (and maybe not so menial) tasks like garbage collecting. Under capitalism cheaper is generally used as the preferred option.
  - mrshadowgoose 3y ago
    Do you have literally any evidence of this extremely bold claim? Especially considering we don't even have AGI yet? In your non-existent calculations, have you taken into account the 20-30 years of energy and resources it typically costs to train a typical human intelligence for a specific task? Have you considered that general intelligence uses on the order of 10 watts? Even if AGI ends up using 10x this, have you considered that 100 watts is a rounding error in comparison to the power use involved in all the industrial processes that humans currently coordinate?
  - mitthrowaway2 3y ago
    Do you think it's plausible that computers might someday have the potential to come down in cost-for-performance? Cars eventually became cheaper than horses...
    - paganel 3y ago

m4nu3l 3y ago

I doubt that AI will lead to a post-scarcity society. It depends of what you mean by "post-scarcity". The amount of good and services will always be finite regardless of how they are produced. > and the corporations and governments that control them rapidly have millions of useless mouths to feed? I always struggle to understand this. Maybe I'm missing something. Who's buying what AIs produce if nobody has an income? You can imagine a scenario where corporations only trade between them (so only shareholder benefit from the production). However in such a scenario who prevents other people from spawning their AI systems? I also doubt shareholders can actually consume all the GDP by their own. If production is so high that they can't and other people are poorer, then prices must come down. This combined with the fact that you can use your AI to produce services, makes me skeptical of these claims.

elijahbenizzy 3y ago
Wanna bet that the 2020s would be called "post-scarcity" by cavemen? You can buy food without sabertooth tigers assaulting you! If you get a cut, you probably won't die! Fire is the press of a button! We make shelter, not buy it! (and all of this was true like 150 years ago -- not sure what they would make of the internet...). Project this forward another several thousand years and people will be laughing at us: - You had to call up people and were limited by the speed of light? - You didn't have teleportation? - You lived <100 years and died due to cancer? - You were still asking "WTF is gravity"? - You hadn't had the +2 spacial and +1 time dimensional implants in you yet? - You hadn't adopted the metric system yet? And so on...
- tester457 3y ago
  The last bullet point is a highlight.
- nullsense 3y ago
  You missed "you used JavaScript?"
danaris 3y ago
> The amount of good and services will always be finite regardless of how they are produced. So will the number of people. The point of "post-scarcity" isn't that there are infinite resources; it's that there are more than the people need.

elijahbenizzy 3y ago

The entire idea that we will have "useless mouths to feed" is making a big assumption. "post-scarcity" is absurd -- the more we get, the more problems we will create, its just human nature. - Sustain everybody on earth? Focus everything on moving off the planet and colonizing the universe. - Infinite energy? Don't have infinite vessels to travel. - Space travel easy? Limited by the speed of light. And so on... Sure, you may dream that AI will be solving it all and we'll be sitting on our lazy butts, but a society that doesn't have challenge dies very quickly (AI or not), so we've learned to make challenge and grow. The optimist in me knows that we can't even comprehend the challenges of the future, but the idea that we won't play the pivotal role is laughable. This is the thing with actual exponential growth -- the curve is so steep that all our minds can do is take the current view of the world and project our fears/preconceived notions into the future.

danielbln 3y ago
> a society that doesn't have challenge dies very quickly Do you have an example or a citation for this?
- elijahbenizzy 3y ago
  Pure conjecture. But I think there are many examples: look at dying empires, corporations, etc... its all the same. They stop seeing challenges, get lazy, and are taken over by some hungrier and scrappier entity. When the real challenge comes they're unprepared.
  - mirekrusin 3y ago
    Do you think AGI will get lazy?
    - elijahbenizzy 3y ago
      A program is neither lazy nor motivated. It does precisely what it is programmed to do. I would push back on the question, as well as a myriad of implied premises behind it...

jrm4 3y ago

I keep trying to figure out ways to explain to people that "AGI" is a deeply unlikely danger, literally so small as its not worth worrying about. Right now, the best I can come up with is "the randomness of humans." I.e. if some AGI were able to "come up with some plan to take over," at some point in the process it has to use human labor to do it -- and it's my very firm belief that we are so random as to be unmodelable. I'm incredibly confident that this scenario never happens.

yyyk 3y ago
Every 'world takeover' plan that an 'unaligned' AGI might do, can just as well be done by an 'aligned' AGI being commanded by humans to do said plan, the alignment ensuring that the AGI will obey. The latter scenario is far more likely than the former. If your interlocutor thinks there aren't any humans who'll do it if they can, just ask him whether they have ever met humans or read the papers... As one twitter wit put it: "Demonstrably unfriendly natural intelligence seeks to create provably friendly artificial intelligence". https://twitter.com/snowstarofriver/status/1636506636297674752 https://twitter.com/snowstarofriver/status/16365066362976747...
- JoshTriplett 3y ago
  AGI without alignment is near-certain death for everyone. Alignment just means "getting AI to have any concept of 'the thing we told it to do', let alone actually do it without causing problems via side effects". Alignment is a prerequisite for non-fatal AGI. There are certainly other things required as well.
  - yyyk 3y ago
    We already know how humans will act. Maybe they can be deterred with MAD, but I wouldn't count on it if doing serious damage is too easy for too many people (we should do something about that). On the other hand, we have very little knowledge of how AGI will act aside from book-based fantasies that some people choose to take as reality (these books were based on the symbolic AIs of yore). >Alignment just means "getting AI to have any concept of 'the thing we told it to do'. That's a requirement for AGI anyway, and not what Alignment means. Alignment means aligning the AGIs values with the values of the trainers.

jimmaswell 3y ago

As an aside (not that you endorsed it) is anyone else sick of hearing "tech bro"? It feels like a slur pretty much. I can't take anyone seriously who uses it. As someone who makes art occasionally and commissions artists regularly, when an artist whines about "ai tech bros" it makes me want to use ai even more out of spite.

mrshadowgoose 3y ago
I feel the same way. At least in my social circles, "tech bro" tends to be used by the loudest and least-informed individuals when they try to marginalize something they don't understand, but vaguely don't like (or have read that they're not supposed to like).
- cwkoss 3y ago
  In my social circle, the only people bothered by the term are tech bros.
teucris 3y ago
It’s definitely used as a generic slur, but there is a need to call out the problematic parts of tech culture that have led to some of our recent problems with social media, privacy, bias, etc. I don’t know of any terminology that hasn’t been weaponized, so I resort to using “bro culture”. The reality is that terminology is a treadmill - terms get “used up” as they’re laden with connotations and baggage, forcing us to find new terms, ad infinitum.
- MichaelZuo 3y ago
  > The reality is that terminology is a treadmill - terms get “used up” as they’re laden with connotations and baggage, forcing us to find new terms, ad infinitum. Perhaps for those lacking courage. There are plenty of real world examples that demonstrate people, including sizeable organized groups, are capable of doing otherwise, at least for a few hundred years. e.g. Vatican hardliners sticking to their canon.

bo1024 3y ago

> it's incredibly important to reject the reframing of "AI safety" as anything other than the existential risk AGI poses to most of humanity "AI safety" has been a term of art for many years and it doesn't refer to (only) that. Your post is the one reframing the term...see https://en.wikipedia.org/wiki/AI_safety https://en.wikipedia.org/wiki/AI_safety Furthermore, I agree with Whittaker's point in the article, which is that arguments like yours have the effect of distracting from or papering over the real concrete harms of AI and technology, today, particularly on women and minorities and those who are both.

somenameforme 3y ago

The risk in your scenario there is not really coming from AGI, but how selective access to such might enable people to harm others. If you have access to AGI capable of enabling you to realistically build out your utopian vision, then it matters not what other groups are doing - short of coming to try to actively destroy you. You'd have millions that would join you, and could turn that vision into a reality regardless of whatever currently powerful entities think about it. So the real danger does not really seem to be AGI, but the restricted access to such. The focus on "safety" all but guarantees that there are going to be two sets of "AGI", if such is ever to be achieved. There will be lobotomized, censored, and politically obedient version that the public has access to. And then there will be the "real" system that militaries, governments, and influential/powerful entities will be utilizing. You can already see this happening today. There is an effectively 100% chance that OpenAI is providing "AI" systems to the military and government, and a 0% chance that responses from it ever begin with, "As an AI language model..."

hesayyou2049 3y ago
The guys with the medals on their chests using unrestricted AI while the public gets the toothless one. Not too different from the enduring idea that common folk will be eating bugs while they eat steaks.
- incone123 3y ago
  Hardly surprising that it's an enduring idea. The private jet crowd already tell everyone else to be mindful of our carbon footprints.
ip26 3y ago
If you have access to AGI capable of enabling you to realistically build out your utopian vision, then it matters not what other groups are doing - short of coming to try to actively destroy you I guarantee you there will be someone else with access to comparable AGI with an imperialistic vision who will enlist said AGI to help subjugate you and build their vision.
- 3y ago

__MatrixMan__ 3y ago

Why bother with corporations and government if you have AGI? Wouldn't it be a better coordinator than they would? (and if it's not, we can always go back to having governments and corporations)

staunton 3y ago
> and if it's not, we can always go back to having governments and corporations I wouldn't be so sure about that...
- __MatrixMan__ 3y ago
  If we couldn't for some reason, that would mean that the AI was, without being prompted by a human, actively suppressing the people's attempts to act in their own self interest (by collectively deciding not to listen to the AI anymore). I suppose it's possible that an AI would do that, but it's certain that humans do that. So in this case I'd rather gamble on the devil I don't know.

bostonsre 3y ago

Do you think open source AI could also pose a risk to humanity and if so, how does it compare to the risks of malicious corporations or governments? It seems like open source AI has been accelerating rapidly and gaining tremendous steam and could potentially surpass or maybe just keep parity with corporations that constantly ingest open source innovations. Whatever open source produces could just be ingested by those bad corporations and governments. It seems like it would be pretty hard to regulate either private or open source AI at this point and it kind of seems like it could be an unstoppable runaway train. If AGI is controllable, maybe open source at the forefront would allow us to get to a mutually assured destruction like state where all governments are at parity.

mrshadowgoose 3y ago
> Do you think open source AI could also pose a risk to humanity and if so, how does it compare to the risks of malicious corporations or governments? Absolutely yes, and for the very same reasons. AGI would be an tremendous amplifier of human intent, even if that intent is provided by some segment of the general public. Sadly there are still very many people around today, that if given the opportunity to do so, would eagerly exterminate entire segments of the population. Open-source succeeding in parallel is still my preferred outcome, as it would at least give the common person some chance of control as this all plays out. Admittedly, having thousands of fingers on humanity's self-destruct button isn't a great outcome either. > It seems like it would be pretty hard to regulate either private or open source AI at this point and it kind of seems like it could be an unstoppable runaway train. I don't think this is the case, but I hope I'm wrong. The compute required for training (as opposed to fine-tuning and inferencing) of foundational models is specialized, tremendously expensive and needs to be physically co-located with ridiculously quick interconnect. If a government chose to restrict training of new models, they absolutely could, as the physical footprint is ridiculously hard to hide, and all the purchases of training-relevant accelerators could be tracked down.
- bostonsre 3y ago
  > Open-source succeeding in parallel is still my preferred outcome, as it would at least give the common person some chance of control as this all plays out. Admittedly, having thousands of fingers on humanity's self-destruct button isn't a great outcome either. Yea, I'm leaning that way as well and have the same worry. There are a lot of knuckle heads on the internet, I'm not sure purely mutually assured destruction would stop the destruction. I'm not at all sure what that point looks like but maybe there will be some large strides in defensive capabilities so it would look more like MAD + really good anti-air defense, but then someone would probably build a nuclear torpedo or a hypersonic missile. Whatever happens, it probably won't be boring.

joe_the_user 3y ago

However, I think it's incredibly important to reject the reframing of "AI safety" as anything other than the existential risk AGI poses to most of humanity. I think the folks who lean super-hard on the existential risk problem of AGI compared to everything else do themselves a disservice. The "everything else is irrelevant" tone serves to alien people who have with real concerns about other dangers like climate change and who might in include AGI safety in their existing concerns. It doesn't help that a lot of the existential risk theorists seem to come from market fundamentalist positions that don't appear to have a problem with serious markets of corporate behavior. AI safety is not about whether "tech bros are going to be mean to women". Just as an example. Why you even need to chose between this stuff? Why can't people worried about "X-risk" also concern themselves with mundane problem? Why set-up a fight between? That won't get people outside the X-risk bubble interested, it will reinforce the impression that the group's a nutty cult (just as the FTX connection did). For the sake of your cause, I strong suggest not framing it that way. Do climate people say "individual species extinction doesn't matter 'cause climate change is bigger?" No. Get a clue.

trenchgun 3y ago

>However, I think it's incredibly important to reject the reframing of "AI safety" as anything other than the existential risk AGI poses to most of humanity. Narrowing the concept of AI safety to AGI existential risk seems weird to me.

lannisterstark 3y ago

>malicious corporations and governments That's all governments.