7 ms·

I'd like to remind people of this single-sentence statement: > Mitigating the risk of extinction from AI should be a global priority alongside other societal-s

I'd like to remind people of this single-sentence statement: > Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. [0] Signed by Demis Hassabis, Sam Altman, and Bill Gates among others [0] https://www.safe.ai/work/statement-on-ai-risk https://www.safe.ai/work/statement-on-ai-risk

preommr 3y ago
I am 90% sure that Altman's views on such issues are a machiavelian attempt at regulatory capture.
- Clubber 3y ago
  This; Gates too. It's becoming an obvious attempt to garner support of the government restricting the use of AI to large players. None of the entrenched interests want any disruption that AI might cause what so ever. Replace "AI" in all the doomsaying with "the internet," and it will become clearer.
- randomcarbloke 3y ago
  the fear-mongers are almost always either ideologues (academic or otherwise), or angling at regulatory capture.
i5heu 3y ago
I’d like to remind people that these ppl have no more knowledge about AGI as anyone else on this planet since there is no knowledge yet and everything they say about this topic is as relevant as something every other random person can say.
- reducesuffering 3y ago
  Yes, let's go with the random layperson knowledge of an HN commenter compared to the people smart enough to actually build all the AGI tech. 50/50 coin toss, I'm sure. Dario Amodei (Anthropic CEO, builder of Claude 3 Opus): "My chance that something goes, you know, really catastrophically wrong on the scale of human civilization, might be between 10 - 25%"
  - i5heu 3y ago
    They did not build a AGI yet so they have build as many AGIs as anyone else and therefore they are also laypersons regarding the effects of AGI on humanity.
    - reducesuffering 3y ago
      What do you call something humans built (artificial) that can solve all topics of AP tests (general) with most being 5/5, better than almost all humans (intelligence)?
      - roarcher 3y ago
        I call it an LLM, not AGI. Consider this scenario: - Ask ChatGPT to produce some output meeting certain criteria - ChatGPT responds, but the output does not meet the criteria in one or more obvious ways - Point this out and ask for a corrected version - ChatGPT says "I apologize, here's the corrected version:" and spits out a nearly identical answer with the same obvious mistake. - Repeat steps 3 and 4 ad nauseam. I experience this frequently when I ask it to do anything somewhat outside the norms of its training. It cannot correct itself or even see its own mistake. It cannot even conclude that it doesn't have the ability to do what I've asked and tell me so. Whatever sort of intelligence it may have, I would categorize it as not "general" enough to fall under AGI. Being able to answer AP test questions cannot be a sufficient measure of generality if a model can do it with flying colors but still fail at far more basic tasks.
        hyperadvanced 3y ago
        I’d say that LLM’s inability to handle DSL’s in particular is an interesting question on the topic of knowledge and language. Should a generally intelligent entity be able to quickly figure out a new dialect with little background? If I were to air-drop you into an extremely foreign nation, how long would it take you to organically decode the local language and be able to meaningfully craft expressions in it? I’m not sure if humans are “generally intelligent” but a common bar for AGI is for it to be able to beat or meet average humans at normal tasks. I’m not sure if something like writing perfect k8s yaml should be a requirement, though I do agree that LLM’s inability to do word puzzles or relatively straightforward math should disqualify them.
- classified 3y ago
  They are just banking on their putative status as influencers to grab more influence.
- ecoquant 3y ago
  [flagged]
tivert 3y ago
So, is this extinction or not? > We will be the first species ever to design our own descendants. My guess is that we can either be the biological bootloader for digital intelligence and then fade into an evolutionary tree branch, or we can figure out what a successful merge looks like. - Sam Altman (https://blog.samaltman.com/the-merge https://blog.samaltman.com/the-merge) I would say it is.
- reducesuffering 3y ago
  Even worse, some factions literally advocate for killing all humans in the pursuit of a synthetic intelligence, and YC's Garry Tan is advocating for these people! Beff Jezos (e/acc founder): "e/acc has no particular allegiance to the biological substrate for intelligence and life, in contrast to transhumanism Parts of e/acc (e.g. Beff) consider ourselves post-humanists; in order to spread to the stars, the light of consciousness/intelligence will have to be transduced to non-biological substrates" https://beff.substack.com/p/notes-on-eacc-principles-and-tenets https://beff.substack.com/p/notes-on-eacc-principles-and-ten...
  - XorNot 3y ago
    Is this particularly controversial? Isn't this just saying "sending not-biological bodies to the stars would be a heck of a lot easier"? Which, it would. The hard disk containing me is going to be a lot easier to keep intact then the biology.
  - halflings 3y ago
    This is the definition of strawman. "Advocate for killing all humans" sounds like someone advocating for a genocide, but instead it's just the same transhumanist thinking (which Yudkowsky also believes in, FYI)
    - reducesuffering 3y ago
      Commenter: "If AI replaced us, it's fine because they're a worthy descendants?" Beff Jezos / Verdon: "Personally, yes." Yudkowsky is transhumanist in that he is hopeful people could voluntarily extend their biological self, but isn't advocating for the elimination of all biological selves in pursuit of other artificial intelligences. Voluntary /= involuntary eradication
antod 3y ago
Part of me wonders if these people are intentionally framing the debate around ethics and potential risks as longer term extinction level problems to distract from the nearer term damage caused by them accelerating the economic inequality of the AI have-nots while they make themselves even richer.
- hyperadvanced 3y ago
  The more I look at AI, the more I get the feeling that this is true. Spinning an intriguing sci-fi tale of apocalypse and extinction is relatively easy and serves to obfuscate any nearer-term concerns about AI behind a hypothetical that sucks the air out of the room. That said, I don’t think that it’s necessarily disingenuous so much as it is myopic - to them of course AI is exciting, world-changing, and profitable, but they (willfully or not) fail to see the downsides or upsides for anyone else but them. Perhaps in the minds of the ultra-rich AI proponents, solutions to nearer-term effects of their tech are someone else’s problem, but the “existential risks” are “everyone’s” problem.
- statuslover9000 3y ago
  The short-term effect is a harbinger of the long-term risk, since capitalism doesn’t inherently care for people who don’t provide economic value. Once superintelligent AI arises, none of us will have value within this system. Even the largest current capital holders will have a hard time holding on to it with an enormous intelligence disadvantage. The logical endpoint is the subjugation or elimination of our species, unless we find a new economic system with human value at its core.
  - semi-extrinsic 3y ago
    There are a lot of assumptions going on here. One of them is that superintelligent AI will arise. We have no reason to believe this will happen in our lifetimes. I posit that we are about as close to superintelligent AI as James Watt was to nuclear fusion. The other assumption is that wealth and power are distributed according to intelligence. This is obviously false, wealth and power are largely distributed according to who you or your father plays golf with. As long as AIs don't play golf and don't have fathers, we are quite safe.
serf 3y ago
given who the people are that signed it really just comes off more like an attempt at creating a regulatory moat around the territory they got to first. "This is extinction level important, so all you people who aren't us need to be careful meddling with the stuff we're meddling with for profit."
mistermann 3y ago
Mitigating the risk of unnecessary global death due to the curious suboptimal manner in which humans have "decided" ("democratically", dontcha know) to distribute wealth on the other hand, nothing to see here!

preommr 3y ago

I am 90% sure that Altman's views on such issues are a machiavelian attempt at regulatory capture.

Clubber 3y ago
This; Gates too. It's becoming an obvious attempt to garner support of the government restricting the use of AI to large players. None of the entrenched interests want any disruption that AI might cause what so ever. Replace "AI" in all the doomsaying with "the internet," and it will become clearer.
randomcarbloke 3y ago
the fear-mongers are almost always either ideologues (academic or otherwise), or angling at regulatory capture.

i5heu 3y ago

I’d like to remind people that these ppl have no more knowledge about AGI as anyone else on this planet since there is no knowledge yet and everything they say about this topic is as relevant as something every other random person can say.

reducesuffering 3y ago
Yes, let's go with the random layperson knowledge of an HN commenter compared to the people smart enough to actually build all the AGI tech. 50/50 coin toss, I'm sure. Dario Amodei (Anthropic CEO, builder of Claude 3 Opus): "My chance that something goes, you know, really catastrophically wrong on the scale of human civilization, might be between 10 - 25%"
- i5heu 3y ago
  They did not build a AGI yet so they have build as many AGIs as anyone else and therefore they are also laypersons regarding the effects of AGI on humanity.
  - reducesuffering 3y ago
    What do you call something humans built (artificial) that can solve all topics of AP tests (general) with most being 5/5, better than almost all humans (intelligence)?
    - roarcher 3y ago
      I call it an LLM, not AGI. Consider this scenario: - Ask ChatGPT to produce some output meeting certain criteria - ChatGPT responds, but the output does not meet the criteria in one or more obvious ways - Point this out and ask for a corrected version - ChatGPT says "I apologize, here's the corrected version:" and spits out a nearly identical answer with the same obvious mistake. - Repeat steps 3 and 4 ad nauseam. I experience this frequently when I ask it to do anything somewhat outside the norms of its training. It cannot correct itself or even see its own mistake. It cannot even conclude that it doesn't have the ability to do what I've asked and tell me so. Whatever sort of intelligence it may have, I would categorize it as not "general" enough to fall under AGI. Being able to answer AP test questions cannot be a sufficient measure of generality if a model can do it with flying colors but still fail at far more basic tasks.
      - hyperadvanced 3y ago
        I’d say that LLM’s inability to handle DSL’s in particular is an interesting question on the topic of knowledge and language. Should a generally intelligent entity be able to quickly figure out a new dialect with little background? If I were to air-drop you into an extremely foreign nation, how long would it take you to organically decode the local language and be able to meaningfully craft expressions in it? I’m not sure if humans are “generally intelligent” but a common bar for AGI is for it to be able to beat or meet average humans at normal tasks. I’m not sure if something like writing perfect k8s yaml should be a requirement, though I do agree that LLM’s inability to do word puzzles or relatively straightforward math should disqualify them.
classified 3y ago
They are just banking on their putative status as influencers to grab more influence.
ecoquant 3y ago
[flagged]

tivert 3y ago

So, is this extinction or not? > We will be the first species ever to design our own descendants. My guess is that we can either be the biological bootloader for digital intelligence and then fade into an evolutionary tree branch, or we can figure out what a successful merge looks like. - Sam Altman (https://blog.samaltman.com/the-merge https://blog.samaltman.com/the-merge) I would say it is.

reducesuffering 3y ago
Even worse, some factions literally advocate for killing all humans in the pursuit of a synthetic intelligence, and YC's Garry Tan is advocating for these people! Beff Jezos (e/acc founder): "e/acc has no particular allegiance to the biological substrate for intelligence and life, in contrast to transhumanism Parts of e/acc (e.g. Beff) consider ourselves post-humanists; in order to spread to the stars, the light of consciousness/intelligence will have to be transduced to non-biological substrates" https://beff.substack.com/p/notes-on-eacc-principles-and-tenets https://beff.substack.com/p/notes-on-eacc-principles-and-ten...
- XorNot 3y ago
  Is this particularly controversial? Isn't this just saying "sending not-biological bodies to the stars would be a heck of a lot easier"? Which, it would. The hard disk containing me is going to be a lot easier to keep intact then the biology.
- halflings 3y ago
  This is the definition of strawman. "Advocate for killing all humans" sounds like someone advocating for a genocide, but instead it's just the same transhumanist thinking (which Yudkowsky also believes in, FYI)
  - reducesuffering 3y ago
    Commenter: "If AI replaced us, it's fine because they're a worthy descendants?" Beff Jezos / Verdon: "Personally, yes." Yudkowsky is transhumanist in that he is hopeful people could voluntarily extend their biological self, but isn't advocating for the elimination of all biological selves in pursuit of other artificial intelligences. Voluntary /= involuntary eradication

antod 3y ago

Part of me wonders if these people are intentionally framing the debate around ethics and potential risks as longer term extinction level problems to distract from the nearer term damage caused by them accelerating the economic inequality of the AI have-nots while they make themselves even richer.

hyperadvanced 3y ago
The more I look at AI, the more I get the feeling that this is true. Spinning an intriguing sci-fi tale of apocalypse and extinction is relatively easy and serves to obfuscate any nearer-term concerns about AI behind a hypothetical that sucks the air out of the room. That said, I don’t think that it’s necessarily disingenuous so much as it is myopic - to them of course AI is exciting, world-changing, and profitable, but they (willfully or not) fail to see the downsides or upsides for anyone else but them. Perhaps in the minds of the ultra-rich AI proponents, solutions to nearer-term effects of their tech are someone else’s problem, but the “existential risks” are “everyone’s” problem.
statuslover9000 3y ago
The short-term effect is a harbinger of the long-term risk, since capitalism doesn’t inherently care for people who don’t provide economic value. Once superintelligent AI arises, none of us will have value within this system. Even the largest current capital holders will have a hard time holding on to it with an enormous intelligence disadvantage. The logical endpoint is the subjugation or elimination of our species, unless we find a new economic system with human value at its core.
- semi-extrinsic 3y ago
  There are a lot of assumptions going on here. One of them is that superintelligent AI will arise. We have no reason to believe this will happen in our lifetimes. I posit that we are about as close to superintelligent AI as James Watt was to nuclear fusion. The other assumption is that wealth and power are distributed according to intelligence. This is obviously false, wealth and power are largely distributed according to who you or your father plays golf with. As long as AIs don't play golf and don't have fathers, we are quite safe.

serf 3y ago

given who the people are that signed it really just comes off more like an attempt at creating a regulatory moat around the territory they got to first. "This is extinction level important, so all you people who aren't us need to be careful meddling with the stuff we're meddling with for profit."

mistermann 3y ago

Mitigating the risk of unnecessary global death due to the curious suboptimal manner in which humans have "decided" ("democratically", dontcha know) to distribute wealth on the other hand, nothing to see here!