5 ms·

> These "safety declarations" are toothless and impossible to enforce. You can't stop AI, you need to adapt. Deepfakes are a distraction from more important th

> These "safety declarations" are toothless and impossible to enforce. You can't stop AI, you need to adapt. Deepfakes are a distraction from more important things here. The point of AI safety is "it doesn't matter who builds unaligned AGI, if someone builds it we all die". If you agree that unaligned AGI is a death sentence for humanity, then it's worth trying to stop it. If you think AGI is unlikely to come about at all, then it should be a no-op to say "don't build it, take steps to avoid building it". If you think AGI is going to come about and magically be aligned and not be a death sentence for humanity, pay close attention to the very large number of AI experts saying otherwise. https://en.wikipedia.org/wiki/P(doom) https://en.wikipedia.org/wiki/P(doom) If your argument is "but some experts don't believe that", ask yourself whether it's reasonable to say "well, experts disagree about whether this will kill us all, so we shouldn't do anything".

janalsncm 2y ago
Alignment is a completely incoherent concept. Humans do not agree on what values are correct. Why is it possible even in principle for an AI to crystallize any set of principles we all agree on?
- hollerith 2y ago
  Humans do not agree on what values are correct, but values can be averaged. So for example if a family with 5 children is on vacation, do you maintain that it is impossible even in principle for the parents to take the preferences of all 5 children into account in approximately equal measure as to what activities or non-activities to pursue? Also: are you pursuing a complete tangent or do you see your point as bearing on whether frontier AI research should be banned? (If so, I cannot tell whether you consider your point to support a ban or oppose a ban.)
  - janalsncm 2y ago
    The vast majority of harms from “AI” are actually harms from the corporations and governments that control them, who have mutually incompatible goals, getting what they want. This is why alignment folks at OpenAI are quickly learning that the first problem they need to solve is what happens when their values don’t align with the company’s (spoiler: they get fired). Therefore the actual solution is not coming up with more and more clever “guardrails” but aligning corporations and governments to human needs. In other words, politics. There are other problems like enabling new types of scams which will require political solutions. At a technical level the best these companies can do is mitigation.
    - JoshTriplett 2y ago
      > The vast majority of harms from “AI” Don't extrapolate from present harms to future harms, here. The problem AI alignment is trying to solve at a most basic level is "don't kill everyone", and even that much isn't solved yet. Solving that (or, rather, buying time to solve it) will require political solutions, in the sense of international diplomacy. But it has absolutely nothing to do with "aligning corporations", and everything to do with teaching computers things on par with (oversimplifying here) "humans are made up of atoms, and if you repurpose those atoms the humans die, don't ever do that".
      - dragonwriter 2y ago
        > The problem AI alignment is trying to solve is "don't kill everyone". No, its not. AI alignment was an active area of concern (and the fundamental problem for useful AI with significant autonomy) before cultists started trying to reduce the scope of its problem space from the wide scope of real problems it concerns to a single speculative apocalypse.
        hollerith 2y ago
        No, what actually happened is that the people you are calling the cultists coined the term alignment, which then got appropriated by the AI labs. But the genesis of the term "alignment" (as applied to AI) is a side issue. What is important is that reinforcement learning with human feedback and the other techniques used on the current crop of AIs to make it less likely that the AI will say things that embarass the owner of the AI are fundamentally different from making sure the an AI that turns out more capable than us will not kill us all or do something else awful.
        dragonwriter 2y ago
        That's simply factually untrue, and even some of the people who have become apocalypse cultists used "alignment" in the original sense before coming to advocate apocalypse as the only issue of concern.
- JoshTriplett 2y ago
  We're not talking about values on the level of politics. We're talking about values on the level of "don't destroy humanity", or even more straightforwardly, understanding "humans are made up of atoms that you may not repurpose for other purposes, doing so kills the human". These are not things that AGI inherently understands or adheres to. There might be a few humans that don't agree with even those values, but I think it's safe to presume that the general-consensus values of humanity include the above points. And AI alignment is not even close to far enough along to provide even the slightest assurances about those points.
  - JumpCrisscross 2y ago
    > We're talking about values on the level of "don't destroy humanity" Practically everyone making the argument that AGI is about to destroy humanity is (a) human and (b) working on AI. It's safe to conclude they're either stupid and suicidal or don't buy their own bunk.
    - JoshTriplett 2y ago
      The former certainly is a tempting conclusion sometimes. But also, some of the people who are making that argument were AI experts who stopped working on AI capabilities.
    - comp_throw7 2y ago
      This is not even close to true, though it's true that many of the people at the big AI labs estimate non-trivial odds of human extinction downstream of AI progress. Some of those people are working on "safety", but some are indeed working on capabilities, and have all sorts of clever reasons for why the thing they're doing is net-good (like putting worse odds on human survival if a less-careful competitor gets there first). But ultimately, most people who think we stand a decent chance of dying because of this are not working at AI labs.
  - janalsncm 2y ago
    > don't destroy humanity Do humans agree on the best way to do this? Aside from the most banal examples of what not to do, is there agreement on e.g. whether a mass extinction event is happening, not happening, or happening but actually tolerable? If the answer is no, then it is not possible for an AI to align with human values on this question. But this is a human problem, not a technical one. Solving it through technical means is not possible.
- fennecfoxy 2y ago
  Tbf I've always thought that AI could do a better job at managing our species than we are ourselves. Look at how we hate each other, how we kill and maim each other. How we let others go hungry and thirsty, without warmth or shelter. Sure AI could be worse than we are; makes for a good movie plot. But it could be a lot better than we are and it's sad that it's such a low bar to it to exceed.
hn_throwaway_99 2y ago
Sorry to be a Debbie Downer, but I think the argument the commenter is making is "It's impossible to reliably restrict AI development", so safety-declarations, etc., are useless theater. I don't think we're on "the cusp" of AGI, but I guess that just means I'm quibbling over the timeframe of what "cusp" means. I certainly think it's possible within the lifetime of people alive today, so whether it comes in 5 years or 75 years is kind of an insignificant detail. And if AGI does get built, I agree there is a significant risk to humanity. And that makes me sad, but I also don't think there is anything that can be built to stop it, certainly not some useless agreements on paper.
philomath_mn 2y ago
> it's worth trying to stop it OP's point has nothing to do with this, OP's point is that you can't stop it. The methods and materials are too diffuse and the biggest players (nation states) have a strong incentive to be first. Do you really expect China to coordinate with the West on this?
- hollerith 2y ago
  I don't expect China to coordinate with the West, but I think there is a good chance that the only reason Beijing is interested in AI beyond the AI tech they need to keep internal potential revolutionaries under surveillance is to prevent a repeat of the Century of Humiliation (which was caused by the West's technological superiority) so that if the Western governments banned AI, Beijing would be glad to ban it inside China, too.
  - philomath_mn 2y ago
    That is a massive bet based on the supposed psychology of a world super power. There are many other less-superficial reasons why Beijing may be interested in AI, plus China may not trust that we actually banned our own AI development. I wouldn't take that bet in a million years.
    - hollerith 2y ago
      You seem to think that if we refuse this bet, you are somehow safe to live out the rest of your life. (If you are old, replace "you" with "your children".) The discussion started when someone argued that even if this AI juggernaut were in fact very dangerous, there is no way to stop it. When I pushed back on the second part of that, you reject my push-back. On what basis? I hope it is not, "I just want things to keep on going the way they are," as if ignoring the AI danger somehow makes the AI danger go away.
      - philomath_mn 2y ago
        No, I do not expect things to just work out. I just think our best chance is for the US to be a leader in AI development and hope that we're able to develop it safely. I don't have a lot of confidence that this will be the case, but I think the US continuing to develop AI is the decision with the best distribution of possible outcomes.
dragonwriter 2y ago
All intelligence is unaligned. Intelligence and alignment are mutually incompatible; natural intelligence is unaligned, too. Unaligned intelligence is not a global death sentence. Fearmongering about unaligned AGI, however, is a tool to keep a tool of broad power—which AI is and will continue to grow as long before it becomes, and even if it never becomes, AGI—in the hands of a narrow, self-selected elite to make their control over everyone else insurmountable, which is also not a global death sentence, but is a global slavery sentence. (It’s also, more immediately, a tool to serve those who benefit from current AI uses which are harmful and unjust to use future speculative harms to deflect from real, present, concrete harms; and those beneficiaries are largely an overlapping elite with the group with a longer term interest in centralizing power over AI.)
- JoshTriplett 2y ago
  To be explicitly clear, in case it is ever ambiguous: "don't build unaligned AGI" is not a statement that some elite group should build unaligned AGI. It's a statement that nobody should build unaligned AGI, ever.
  - dragonwriter 2y ago
    “Don't build unaligned AGI” is an excuse to give a narrow elite exclusive control of what AI is produced under the pretext of preventing anyone from building unaligned AGI; all actionable policy under that banner fits that description. Whether or not that elite group produces AGI, much less, “unaligned AGI”, is largely immaterial to the practical impacts (also, from the perspective of anyone outside the controlling elite, what the controlling elite would view as aligned, whether or not it is a general intelligence, is unaligned; alignment is not an objective property.)
    - JoshTriplett 2y ago
      > “Don't build unaligned AGI” is an excuse False. There are people working on frontier AI who have co-opted some of the safety terminology in the interests of discrediting it, and discussions like this suggest that that strategy is working. > all actionable policy under that banner fits that description Actionable policy: "Do not do any further frontier AI capability research. Do not build any models larger or more capable than the current state of the art. Stop anyone who does as you would stop someone refining fissile materials, with no exceptions." > (also, from the perspective of anyone outside the controlling elite, what the controlling elite would view as aligned, whether or not it is a general intelligence, is unaligned; alignment is not an objective property.) You are mistaking "alignment" for things like "politics", rather than "not killing everyone".
Nasrudith 2y ago
The doomerism on AI is frankly, barking madness, a lack of sense of probability and scale, mixed with utterly batshit paranoia. It is like living paralyzed in fear of every birth, for fear that random variance will produce one baby born smarter than Einstein will be capable of developing an infinite cascade of progressively smarter babies and concluding that therefore we must stop all breeding. No matter how smart the baby super-Einstein winds up being there is no unstoppable, unopposable omnicide mechanism. You can't theorem your way out of a paper bag.
- realce 2y ago
  The problem with your analogy is that these babies are HUMANS and not some distinctly different cyber-species. The basis of "human alignment" is that we all require basically the same conditions and environment in order to live, we all feel pain and pleasure, we all need food - that's what produces any amount of human cooperation. What's being feverishly developed is the seed of a different species that doesn't share those restrictions. We've already found ourselves on a trajectory where un-employing millions or billions of people without any system to protect them afterwards is just accepted, and that's simply the first step of many in the destruction-of-empathy path that creating AI/AGI brings people down.

janalsncm 2y ago

Alignment is a completely incoherent concept. Humans do not agree on what values are correct. Why is it possible even in principle for an AI to crystallize any set of principles we all agree on?

hollerith 2y ago
Humans do not agree on what values are correct, but values can be averaged. So for example if a family with 5 children is on vacation, do you maintain that it is impossible even in principle for the parents to take the preferences of all 5 children into account in approximately equal measure as to what activities or non-activities to pursue? Also: are you pursuing a complete tangent or do you see your point as bearing on whether frontier AI research should be banned? (If so, I cannot tell whether you consider your point to support a ban or oppose a ban.)
- janalsncm 2y ago
  The vast majority of harms from “AI” are actually harms from the corporations and governments that control them, who have mutually incompatible goals, getting what they want. This is why alignment folks at OpenAI are quickly learning that the first problem they need to solve is what happens when their values don’t align with the company’s (spoiler: they get fired). Therefore the actual solution is not coming up with more and more clever “guardrails” but aligning corporations and governments to human needs. In other words, politics. There are other problems like enabling new types of scams which will require political solutions. At a technical level the best these companies can do is mitigation.
  - JoshTriplett 2y ago
    > The vast majority of harms from “AI” Don't extrapolate from present harms to future harms, here. The problem AI alignment is trying to solve at a most basic level is "don't kill everyone", and even that much isn't solved yet. Solving that (or, rather, buying time to solve it) will require political solutions, in the sense of international diplomacy. But it has absolutely nothing to do with "aligning corporations", and everything to do with teaching computers things on par with (oversimplifying here) "humans are made up of atoms, and if you repurpose those atoms the humans die, don't ever do that".
    - dragonwriter 2y ago
      > The problem AI alignment is trying to solve is "don't kill everyone". No, its not. AI alignment was an active area of concern (and the fundamental problem for useful AI with significant autonomy) before cultists started trying to reduce the scope of its problem space from the wide scope of real problems it concerns to a single speculative apocalypse.
      - hollerith 2y ago
        No, what actually happened is that the people you are calling the cultists coined the term alignment, which then got appropriated by the AI labs. But the genesis of the term "alignment" (as applied to AI) is a side issue. What is important is that reinforcement learning with human feedback and the other techniques used on the current crop of AIs to make it less likely that the AI will say things that embarass the owner of the AI are fundamentally different from making sure the an AI that turns out more capable than us will not kill us all or do something else awful.
        dragonwriter 2y ago
        That's simply factually untrue, and even some of the people who have become apocalypse cultists used "alignment" in the original sense before coming to advocate apocalypse as the only issue of concern.
JoshTriplett 2y ago
We're not talking about values on the level of politics. We're talking about values on the level of "don't destroy humanity", or even more straightforwardly, understanding "humans are made up of atoms that you may not repurpose for other purposes, doing so kills the human". These are not things that AGI inherently understands or adheres to. There might be a few humans that don't agree with even those values, but I think it's safe to presume that the general-consensus values of humanity include the above points. And AI alignment is not even close to far enough along to provide even the slightest assurances about those points.
- JumpCrisscross 2y ago
  > We're talking about values on the level of "don't destroy humanity" Practically everyone making the argument that AGI is about to destroy humanity is (a) human and (b) working on AI. It's safe to conclude they're either stupid and suicidal or don't buy their own bunk.
  - JoshTriplett 2y ago
    The former certainly is a tempting conclusion sometimes. But also, some of the people who are making that argument were AI experts who stopped working on AI capabilities.
  - comp_throw7 2y ago
    This is not even close to true, though it's true that many of the people at the big AI labs estimate non-trivial odds of human extinction downstream of AI progress. Some of those people are working on "safety", but some are indeed working on capabilities, and have all sorts of clever reasons for why the thing they're doing is net-good (like putting worse odds on human survival if a less-careful competitor gets there first). But ultimately, most people who think we stand a decent chance of dying because of this are not working at AI labs.
- janalsncm 2y ago
  > don't destroy humanity Do humans agree on the best way to do this? Aside from the most banal examples of what not to do, is there agreement on e.g. whether a mass extinction event is happening, not happening, or happening but actually tolerable? If the answer is no, then it is not possible for an AI to align with human values on this question. But this is a human problem, not a technical one. Solving it through technical means is not possible.
fennecfoxy 2y ago
Tbf I've always thought that AI could do a better job at managing our species than we are ourselves. Look at how we hate each other, how we kill and maim each other. How we let others go hungry and thirsty, without warmth or shelter. Sure AI could be worse than we are; makes for a good movie plot. But it could be a lot better than we are and it's sad that it's such a low bar to it to exceed.

hn_throwaway_99 2y ago

Sorry to be a Debbie Downer, but I think the argument the commenter is making is "It's impossible to reliably restrict AI development", so safety-declarations, etc., are useless theater. I don't think we're on "the cusp" of AGI, but I guess that just means I'm quibbling over the timeframe of what "cusp" means. I certainly think it's possible within the lifetime of people alive today, so whether it comes in 5 years or 75 years is kind of an insignificant detail. And if AGI does get built, I agree there is a significant risk to humanity. And that makes me sad, but I also don't think there is anything that can be built to stop it, certainly not some useless agreements on paper.

philomath_mn 2y ago

> it's worth trying to stop it OP's point has nothing to do with this, OP's point is that you can't stop it. The methods and materials are too diffuse and the biggest players (nation states) have a strong incentive to be first. Do you really expect China to coordinate with the West on this?

hollerith 2y ago
I don't expect China to coordinate with the West, but I think there is a good chance that the only reason Beijing is interested in AI beyond the AI tech they need to keep internal potential revolutionaries under surveillance is to prevent a repeat of the Century of Humiliation (which was caused by the West's technological superiority) so that if the Western governments banned AI, Beijing would be glad to ban it inside China, too.
- philomath_mn 2y ago
  That is a massive bet based on the supposed psychology of a world super power. There are many other less-superficial reasons why Beijing may be interested in AI, plus China may not trust that we actually banned our own AI development. I wouldn't take that bet in a million years.
  - hollerith 2y ago
    You seem to think that if we refuse this bet, you are somehow safe to live out the rest of your life. (If you are old, replace "you" with "your children".) The discussion started when someone argued that even if this AI juggernaut were in fact very dangerous, there is no way to stop it. When I pushed back on the second part of that, you reject my push-back. On what basis? I hope it is not, "I just want things to keep on going the way they are," as if ignoring the AI danger somehow makes the AI danger go away.
    - philomath_mn 2y ago
      No, I do not expect things to just work out. I just think our best chance is for the US to be a leader in AI development and hope that we're able to develop it safely. I don't have a lot of confidence that this will be the case, but I think the US continuing to develop AI is the decision with the best distribution of possible outcomes.

dragonwriter 2y ago

All intelligence is unaligned. Intelligence and alignment are mutually incompatible; natural intelligence is unaligned, too. Unaligned intelligence is not a global death sentence. Fearmongering about unaligned AGI, however, is a tool to keep a tool of broad power—which AI is and will continue to grow as long before it becomes, and even if it never becomes, AGI—in the hands of a narrow, self-selected elite to make their control over everyone else insurmountable, which is also not a global death sentence, but is a global slavery sentence. (It’s also, more immediately, a tool to serve those who benefit from current AI uses which are harmful and unjust to use future speculative harms to deflect from real, present, concrete harms; and those beneficiaries are largely an overlapping elite with the group with a longer term interest in centralizing power over AI.)

JoshTriplett 2y ago
To be explicitly clear, in case it is ever ambiguous: "don't build unaligned AGI" is not a statement that some elite group should build unaligned AGI. It's a statement that nobody should build unaligned AGI, ever.
- dragonwriter 2y ago
  “Don't build unaligned AGI” is an excuse to give a narrow elite exclusive control of what AI is produced under the pretext of preventing anyone from building unaligned AGI; all actionable policy under that banner fits that description. Whether or not that elite group produces AGI, much less, “unaligned AGI”, is largely immaterial to the practical impacts (also, from the perspective of anyone outside the controlling elite, what the controlling elite would view as aligned, whether or not it is a general intelligence, is unaligned; alignment is not an objective property.)
  - JoshTriplett 2y ago
    > “Don't build unaligned AGI” is an excuse False. There are people working on frontier AI who have co-opted some of the safety terminology in the interests of discrediting it, and discussions like this suggest that that strategy is working. > all actionable policy under that banner fits that description Actionable policy: "Do not do any further frontier AI capability research. Do not build any models larger or more capable than the current state of the art. Stop anyone who does as you would stop someone refining fissile materials, with no exceptions." > (also, from the perspective of anyone outside the controlling elite, what the controlling elite would view as aligned, whether or not it is a general intelligence, is unaligned; alignment is not an objective property.) You are mistaking "alignment" for things like "politics", rather than "not killing everyone".

Nasrudith 2y ago

The doomerism on AI is frankly, barking madness, a lack of sense of probability and scale, mixed with utterly batshit paranoia. It is like living paralyzed in fear of every birth, for fear that random variance will produce one baby born smarter than Einstein will be capable of developing an infinite cascade of progressively smarter babies and concluding that therefore we must stop all breeding. No matter how smart the baby super-Einstein winds up being there is no unstoppable, unopposable omnicide mechanism. You can't theorem your way out of a paper bag.

realce 2y ago
The problem with your analogy is that these babies are HUMANS and not some distinctly different cyber-species. The basis of "human alignment" is that we all require basically the same conditions and environment in order to live, we all feel pain and pleasure, we all need food - that's what produces any amount of human cooperation. What's being feverishly developed is the seed of a different species that doesn't share those restrictions. We've already found ourselves on a trajectory where un-employing millions or billions of people without any system to protect them afterwards is just accepted, and that's simply the first step of many in the destruction-of-empathy path that creating AI/AGI brings people down.