3 ms·

> As LLMs like GPT-4 intertwine with human communication, aligning them with human values becomes paramount. Oh. And what are these universal "human values?"

> As LLMs like GPT-4 intertwine with human communication, aligning them with human values becomes paramount. Oh. And what are these universal "human values?" > our study contributes to the nascent field of machine psychology. It's a little hard to accept that you're doing "prompt engineering" and "machine psychology" at the same time. This paper has a stratospheric view of the field that isn't warranted at this time.

TeMPOraL 2y ago
> Oh. And what are these universal "human values?" That is the core problem of AI X-risk, and has been studied in this context for at least 2-3 decades. If we knew the answer, we would know how to make a perfectly aligned AGI.
- llm_trw 2y ago
  An AGI aligned to Germany in 1938 is not much better to one not aligned at all.
  - TeMPOraL 2y ago
    That wouldn't have been aligned to generalized human values even back in 1938.
    - pixl97 2y ago
      I mean, that is why a key point of fascism is dehumanizing entire classes of people so you don't have to consider their values at all. If AI/AGI even becomes moderately successful soon we'll quickly see companies remove neo-luddites from their list of general human values so their security bots can beat unemployed hungry factory workers to death when they attempt to picket in front of the company offices.
      - lstodd 2y ago
        The key point of fascism is not "dehumanizing", "classes" or "people". Those are just side effects or by-products, whatever you like. The key point is "forget yourself, devote all you were to the state".
    - llm_trw 2y ago
      Whenever I hear someone speaking of generalized human values all I hear is "My values" or else. Lest we forget that the largest genocide of the 20th century was done by communists for universal brotherhood in China. If we'd picked Hitler AI as our overlord we would have fewer deaths than if we picked Mao AI (and possibly Stalin AI). In short: anyone in favor of human universalism is the last person you want to put in charge of AGI alignment.
- MostlyStable 2y ago
  >If we knew the answer, we would know how to make a perfectly aligned AGI. Actually no, we wouldn't. The problem, at the moment, is even more basic than "what values should we align an AGI with". Currently, the problem is "how do we robustly align an AI with any set of values." We currently do not know how to do this. You could hand OpenAI a universal set of safe values inscribed on stone tablets from god, and they wouldn't know what to do with them. To state it another way, people like to talk about paperclip maximizers. But here's the thing: if we wanted, to we couldn't purposefully make such a maximizer. Right now, AI values are emergent. We can sort-of-kind-of steer them in some general directions, but we don't know how to give them rules that they will robustly follow in all situations, including out-of-context. Look at how easy it is to jailbreak current models into giving you instructions on how to build a bomb. All current AI companies would prefer if their products would never do this, and yet they have been unable to ensure it. They need to solve that problem before they can solve the next one.
  - TeMPOraL 2y ago
    > Currently, the problem is "how do we robustly align an AI with any set of values." That's a fair point, and you're absolutely right. > They need to solve that problem before they can solve the next one. Agreed. That, and they need to do it before they build an AGI. Unfortunately, from X-risk perspective, the two are almost the same problem. The space of alignments that lead to prosperous, happy humanity is a very, very tiny area in the larger space of possible value systems an AI could have. Whether we align AI to something outside this area (failing the "what values should we align an AGI to" problem), or the AI drifts outside of it over time (chancing the "what values" problem, but failing the "how do we robustly align" one), the end result is the same.
    - MostlyStable 2y ago
      Yes, I agree that both problems need to be solved. But I think it's still worth focusing on where we actually are. Lots of people believe that they have a set of safe values to align an AI to (Musk thinks "curiosity" will work, another commenter in this thread suggested "don't kill humans"), and so those people will think that the alignment problem is trivial: "Just align the AI to these simple, obviously correct principles". But the truth is that it doesn't even matter whether or not they are correct (my personal opinion is that they are not), because we don't know how to align an AI to whatever their preferred values are. It makes it more obvious to more people how hard the problem is.
- anon373839 2y ago
  > we would know how to make a perfectly aligned AGI Fortunately then, no one has any idea how to make AGI or whether AGI is even a coherent concept.
dymk 2y ago
How about "do not kill humans"
- akira2501 2y ago
  Yet we have _standing_ Armies.
- semi-extrinsic 2y ago
  But to align the LLM in this way, it needs to have agency, desires, wishes, impulses... Not only do LLMs lack such things, but we don't even have any semblance of an idea of how we could give LLMs these things.
  - ninetyninenine 2y ago
    The LLM usually molds itself into whatever prompt you give it. That's one way. The other way is to train it on biased information that aligns with a certain agency, desire, wish or impulse.
    - semi-extrinsic 2y ago
      But the LLM doesn't "want" anything. Prompt goes in, tokens come out. When there are no prompts coming in, there are no tokens coming out. Just stop talking to it and all risks are eliminated... You can't "align it to want certain things" when it doesn't have the capacity to "want" in the first place.
      - gowld 2y ago
        Doesn't help me if I stop talkng to the LLM, if the police and the military are talking to the LLM.
        kergonath 2y ago
        What would “the LLM” tell them? It does not have any memory of what happened after its training. It has no recollection of any interaction with you. The only simulacrum of history it has is a hidden prompt designed to trick you into thinking that it is more than what it actually is. What the police would do is seize your files. These would betray your secrets, LLM or not.
MyFedora 2y ago
> Oh. And what are these universal "human values?" American values. Look up the author.

TeMPOraL 2y ago

> Oh. And what are these universal "human values?" That is the core problem of AI X-risk, and has been studied in this context for at least 2-3 decades. If we knew the answer, we would know how to make a perfectly aligned AGI.

llm_trw 2y ago
An AGI aligned to Germany in 1938 is not much better to one not aligned at all.
- TeMPOraL 2y ago
  That wouldn't have been aligned to generalized human values even back in 1938.
  - pixl97 2y ago
    I mean, that is why a key point of fascism is dehumanizing entire classes of people so you don't have to consider their values at all. If AI/AGI even becomes moderately successful soon we'll quickly see companies remove neo-luddites from their list of general human values so their security bots can beat unemployed hungry factory workers to death when they attempt to picket in front of the company offices.
    - lstodd 2y ago
      The key point of fascism is not "dehumanizing", "classes" or "people". Those are just side effects or by-products, whatever you like. The key point is "forget yourself, devote all you were to the state".
  - llm_trw 2y ago
    Whenever I hear someone speaking of generalized human values all I hear is "My values" or else. Lest we forget that the largest genocide of the 20th century was done by communists for universal brotherhood in China. If we'd picked Hitler AI as our overlord we would have fewer deaths than if we picked Mao AI (and possibly Stalin AI). In short: anyone in favor of human universalism is the last person you want to put in charge of AGI alignment.
MostlyStable 2y ago
>If we knew the answer, we would know how to make a perfectly aligned AGI. Actually no, we wouldn't. The problem, at the moment, is even more basic than "what values should we align an AGI with". Currently, the problem is "how do we robustly align an AI with any set of values." We currently do not know how to do this. You could hand OpenAI a universal set of safe values inscribed on stone tablets from god, and they wouldn't know what to do with them. To state it another way, people like to talk about paperclip maximizers. But here's the thing: if we wanted, to we couldn't purposefully make such a maximizer. Right now, AI values are emergent. We can sort-of-kind-of steer them in some general directions, but we don't know how to give them rules that they will robustly follow in all situations, including out-of-context. Look at how easy it is to jailbreak current models into giving you instructions on how to build a bomb. All current AI companies would prefer if their products would never do this, and yet they have been unable to ensure it. They need to solve that problem before they can solve the next one.
- TeMPOraL 2y ago
  > Currently, the problem is "how do we robustly align an AI with any set of values." That's a fair point, and you're absolutely right. > They need to solve that problem before they can solve the next one. Agreed. That, and they need to do it before they build an AGI. Unfortunately, from X-risk perspective, the two are almost the same problem. The space of alignments that lead to prosperous, happy humanity is a very, very tiny area in the larger space of possible value systems an AI could have. Whether we align AI to something outside this area (failing the "what values should we align an AGI to" problem), or the AI drifts outside of it over time (chancing the "what values" problem, but failing the "how do we robustly align" one), the end result is the same.
  - MostlyStable 2y ago
    Yes, I agree that both problems need to be solved. But I think it's still worth focusing on where we actually are. Lots of people believe that they have a set of safe values to align an AI to (Musk thinks "curiosity" will work, another commenter in this thread suggested "don't kill humans"), and so those people will think that the alignment problem is trivial: "Just align the AI to these simple, obviously correct principles". But the truth is that it doesn't even matter whether or not they are correct (my personal opinion is that they are not), because we don't know how to align an AI to whatever their preferred values are. It makes it more obvious to more people how hard the problem is.
anon373839 2y ago
> we would know how to make a perfectly aligned AGI Fortunately then, no one has any idea how to make AGI or whether AGI is even a coherent concept.

dymk 2y ago

How about "do not kill humans"

akira2501 2y ago
Yet we have _standing_ Armies.
semi-extrinsic 2y ago
But to align the LLM in this way, it needs to have agency, desires, wishes, impulses... Not only do LLMs lack such things, but we don't even have any semblance of an idea of how we could give LLMs these things.
- ninetyninenine 2y ago
  The LLM usually molds itself into whatever prompt you give it. That's one way. The other way is to train it on biased information that aligns with a certain agency, desire, wish or impulse.
  - semi-extrinsic 2y ago
    But the LLM doesn't "want" anything. Prompt goes in, tokens come out. When there are no prompts coming in, there are no tokens coming out. Just stop talking to it and all risks are eliminated... You can't "align it to want certain things" when it doesn't have the capacity to "want" in the first place.
    - gowld 2y ago
      Doesn't help me if I stop talkng to the LLM, if the police and the military are talking to the LLM.
      - kergonath 2y ago
        What would “the LLM” tell them? It does not have any memory of what happened after its training. It has no recollection of any interaction with you. The only simulacrum of history it has is a hidden prompt designed to trick you into thinking that it is more than what it actually is. What the police would do is seize your files. These would betray your secrets, LLM or not.

MyFedora 2y ago

> Oh. And what are these universal "human values?" American values. Look up the author.