4 ms·

Dario Amodei's recent post had a good analysis about which fields are and are not limited by intelligence. https://darioamodei.com/machines-of-loving-grace htt

Dario Amodei's recent post had a good analysis about which fields are and are not limited by intelligence. https://darioamodei.com/machines-of-loving-grace https://darioamodei.com/machines-of-loving-grace

epcoa 2y ago
“An aligned AI would not want to do these things (and if we have an unaligned AI, we’re back to talking about risks).” An aligned AI is not AGI, or whatever they want to call it.
- ben_w 2y ago
  > An aligned AI is not AGI, or whatever they want to call it. There's a few ways I can interpret that. If you mean "alignment and competence are separate axies" then yes. That's well understood by the people running most of these labs. (Or at least, they know how to parrot the clichés stochastically :P) If you mean "alignment precludes intelligence", then no. Consider a divisive presidential election between Alice and Bob, no this isn't a reference to the USA, each polling 50%: regardless of personal feelings or the candidates themselves, clearly the campaign teams are both competent and intelligent… yet each candidate is only aligned with 50% of the population.
  - epcoa 2y ago
    Campaign team members and even candidates switch teams often enough. Weak analogy. What is the alignment of human GI, completely generalized?
    - ben_w 2y ago
      > What is the alignment of human GI, completely generalized? Of any specific human to any other specific human? https://benwheatley.github.io/blog/2019/05/25-15.09.10.html https://benwheatley.github.io/blog/2019/05/25-15.09.10.html Of any specific human to a nation? That's the example you replied to. Of all the people of a nation to each other? Best we've done there is what we see in countries in normal times, with all the strife and struggles within. We have yet to fully extend from nation to the world; the closest for that is the UN, which is even less in agreement with itself than are nations.
      - epcoa 2y ago
        I think that's my point. The notion of maintaining an alignment, pro-human or whatever for a replicable general AI, doesn't seem to make sense. The traits of planning, learning and goal setting don't seem concordant with maintaining an alignment. I think this discussion has veered to much to anthrocentrism to be interesting, but alignment however loosely defined here isn't some constant for an individual through their life either. It can be imprecisely manipulated especially in a population by outside forces, but it can't be directly controlled.
        ben_w 2y ago
        I think I understand, but let's check by rephrasing: "Alignment" is only possible up to a vague approximation, and an entirely perfectly aligned with another entity would essentially be a shadow rather than a useful assistant because by being perfectly aligned the agent would act tired exactly when the person was tired, go shopping exactly when the human would, forget their keys exactly when the human would, respond exactly like the human to all ads and slogans, etc.? I agree, though: (1) this has already been observed, last year's OpenAI dev day had (IIRC) a story about a writer who fine tuned a model on their slack (?) messages, they asked it to write something for them, the response was ~"sure, I'll get on it tomorrow". (2) for many of those concerned with "solving alignment", it's sufficient for the agent to never try to kill everyone just to make more paperclips etc.

epcoa 2y ago

“An aligned AI would not want to do these things (and if we have an unaligned AI, we’re back to talking about risks).” An aligned AI is not AGI, or whatever they want to call it.

ben_w 2y ago
> An aligned AI is not AGI, or whatever they want to call it. There's a few ways I can interpret that. If you mean "alignment and competence are separate axies" then yes. That's well understood by the people running most of these labs. (Or at least, they know how to parrot the clichés stochastically :P) If you mean "alignment precludes intelligence", then no. Consider a divisive presidential election between Alice and Bob, no this isn't a reference to the USA, each polling 50%: regardless of personal feelings or the candidates themselves, clearly the campaign teams are both competent and intelligent… yet each candidate is only aligned with 50% of the population.
- epcoa 2y ago
  Campaign team members and even candidates switch teams often enough. Weak analogy. What is the alignment of human GI, completely generalized?
  - ben_w 2y ago
    > What is the alignment of human GI, completely generalized? Of any specific human to any other specific human? https://benwheatley.github.io/blog/2019/05/25-15.09.10.html https://benwheatley.github.io/blog/2019/05/25-15.09.10.html Of any specific human to a nation? That's the example you replied to. Of all the people of a nation to each other? Best we've done there is what we see in countries in normal times, with all the strife and struggles within. We have yet to fully extend from nation to the world; the closest for that is the UN, which is even less in agreement with itself than are nations.
    - epcoa 2y ago
      I think that's my point. The notion of maintaining an alignment, pro-human or whatever for a replicable general AI, doesn't seem to make sense. The traits of planning, learning and goal setting don't seem concordant with maintaining an alignment. I think this discussion has veered to much to anthrocentrism to be interesting, but alignment however loosely defined here isn't some constant for an individual through their life either. It can be imprecisely manipulated especially in a population by outside forces, but it can't be directly controlled.
      - ben_w 2y ago
        I think I understand, but let's check by rephrasing: "Alignment" is only possible up to a vague approximation, and an entirely perfectly aligned with another entity would essentially be a shadow rather than a useful assistant because by being perfectly aligned the agent would act tired exactly when the person was tired, go shopping exactly when the human would, forget their keys exactly when the human would, respond exactly like the human to all ads and slogans, etc.? I agree, though: (1) this has already been observed, last year's OpenAI dev day had (IIRC) a story about a writer who fine tuned a model on their slack (?) messages, they asked it to write something for them, the response was ~"sure, I'll get on it tomorrow". (2) for many of those concerned with "solving alignment", it's sufficient for the agent to never try to kill everyone just to make more paperclips etc.