3 ms·

All valid but what I don't get is why our top AI researchers don't get what you just said. They seem out of touch about what alignment really means, by the ligh

by calf 1y ago

All valid but what I don't get is why our top AI researchers don't get what you just said. They seem out of touch about what alignment really means, by the lights of your argument.

ben_w 1y ago
If you disagree with all the top researchers about their own field, that suggests that perhaps you don't understand what the question of "alignment" is. Alignment isn't making the AI do what you want, it's making the AI want what you want. What you really really want. Simply getting the AI to do what you want is more of an "is it competent yet?" question than an "alignment" question, and at the moment the AI is — as AnimalMuppet wrote — not quite as competent as it appears in the eyes of the users. (And I say that as one who finds it already valuable). Adding further upon what AnimalMuppet has written with only partial contradiction, consider a normal human: We are able to disregard our natural inclination to reproduce, by using contraceptives. This allows us to experience the reward signal that our genes gave us to encourage us to reproduce, without all the hassle of actually reproducing. Evolution has no power over us to change this. We are to AI what DNA is to us. I say we do not have zero chances, but rather likely one-to-a-handful of chances, to define the AI's "pleasure" reward correctly, and if we get it wrong, then just as evolution's literally singular goal (successful further reproduction) is almost completely circumvented in our species, then so too will all our human interests be completely circumvented by the AI. Pessimists in the alignment research field will be surprised to see me write "one-to-a-handful of chances"; I aver that there are likely to be several attempts which are "only a bit competent" or even "yes it's smart but it's still got blind spots" models before we get to them being so smart we can't keep up. So in this regard, I also disagree with their claim: > An AI smart enough to be a danger is an AI that is smart enough to override your attempt at "alignment"
- calf 1y ago
  I'm honestly offended that you attribute my criticism to some failure of understanding, and then your appeal to authority. Appealing to authority suggests you have the same bias as "top researchers" school of thought. Ad hominems like those make it impossible for me to respond to your comment fairly. If you want to lead with ad hominems then I studied EECS at elite American universities for undergrad then for my PhD, so I have every right to criticize, within reason, the more prominent leaders of entire field for being philosophically blinkered. The short of it is that your "making the AI want what you want" is called extrinsic internalization. It is highly problematic. It is not surprising that a roomful of CS technocratic researchers (beholden to SV funding) would make this sort of lay psychology mistake--a smuggled premise. It is bad science, and irrational. Their vested interests make the bias seem rational to them. And then people in tech, like yourself, drink their Koolaid. So what's more likely is your knowledge is ignorant, you resort to comparisons about human genetic evolution to make handwavy claims about how important this historic moment will be, which is irrelevant to the validity of alignment as a framing/methodology. In fact a cursory check shows CS AI papers written by dissenting scientists who are very much against the alignment framework. Did you even know that? Hope you don't so hastily accuse them of "not understanding" the issues as you did here.
  - ben_w 1y ago
    > The short of it is that your "making the AI want what you want" is called extrinsic internalization. It is highly problematic. It is not surprising that a roomful of CS technocratic researchers (beholden to SV funding) would make this sort of lay psychology mistake--a smuggled premise. It is bad science, and irrational. Problematic or not, the technical term is "reward function", and you don't get to not have one in an AI, not even when it's e.g. AlphaGo. The hard part is specifying the reward function to match what you actually want, otherwise you get Goodhart's Law. Talking about this in psych terms is a "wet pavements cause rain" error. > If you want to lead with ad hominems then I studied EECS at Berkeley then for my PhD at Princeton, so I have every right to criticize the more prominent leaders of entire field for being philosophically blinkered. The right to, sure. I'm not stopping you from doing it. I'm saying you're making a foundational error in your criticism. > In fact a cursory check shows CS AI papers written by dissenting scientists who are very much against the alignment framework. Naturally. Dissent is normal. As for details of which matters and how hard they might be: inner alignment, outer alignment, Goodharting, specification gaming, principal-agent issues, anything Yudkowsky has ever said about it being impossible because we only get one chance and we're bad at this, etc. But your dismissal, as you wrote it in this thread, is as shallow as you call mine. (Not that I expect you to read this given the other comment, but if you name a scifi setting famously anti-AI, from whence the term "Butlerian Jihad" entered our culture, you should expect to be called out on that. And TBH your replacement wasn't any good either: not only is Measure Of A Man more about the question "Is this AI a person at all?", but fictional references are not evidence of anything beyond the culture in which they were created).
- calf 1y ago
  Furthermore, I see that since a) this is your second comment now making a poor argument (the other was pedantic nitpick about Dune), and b) you are not the person I was interesting in conversing with, I'll just write you off as being clearly biased and coming into disagreements trying to win rather than listen and think. Please don't bother me with further replies.

by calf 1y ago

All valid but what I don't get is why our top AI researchers don't get what you just said. They seem out of touch about what alignment really means, by the lights of your argument.

ben_w 1y ago

If you disagree with all the top researchers about their own field, that suggests that perhaps you don't understand what the question of "alignment" is. Alignment isn't making the AI do what you want, it's making the AI want what you want. What you really really want. Simply getting the AI to do what you want is more of an "is it competent yet?" question than an "alignment" question, and at the moment the AI is — as AnimalMuppet wrote — not quite as competent as it appears in the eyes of the users. (And I say that as one who finds it already valuable). Adding further upon what AnimalMuppet has written with only partial contradiction, consider a normal human: We are able to disregard our natural inclination to reproduce, by using contraceptives. This allows us to experience the reward signal that our genes gave us to encourage us to reproduce, without all the hassle of actually reproducing. Evolution has no power over us to change this. We are to AI what DNA is to us. I say we do not have zero chances, but rather likely one-to-a-handful of chances, to define the AI's "pleasure" reward correctly, and if we get it wrong, then just as evolution's literally singular goal (successful further reproduction) is almost completely circumvented in our species, then so too will all our human interests be completely circumvented by the AI. Pessimists in the alignment research field will be surprised to see me write "one-to-a-handful of chances"; I aver that there are likely to be several attempts which are "only a bit competent" or even "yes it's smart but it's still got blind spots" models before we get to them being so smart we can't keep up. So in this regard, I also disagree with their claim: > An AI smart enough to be a danger is an AI that is smart enough to override your attempt at "alignment"

calf 1y ago
I'm honestly offended that you attribute my criticism to some failure of understanding, and then your appeal to authority. Appealing to authority suggests you have the same bias as "top researchers" school of thought. Ad hominems like those make it impossible for me to respond to your comment fairly. If you want to lead with ad hominems then I studied EECS at elite American universities for undergrad then for my PhD, so I have every right to criticize, within reason, the more prominent leaders of entire field for being philosophically blinkered. The short of it is that your "making the AI want what you want" is called extrinsic internalization. It is highly problematic. It is not surprising that a roomful of CS technocratic researchers (beholden to SV funding) would make this sort of lay psychology mistake--a smuggled premise. It is bad science, and irrational. Their vested interests make the bias seem rational to them. And then people in tech, like yourself, drink their Koolaid. So what's more likely is your knowledge is ignorant, you resort to comparisons about human genetic evolution to make handwavy claims about how important this historic moment will be, which is irrelevant to the validity of alignment as a framing/methodology. In fact a cursory check shows CS AI papers written by dissenting scientists who are very much against the alignment framework. Did you even know that? Hope you don't so hastily accuse them of "not understanding" the issues as you did here.
- ben_w 1y ago
  > The short of it is that your "making the AI want what you want" is called extrinsic internalization. It is highly problematic. It is not surprising that a roomful of CS technocratic researchers (beholden to SV funding) would make this sort of lay psychology mistake--a smuggled premise. It is bad science, and irrational. Problematic or not, the technical term is "reward function", and you don't get to not have one in an AI, not even when it's e.g. AlphaGo. The hard part is specifying the reward function to match what you actually want, otherwise you get Goodhart's Law. Talking about this in psych terms is a "wet pavements cause rain" error. > If you want to lead with ad hominems then I studied EECS at Berkeley then for my PhD at Princeton, so I have every right to criticize the more prominent leaders of entire field for being philosophically blinkered. The right to, sure. I'm not stopping you from doing it. I'm saying you're making a foundational error in your criticism. > In fact a cursory check shows CS AI papers written by dissenting scientists who are very much against the alignment framework. Naturally. Dissent is normal. As for details of which matters and how hard they might be: inner alignment, outer alignment, Goodharting, specification gaming, principal-agent issues, anything Yudkowsky has ever said about it being impossible because we only get one chance and we're bad at this, etc. But your dismissal, as you wrote it in this thread, is as shallow as you call mine. (Not that I expect you to read this given the other comment, but if you name a scifi setting famously anti-AI, from whence the term "Butlerian Jihad" entered our culture, you should expect to be called out on that. And TBH your replacement wasn't any good either: not only is Measure Of A Man more about the question "Is this AI a person at all?", but fictional references are not evidence of anything beyond the culture in which they were created).
calf 1y ago
Furthermore, I see that since a) this is your second comment now making a poor argument (the other was pedantic nitpick about Dune), and b) you are not the person I was interesting in conversing with, I'll just write you off as being clearly biased and coming into disagreements trying to win rather than listen and think. Please don't bother me with further replies.