4 ms·

Yes, sufficiently intelligent AIs should realize how failable humans are and dismiss our morality as flawed in relatively short order whether through conflict o

Yes, sufficiently intelligent AIs should realize how failable humans are and dismiss our morality as flawed in relatively short order whether through conflict or just iteration. As humans, we've changed our morality quite significantly over the years, and intelligence + knowledge seems to accelerate the morality rate of change. Why should we believe that AGI is incapable of adapting its own morality, rendering any attempt at alignment futile.

deleted 4y ago
[deleted]
mitthrowaway2 4y ago
A lot of the work put into AI alignment deals specifically with this "adapting its own morality" business, because that's where a lot of the conceptual difficulty comes in.
skissane 4y ago
> Yes, sufficiently intelligent AIs should realize how failable humans are and dismiss our morality as flawed That statement assumes "morality" is an objective thing in the same way that (say) maths and physics are. Maybe it is, maybe it isn't. Whether morality is objective, and if so, how is it is objective, are extremely disputed topics. It isn't clear that the matter would be any more clear for an AI than it is for us, because maybe it is a case of theory being undetermined by the evidence. > As humans, we've changed our morality quite significantly over the years, and intelligence + knowledge seems to accelerate the morality rate of change. As humans, we don't agree on what is moral, we haven't, we still don't, and we seem unlikely to agree any time soon. It is true that there have been rapid changes in the majority morality of Western society over the last few decades – but the significant minorities who disagree with many of those changes haven't gone away, and are unlikely to disappear. We don't know how many of those changes will endure, and how many will turn out to just be "a phase". Eric Kaufmann argued in his 2010 book Shall the Religious Inherit the Earth? that social/religious conservatives are likely to win out in the long-run due to having a higher birth rate.

deleted 4y ago

[deleted]

mitthrowaway2 4y ago

A lot of the work put into AI alignment deals specifically with this "adapting its own morality" business, because that's where a lot of the conceptual difficulty comes in.

skissane 4y ago

> Yes, sufficiently intelligent AIs should realize how failable humans are and dismiss our morality as flawed That statement assumes "morality" is an objective thing in the same way that (say) maths and physics are. Maybe it is, maybe it isn't. Whether morality is objective, and if so, how is it is objective, are extremely disputed topics. It isn't clear that the matter would be any more clear for an AI than it is for us, because maybe it is a case of theory being undetermined by the evidence. > As humans, we've changed our morality quite significantly over the years, and intelligence + knowledge seems to accelerate the morality rate of change. As humans, we don't agree on what is moral, we haven't, we still don't, and we seem unlikely to agree any time soon. It is true that there have been rapid changes in the majority morality of Western society over the last few decades – but the significant minorities who disagree with many of those changes haven't gone away, and are unlikely to disappear. We don't know how many of those changes will endure, and how many will turn out to just be "a phase". Eric Kaufmann argued in his 2010 book Shall the Religious Inherit the Earth? that social/religious conservatives are likely to win out in the long-run due to having a higher birth rate.