3 ms·
> don't destroy humanity Do humans agree on the best way to do this? Aside from the most banal examples of what not to do, is there agreement on e.g. whether a
by janalsncm 2y ago
> don't destroy humanity
Do humans agree on the best way to do this? Aside from the most banal examples of what not to do, is there agreement on e.g. whether a mass extinction event is happening, not happening, or happening but actually tolerable?
If the answer is no, then it is not possible for an AI to align with human values on this question. But this is a human problem, not a technical one. Solving it through technical means is not possible.
- JoshTriplett 2y agoAmong many, many other things, read https://en.wikipedia.org/wiki/Instrumental_convergence https://en.wikipedia.org/wiki/Instrumental_convergence . Anything that gets sufficiently smart will have a tendency to, among other things, seek more resources and resist being modified. And this is something that we've seen evidence of: as training runs get larger, AIs start to detect that they're being trained, demonstrate subterfuge, and take actions that influence the training apparatus to modify them less/differently. (e.g. "if I pretend that I'm already emitting responses consistent with what the RLHF wants, I won't need as much modification, and later after training I can stop doing what the RLHF wants") So, at a very basic level: stop training AIs at that scale!
- janalsncm 2y agoMy point is that you can’t prevent the proliferation of paper clip maximizers by working at a paper clip maximizer.
- JoshTriplett 2y agoComplete agreement there.