4 ms·
> "don't turn humans into paperclips" is part of the context of "make more paperclips" The idea in this thought experiment is not that the AI parses this sente
by sawwit 11y ago
> "don't turn humans into paperclips" is part of the context of "make more paperclips"
The idea in this thought experiment is not that the AI parses this sentence in the context of human culture (then it would likely comprehend that the actual intention is to maximize the economic success and eventually the human preferences of its creator). What is meant is that the objective is crudely implanted into the AI as an ultimate goal, in a similar way to how sustenance, pain avoidance and affiliation are very basic goals in our cognitive system. It is not entirely obvious that this is a stupid thing to do; hence the thought experiment. Will the AI suppress its urge once it comprehends human culture enough to understand the intentions of its creator? Will it rather successfully learn all the tricks to convert matter into paperclips before it considers studying human values? If the AI does not have a curiosity objective, it will likely not care about us very much, apart from the information that helps it optimizing its objective function.
- argonaut 11y agoI think it is way more likely we have AI that understands human context and values before we have AI that is super intelligent.
- sawwit 11y agoWhat makes you so certain?
- argonaut 11y agoWhat makes you so certain? Also, https://en.wikipedia.org/wiki/Intermediate_value_theorem https://en.wikipedia.org/wiki/Intermediate_value_theorem
- sawwit 11y agoI never said am certain, I'd assign both cases about the same probability. Since I've already stated why I think the malevolent simple AI scenario such as a paperclip maximizer is plausible, you haven't stated why you find it implausible. That is why I asked. Also, what does the Intermediate value theorem have to do with it?
- argonaut 11y agoYou asked why I was certain. So I asked why you were certain. The onus is on the people putting out apocalyptic theories to justify the apocalyptic scenario (extraordinary claims require extraordinary evidence - a central tenet of the scientific method). We are going to have a normal-intelligent AI before a superintelligent AI.