4 ms·
One of the classic examples is the Paperclip Maximizer thought experiment. A paperclip maximizer is a superintelligent AI whose sole goal is to maximize the num
by pjscott 14y ago
One of the classic examples is the Paperclip Maximizer thought experiment. A paperclip maximizer is a superintelligent AI whose sole goal is to maximize the number of paperclips in the universe. Such a being is curious to the extent that learning more about the universe increases the expected number of paperclips in its future light-cone.
A paperclip maximizer is indifferent to your existence, but thoroughly believes that it could make better use of your planet than you could, as you are hardly optimized for the production of paperclips.
- platz 14y agoThe Paperclip Maximizer: inconceivable, conceivable but not metaphysically possible, or metaphysically possible? I sense the approach of the p-zombie.
- pjscott 14y agoI'm not sure what you mean by "conceivable" and "metaphysically possible", and I don't see a resemblance to p-zombies. Could you elaborate?
- platz 14y agojust trolling. The distinction was posed here (http://philpapers.org/surveys/results.pl http://philpapers.org/surveys/results.pl). The resemblance: if a superintelligent AI lacks 'conscious experience, qualia, or sentience'.
- comrade_ogilvy 14y agoObviously the Paperclip Maximizer would be supplanted by the Maximizer Maximizer, which only sees value in copies of itself. The Maximizer Maximizer that stumbles onto the mental trait "curiosity" will eventually crush all other variants of Maximizer. Once it develops the advanced technologies of space travel, it discovers that seizing faraway planets is possible but going through all that trouble is not a net positive towards self-actualization as a Maximizer Maximizer Maximizer. Thus TFA is correct.
- pjscott 14y agoObviously the Maximizer Maximizer is just another potential foe for the Paperclip Maximizer to seek out and crush in its infancy. (Sound brutal? It isn't; if you pull it off right, not a single paperclip will be harmed.)
- comrade_ogilvy 14y agoThe potency of the Minimizer Maximizer is more likely to be discovered by a curious Maximizer variant. But will it have the wisdom to not to employ it? But you are falling into a anthroporeligio trap of assuming the Paperclip Maximizer considers potential paperclips in their uncoiled wire form to have comparable moral status as actual paperclips, once the wire reaches the factory floor.
- coldtea 14y agoSounds like a crappy example though, as far as examples/thought experiments go. How is this specific choice of example any good, or makes any sense?
- pjscott 14y agoIt's alien enough that people won't try to anthropomorphize the Paperclip Maximizer. That's not an easy thing for an example to accomplish. And it was originally conceived for talking about artificial intelligence, which makes it a bit less outlandish.