3 ms·
> That may well be the original intention. But that's not how it's used in practice, is it?... > But please stop alluding to hidden treasure troves of argument
by jessriedel 12y ago
> That may well be the original intention. But that's not how it's used in practice, is it?...
> But please stop alluding to hidden treasure troves of arguments that cannot be reproduced and made apparent. I've asked enough times in this thread already if there is any highlight in there.
Perhaps I'm misinterpreting, but it sounds like you got the wrong impression of the point of the story by reading about it from some folks on the internet, and that you at least provisionally accept now that it could have a useful point, and that your appreciation of this came from someone (me) who read and cites the original material. That seems like good evidence that the book contains reasonable arguments that haven't yet made it to you unadulterated.
> (Or, to strip away the last shreds of this idea: you can tell it to make no more than 100 million paper clips, and to rest on Sundays.)
Actually, no. If the machine wanted to maximize the likelihood that is successfully built 100 million paperclips, and the machine is smart enough that it can take over the world with very high likelihood (or if it worried about the small chance of humans destroying the world with nuclear weapons), then it will first take over the world and then build the paperclips.
> Yudkowsky [1], who runs that site (which I don't frequent btw), seems to be pretty close to Bostrom.
Yudkowsky runs that site as much as Paul Graham runs Hacker News. (It's a public forum, where anyone can post and content is voted to the top, etc.) I presume you would recommend that someone actually read Paul Graham's writing before dismissing his philosophy on start-ups based on the conversations of his followers on HN. And even if HN was on the decline and there was only trolls and bad thinkers left, you say the same thing. All I'm recommending is the same for Bostrom.