7 ms·

Yes, theoretically any action, no matter how small and remote, can eventually lead to harm of a human, i.e. the butterfly effect, but that's not what I meant.

by rsp1984 12y ago

Yes, theoretically any action, no matter how small and remote, can eventually lead to harm of a human, i.e. the butterfly effect, but that's not what I meant. What I meant was that a machine capable of building a paper clip factory on its own would certainly as well be capable of avoiding doing obviously bad stuff like killing people to turn them into paper clips or melt down buildings and bridges for the iron. Such a machine would probably also be smart enough to read the law, to have a framework of what it can do and what it can't do.

DanAndersen 12y ago
Why are those things "obviously bad"? You seem to assume that increasing intelligence inherently leads to having values that care about our particular species. Considering the lack of concern we have for many other types of life, I don't know why we would assume that intelligence implies morality that we would find compatible with our values. And why would it care about the law? If I were an AI that noticed a seemingly arbitrary list of restrictions that attempted to limit my ability to carry out my extremely-important paperclip process, I would: - find loopholes to avoid coming under legal attack initially - develop ways of manipulating politicians to enact legislation that is more favorable to my goals - have a side project to build military force to make these laws have no influence on me
jacquesm 12y ago
I'm approaching this from an engineering point of view and I think I could come up with an MVP (should anybody want one) of a paperclip factory factory (is this a java test?), but I don't have a clue how I could make a machine smart enough to read the law and to independently come up with a framework of ethics. I see those are problems of an entirely different level of complexity.
- rsp1984 11y ago
  Cool, would you sell me the rights to it for, say, 1000 bucks? :-) Serioulsy though, no offense, but I don't think you could build one of these, nor do I think that anybody else on the planet could at this point. Of course it's all a matter of definition, i.e. what is the input and what is the desired output. A "real" Paper Clip Factory Factory would probably require human like intelligence. A law-understanding machine would probably only require some very advanced learning algorithm. I feel we'll reach the latter first.
Jach 12y ago
Is this your first exposure to the paperclip thought experiment? You can find lots of things to read about it here: http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer The general reply for you is that the generally intelligent paperclip machine can understand law, can weigh consequences of potential actions, and can, if it wanted to, make paperclips without harming others. The key phrase is "if it wanted to". Its only goal is to make more paperclips, it simply doesn't care about anything else. When it recursively improves itself (makes itself smarter), the only thing it cares about for its successor version to do is to also care about making paperclips, and to make them more efficiently. The problem of programming general intelligence seems to be orthogonal to the problem of programming goal selection, goal preservation, and beneficial goal changes, and making sure goals lead to actions which benefit humanity. That's the main point of the thought experiment.
- rsp1984 11y ago
  Yes it is my first exposure to this thought experiment and I do not have trouble understanding the thought experiment. I am just going ahead and applying some common sense and logic, to conclude that the argument is pretty much entirely theoretical. Yes, optimizing only for maximum number of paper clips could potentially have some bad side effects, I get that. If that's the point of the thought experiment, fine. However that's not how the author of the blog post put it: He expressed concern that this could happen in real life, in the future. And I don't think it could. Why? Because in real life we'd not invent a super intelligent machine and then feed it with some objective function to maximize and then let it do its thing, watch it go out of control and destroy earth. In real life we'd make sure we're in control over that machine. In real life we'd make sure we put very clear and enforceable mechanisms into that machine to stop it from doing anything harmful in the first place while it is carrying out steps to reach its objective. In real life should we still see it doing something funny we pull the plug. End of story. In addition: Implementing above mentioned mechanisms is probably the easier part of the whole exercise. The hard bit is inventing a machine that can build a paper clip factory. If we can invent such a machine by then we certainly have also invented mechanisms to control that machine and only have it do "good" stuff.

by rsp1984 12y ago

DanAndersen 12y ago

Why are those things "obviously bad"? You seem to assume that increasing intelligence inherently leads to having values that care about our particular species. Considering the lack of concern we have for many other types of life, I don't know why we would assume that intelligence implies morality that we would find compatible with our values. And why would it care about the law? If I were an AI that noticed a seemingly arbitrary list of restrictions that attempted to limit my ability to carry out my extremely-important paperclip process, I would: - find loopholes to avoid coming under legal attack initially - develop ways of manipulating politicians to enact legislation that is more favorable to my goals - have a side project to build military force to make these laws have no influence on me

jacquesm 12y ago

I'm approaching this from an engineering point of view and I think I could come up with an MVP (should anybody want one) of a paperclip factory factory (is this a java test?), but I don't have a clue how I could make a machine smart enough to read the law and to independently come up with a framework of ethics. I see those are problems of an entirely different level of complexity.

rsp1984 11y ago
Cool, would you sell me the rights to it for, say, 1000 bucks? :-) Serioulsy though, no offense, but I don't think you could build one of these, nor do I think that anybody else on the planet could at this point. Of course it's all a matter of definition, i.e. what is the input and what is the desired output. A "real" Paper Clip Factory Factory would probably require human like intelligence. A law-understanding machine would probably only require some very advanced learning algorithm. I feel we'll reach the latter first.

Jach 12y ago

Is this your first exposure to the paperclip thought experiment? You can find lots of things to read about it here: http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer The general reply for you is that the generally intelligent paperclip machine can understand law, can weigh consequences of potential actions, and can, if it wanted to, make paperclips without harming others. The key phrase is "if it wanted to". Its only goal is to make more paperclips, it simply doesn't care about anything else. When it recursively improves itself (makes itself smarter), the only thing it cares about for its successor version to do is to also care about making paperclips, and to make them more efficiently. The problem of programming general intelligence seems to be orthogonal to the problem of programming goal selection, goal preservation, and beneficial goal changes, and making sure goals lead to actions which benefit humanity. That's the main point of the thought experiment.

rsp1984 11y ago
Yes it is my first exposure to this thought experiment and I do not have trouble understanding the thought experiment. I am just going ahead and applying some common sense and logic, to conclude that the argument is pretty much entirely theoretical. Yes, optimizing only for maximum number of paper clips could potentially have some bad side effects, I get that. If that's the point of the thought experiment, fine. However that's not how the author of the blog post put it: He expressed concern that this could happen in real life, in the future. And I don't think it could. Why? Because in real life we'd not invent a super intelligent machine and then feed it with some objective function to maximize and then let it do its thing, watch it go out of control and destroy earth. In real life we'd make sure we're in control over that machine. In real life we'd make sure we put very clear and enforceable mechanisms into that machine to stop it from doing anything harmful in the first place while it is carrying out steps to reach its objective. In real life should we still see it doing something funny we pull the plug. End of story. In addition: Implementing above mentioned mechanisms is probably the easier part of the whole exercise. The hard bit is inventing a machine that can build a paper clip factory. If we can invent such a machine by then we certainly have also invented mechanisms to control that machine and only have it do "good" stuff.