3 ms·
These are not asking the right questions, although they kind of hint at it, and they are not fundamentally questions about AI. Example: "Can we transform an R
by fiatmoney 10y ago
These are not asking the right questions, although they kind of hint at it, and they are not fundamentally questions about AI. Example: "Can we transform an RL agent's reward function to avoid undesired effects on the environment?" Trivially, the answer is yes; put a weight on whatever effect you're trying to mitigate, to the extent you care about trading off potential benefits. They qualify this by saying essentially "... but without specifying every little thing". So - what you're trying to do is build a rigorous (ie, specified by code or data) model of what a human would think is "reasonable" behavior, while still preserving freedom for gordian knot style solutions that trade off things you don't care about in unexpected ways.
The hard part is actually figuring out what you care about, particularly in the context of a truly universal optimizer that can decide to trade off anything in the pursuit of its objectives.
This has been a core problem of philosophy for 3000 years - that is, putting some amount of rigorous codification behind human preferences. You could think of it as a branch of deontology, or maybe aesthetics. It is extremely unlikely that a group sponsored by Sam Altman, whose brilliant idea was "let's put the government in charge of it" [1], will make a breakthrough there.
I don't actually doubt that AIs would lead to philosophical implications, and philosophers like Nick Land have actually explored some of that area. But I severely doubt the ability of AI researchers to do serious philosophy and simultaneously build an AI that reifies those concepts.
[1] http://blog.samaltman.com/machine-intelligence-part-2 http://blog.samaltman.com/machine-intelligence-part-2
- argonaut 10y agoYou're dismissing the paper for not asking the right questions, but you don't propose any questions that you think are better. > The hard part is actually figuring out what you care about, particularly in the context of a truly universal optimizer that can decide to trade off anything in the pursuit of its objectives. This seems basically equivalent to what they are saying. A reward function that rewards "what we actually care about." This might seem vague, but that's fine because these are only proposed problems.
- akvadrako 10y agoI'm not sure what point you are trying to make. It's possible to dismiss an idea without providing an alternative. Yes, finding a reward function is equivalent to figuring out what we care about. Both are about as hard as teaching a bacteria to play piano. The goal is avoiding unsafe AI. The reason such pointless efforts are wasted on this approach is we don't have a good alternative. The only thing I can think of is delaying it's creation indefinitely, but that's also a difficult challenge. For example, in the Dune books, the government outlaws all computers. That might work for a while.
- stevetrewick 10y agoFrom my cold, dead hands.
- argonaut 10y agoLet me elaborate. It is easy easy easy to nitpick and find holes in someone's proposals, someone's problem statements, and someone's goals in life. Statements are adding noise and less than nothing of value if they just consist of telling people they are working on the wrong thing... and not proceeding to tell them what they should instead be working on, and giving clear positive reasons why (instead of negative reasons someone should not be working on something). Incidentally this is a broader problem with HN discourse.
- GuiA 10y ago> You're dismissing the paper for not asking the right questions, but you don't propose any questions that you think are better. One might wish to point out that the emperor has no clothes and yet have no desire to plan his majesty's outfits for the next 6 months.
- argonaut 10y agoA witty saying proves nothing. See my reply to the other comment.
- fiatmoney 10y ago
- marvin 10y agoI think you've hit the nail on the head here, this feels to me like one of a few responses in this thread that gets to the core of the issue if we're thinking about long-term AI risk. In fact, I'm surprised that there doesn't seem to be any reference in the article to previous work on these philosophical implications, e.g. the stuff that has been written by Nick Bostrom or MIRI. Perhaps there are some in the paper? I think that for the forseeable future, we will inevitably end up with two of the problems that various philosophers have outlined over the last few years: (1) How do we ensure that an AI agent does exactly what we want it to do and (2) What do we ultimately want if we can desire anything? I think that any developer trying to approach this will be doomed to hack around these two issues. We can probably come a long way in AI capabilities without having the optimal solution to this, but the core problem will remain for a long time and haunt those who are cautious.
- argonaut 10y ago1) Is exactly what this paper is addressing. The fact that their is philosophical ambiguity is precisely why these are problems, and not solved already.