3 ms·

As far as I'm concerned this whole discussion is severely hampered by failing to differentiate between intelligence and agency. Almost all of the bugaboo about

by etiam 9y ago

As far as I'm concerned this whole discussion is severely hampered by failing to differentiate between intelligence and agency. Almost all of the bugaboo about runaway superhuman organisms comes down not to machines learning and reasoning about the world but to the effective high-level objective function controlling the actions of an autonomous system. Not making the distinction obscures important things. For one thing we seem to be well on the way to a situation where we arguably have something worthy of the moniker artificial intelligence but the agency is delegated to the human objective function. Considering what complete refuse of human specimens are likely to command some of the first moderately general AI systems that concerns me far more than any summoned demon of Musk's for the foreseeable future. Also, studying these high-level objective functions for autonomous behavior is a very worthy goal, but going first for issues of "value alignment" and "safety", without any specifics of what works for an implementation?? Sure, do it if you enjoy it and have resources to burn. But be prepared to spend heroic efforts coming up with results that are either trivial or non-issues if you were to consider them with a working mechanism in front of you.

Synaesthesia 9y ago
Yes. We can’t even define what so called “AGI” will even be. And we have still not solved many mysteries of human consciousness, or begun to touch on them. I for one have been looking at the problem of ai’s playing Starcraft 2, and the decision making required, such as when you scout your opponent’s army choice, or tech, how to respond. So far they’re very far from solving that, but if progress is made, I’ll be impressed. That’s a very different kind of problem as say image recognition and classification. It requires planning. It’s a very difficult game even for humans to understand. Currently the autonomous systems can’t even play it.
sullyj3 9y ago
> without any specifics of what works for an implementation As I understand it, if you assume that your agent is rational in certain basic ways, for instance that it has ordered, rather than circular preferences, or it can't be Dutch booked/money pumped, it can modelled as having a utility function. Note that this is different from assuming that an explicit utility function will be programmed in, rather the basic level of rationality implies it into existence. Once you know that an agent has a utility function, you can use that to do a fair amount of reasoning about its reasoning. Better (and less briefly) explained here: https://intelligence.org/2016/12/28/ai-alignment-why-its-hard-and-where-to-start/#coherent-decisions-imply-a-utility-function https://intelligence.org/2016/12/28/ai-alignment-why-its-har...

by etiam 9y ago

Synaesthesia 9y ago

Yes. We can’t even define what so called “AGI” will even be. And we have still not solved many mysteries of human consciousness, or begun to touch on them. I for one have been looking at the problem of ai’s playing Starcraft 2, and the decision making required, such as when you scout your opponent’s army choice, or tech, how to respond. So far they’re very far from solving that, but if progress is made, I’ll be impressed. That’s a very different kind of problem as say image recognition and classification. It requires planning. It’s a very difficult game even for humans to understand. Currently the autonomous systems can’t even play it.

sullyj3 9y ago

> without any specifics of what works for an implementation As I understand it, if you assume that your agent is rational in certain basic ways, for instance that it has ordered, rather than circular preferences, or it can't be Dutch booked/money pumped, it can modelled as having a utility function. Note that this is different from assuming that an explicit utility function will be programmed in, rather the basic level of rationality implies it into existence. Once you know that an agent has a utility function, you can use that to do a fair amount of reasoning about its reasoning. Better (and less briefly) explained here: https://intelligence.org/2016/12/28/ai-alignment-why-its-hard-and-where-to-start/#coherent-decisions-imply-a-utility-function https://intelligence.org/2016/12/28/ai-alignment-why-its-har...