3 ms·
Lying as an emergent phenomenon: > Our algorithm’s performance is only as good as the human evaluator’s intuition about what behaviors look correct, so if the
by moyix 9y ago
Lying as an emergent phenomenon:
> Our algorithm’s performance is only as good as the human evaluator’s intuition about what behaviors look correct, so if the human doesn’t have a good grasp of the task they may not offer as much helpful feedback. Relatedly, in some domains our system can result in agents adopting policies that trick the evaluators. For example, a robot which was supposed to grasp items instead positioned its manipulator in between the camera and the object so that it only appeared to be grasping it, as shown below.
- metalliqaz 9y agoI think there is a case to be made that it's a feature, not a bug. Humans have so much baggage we carry into everything we do. Is there any difference between a person who has memorized multiplication tables vs a human well trained in using a calculator? In most situations there is not, though we seem to personally value the memorization method despite the increased cost. The principle scales. What you've pointed out is a wonderful demonstration of our need to improve measurement and testing, which humans are typically horrible at. For a good example, just look at government under any party. Policies are continually argued over when that time would be better spent developing a good test and solid measurements for policies and laws.
- deleted 9y ago[deleted]
- euyyn 9y ago> developing a good test and solid measurements for policies and laws Most regulations are closed-loop in that sense. They create the way to measure their effectiveness and be reevaluated accordingly.