OpenAI did a thing[0] where they used a human to help train a machine learning model by picking the best of 2 models. > Our algorithm’s performance is only as

OpenAI did a thing[0] where they used a human to help train a machine learning model by picking the best of 2 models. > Our algorithm’s performance is only as good as the human evaluator’s intuition about what behaviors look correct, so if the human doesn’t have a good grasp of the task they may not offer as much helpful feedback. Relatedly, in some domains our system can result in agents adopting policies that trick the evaluators. For example, a robot which was supposed to grasp items instead positioned its manipulator in between the camera and the object so that it only appeared to be grasping it, as shown below. [0] https://blog.openai.com/deep-reinforcement-learning-from-human-preferences/ https://blog.openai.com/deep-reinforcement-learning-from-hum... https://news.ycombinator.com/item?id=14545298 https://news.ycombinator.com/item?id=14545298 78 upvotes, 7 comments