4 ms·
"A friend is training a simulated robot arm to reach towards a point above a table. It turns out the point was defined with respect to the table, and the table
by tehsauce 9y ago
"A friend is training a simulated robot arm to reach towards a point above a table. It turns out the point was defined with respect to the table, and the table wasn’t anchored to anything. The policy learned to slam the table really hard, making the table fall over, which moved the target point too."
Seems as though the problem of learning unintended techniques sometimes may be better described as the model being too creative! Hitting the table is a really clever solution for the problem it was given. These examples show that the real challenge for researchers is constraining the models enormous capacity for creativity without stifling its ability to learn.
- mtgx 9y agoCan't wait to see how "creative" our future autonomous drone strikes will be.
- IshKebab 9y agoLike if we program our AI overlords to minimise unavoidable deaths, and it learns the best way to do in the long term is to sterilise everyone and wipe out the human race!
- PeterisP 9y agoWell, sure, 7 billion quick deaths means less death and suffering than even a measly one gruesome random accident per year for 8 billion years in the future.
- deleted 9y ago[deleted]
- deepGem 9y agoReminds me of this article, about how a single reward could potentially destroy humanity. It's just a thought experiment though. https://www.salon.com/2014/08/17/our_weird_robot_apocalypse_why_the_rise_of_the_machines_could_be_very_strange/ https://www.salon.com/2014/08/17/our_weird_robot_apocalypse_...
- atupis 9y agoIt's happening already https://arstechnica.com/information-technology/2016/02/the-nsas-skynet-program-may-be-killing-thousands-of-innocent-people/ https://arstechnica.com/information-technology/2016/02/the-n...
- rmrfrmrf 9y agoI felt the same, and also thought that the article didn't really do enough to explain why "write better reward functions" isn't enough, especially because in gaming, if anyone set the same goals these researchers did, humans would eventually learn the same patterns.
- gchadwick 9y ago> constraining the models enormous capacity for creativity without stifling its ability to learn If I recall my AI history correctly all the 60s/70s research focused around logic based systems and inference producing things like Prolog. It was thought that one just simply needed to come up with an appropriate rule set to generate powerful AI. The issue of course is writing enough rules to give powerful AI outside some very constrained problem domains (see https://en.wikipedia.org/wiki/SHRDLU https://en.wikipedia.org/wiki/SHRDLU) just isn't feasible. The problem of current machine learning models being too clever for their own good and needing appropriate constraints feels similar. If only you had the correct constraints you could do all kinds of things. History repeating itself?