4 ms·
"To develop robots, you have two options: You can either simulate an environment and robot with software and hope the results are accurate enough that you can l
by mixmax 11y ago
"To develop robots, you have two options: You can either simulate an environment and robot with software and hope the results are accurate enough that you can load it into a machine and watch it walk."
This is much harder than you'd think.
Here's a fun story from 15 years ago when a friend of mine tried to do some simple AI:
His goal was to have a humanoid shape created is software learn how to walk using simple AI. The idea was that it would obey some basic laws (gravity, the limits of its joints, etc.), do something random, check whether or not it was closer to the goal of walking, tweak its parameters, and iterate. The chosen goal was not to fall over.
He set up the program, let it run over the weekend and let it do millions of tries. Hopefully when he came back to the office it would have learned how to walk, or at least stand up without falling.
His disappointment was huge when he came into the office: The simulated robot was sitting down with its knees bent, thus having achieved the goal of not falling over.
- swsieber 11y agoI don't know if that's the best illustration - it seems like he forgot to include forward motion in his fitness function.
- oh_sigh 11y agoYes...it's not that hard. There are literally dozens of genetic algorithm simulators available to run in your browser which do exactly that.
- zxcvvcxz 11y agoFunny anecdote, but "artificial intelligence" (which people misuse as a fancy term when what they really mean is task optimization) requires setting the right goals. You know, like moving from point A to point B.
- hughperkins 11y agoYes, seems like more anecdote than reality. Reality is, you wouldnt just 'code an AI', and leave it running for the weekend, and then act all surprised when it has bugs in. You'd work your way eg through Sutton's tasks, like drive a car up a hill https://en.wikipedia.org/wiki/Mountain_Car https://en.wikipedia.org/wiki/Mountain_Car , try not to fall off a cliff, http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node65.html http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node65.html balance an arm http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node110.html http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node110.htm... , and so on. And since these mostly learn really quickly, and ones initial implementation will be buggy, you wouldnt go away for a weekend and leave it running, youd just sit there running it for a minute or two, fixing bugs, running again, and so on.
- afarrell 11y agoHmm.... I now want to see if I can train a neural net to play QWOP.
- trentlott 11y agoThat would be incredible, and incredibly popular
- JimmyAustin 11y agoI've heard a similar story about small self driving cars. The cars would drive around using a random algorithm, then copy and tweak the algorithm of the longest running car when they crashed. The researcher left the room to let the cars work, only to come back and find that each of the cars had deduced that the perfect solution was to remain perfectly still. After all, if they didn't move, they couldn't crash!
- ghostDancer 11y agoThe only winning move is not to play - WOPR A.k.a. Joshua
- komali2 11y agoThat's the story story of a game-learning program that found the best way to "not lose" at tetris was to pause right before a brick extended above the top of the level, and leave it paused indefinitely.
- agumonkey 11y agoReaching zen in a weekend. Over archiever.
- i000 11y agoIt is actually possible to do this (the gaits are discovered not programmed): https://www.youtube.com/watch?v=yci5FuI1ovk https://www.youtube.com/watch?v=yci5FuI1ovk
- pcl 11y agoWow, that's a great video. The generation-comparison section (0:55) and the outtakes (4:50) were my favorites.
- argonaut 11y agoFortunately, this is not an issue in reinforcement learning any more. It's quite simple to train humanoids in simulators to walk. But that knowledge is completely untransferable to the real world.