NIPS 2004 Demonstrations: https://web.archive.org/web/20041204162656/http://www.nips.cc/Conferences/2004/Demonstrations/Demonstrations.php https://web.archive.

NIPS 2004 Demonstrations: https://web.archive.org/web/20041204162656/http://www.nips.cc/Conferences/2004/Demonstrations/Demonstrations.php https://web.archive.org/web/20041204162656/http://www.nips.c... Virtual Bicycle — Matthew Cook, California Institute of Technology: http://www.paradise.caltech.edu/cook/papers/index.html http://www.paradise.caltech.edu/cook/papers/index.html In this work I created an environment in which I assumed learning of higher order concepts would be necessary. I created a general purpose physics-based hinged rigid body simulator and used it to simulate a bicycle for a learning agent to learn to ride. But as others have found in other contexts, most environments do not require higher order concepts, and in this case it turned out that a simple two neuron circuit was already sufficient for controlling the bicycle, indeed, better than a human using the keyboard. The physics simulator was a significant project in itself: I studied rigid body mechanics (which is more complex than most of us realize) and designed the simulator explicitly so as to simultaneously exactly preserve both angular momentum and kinetic energy, as previous simulations of mine (in quantum mechanics!) had shown me that preserving conserved quantities can be crucial for getting accurate results. The simulator works nicely, and I later read in Sam Buss's 2001 paper, Accurate and Efficient Simulation of Rigid Body Rotations, that my precautions were well warranted. Later I was able to reduce the controller to just one neuron, and then to an even simpler plain linear feedback system, confirming the finding that many real problems are best solved by a hack. The simulator has been useful in other projects since then. Virtual Bicycle Download Page: http://www.dna.caltech.edu/~cook/ http://www.dna.caltech.edu/~cook/