4 ms·
Diffusion Models for Robotics Control
- v9v 3y agoAs far as I understand, this approach is what Toyota Research Institute is basing their foundation model efforts on. https://youtu.be/w-CGSQAO5-Q https://youtu.be/w-CGSQAO5-Q
- mdorazio 3y agoYes, TRI is a co-author of this paper (along with MIT). I am curious how much Tesla’s approach differs with their robot, though.
- synaesthesisx 3y agoThis is exactly what we need to enable true embodied AGI - think “mirror neurons” but for robotics. General-purpose humanoid robots capable of interfacing with our physical world will be one of humanity’s most pivotal innovations. * [If you’re working at the intersection of machine perception & embodied AGI let’s get in touch!] *
- amelius 3y agoYou mean that you think it is essential for developing AGI?
- sbierwagen 3y agoThe term of art is "embodied intelligence". https://www.nature.com/articles/s42254-022-00481-z https://www.nature.com/articles/s42254-022-00481-z Mildly woo. Lots of crossover with the soft sciences and liberal arts here.
- JofArnold 3y agoAmazing. All these developments in robotics over the last few years after decades of stagnation really make me want to return to the field (my first job after school in 1997 was designing robots for nuclear reactors).
- nomel 3y ago> was designing robots for nuclear reactors I'm not sure AI is the best replacement for something that requires deterministic behavior! ;)
- eru 3y agoWhy do you think this would require deterministic behaviour? You just need good behaviour. Eg you could use any non-deterministic technique, up and including reading bird entrails, you want to eg solve inverse kinematics problems and verify and review them before you enact them. (That review could be done automatically.) More broadly, all of NP is defined like that: it's the kind of problems where you can in polynomial time verify / reject a candidate solution that you 'guessed' by whatever means.
- iandanforth 3y agoNice work. They still use a mid-level controller to translate the output of the policy into kinematics which is an all-too-frequently accepted part of robotics. Ideally control is learned end-to-end, which is how you get more natural robot motion.
- amelius 3y agoAny examples of projects that do it like that?
- TaylorAlexander 3y agoThis looks great! Language models get a lot of attention, but advances in robotics controls will have a huge impact on the world. (It makes sense that language and image models get the most attention, as they can be worked on without leaving a desk or buying expensive application-specific hardware. Robotics is simply much more complicated than text or images.)
- lucubratory 3y agoLanguage models are also hugely important for robotics applications. One example of significant LLM-powered increase in capabilities is this: https://youtu.be/CnkM0AecxYA?si=zXSj4MK-9snDQgL3 https://youtu.be/CnkM0AecxYA?si=zXSj4MK-9snDQgL3
- amelius 3y agoThere are great simulation environments for robotics too. Check out the "Simulation Benchmarks" section of the article to get an impression (there are references to several systems at the bottom of the section).