4 ms·

There was a recent project that combined transformer breakthroughs (such as those used in ChatGPT) with human motion. [1] https://guytevet.github.io/mdm-page/

• modeless 4y ago
  I bet everyone in robotics is thinking about combining large language models with robot control right now. I expect some pretty awesome demos in the next few years. Robots navigating real spaces and performing actual mundane tasks from natural language commands feels a lot closer than it did a year ago.
  • eternalban 4y ago
    I wonder what NSA and the like are doing with LLMs in context of cryptography. Is there a Shannon information theoretic limit that would make that thought go away? (Doesn't have to be end to end. Could just hugely reduce the search spaces..) -- ps 2008: https://www.cs.unc.edu/~fabian/papers/oakland08.pdf https://www.cs.unc.edu/~fabian/papers/oakland08.pdf
  • mike_hearn 4y ago
    Such demos already exist. I watched one last year. The trick is a (by now) common one: you train the LLM on a simple programming language that's interpreted by the robot's software. So natural language goes in and the LLM then thinks out loud to figure out the needed steps, then emits Logo-like instructions to control where the robot goes. More conventional robotics AI then takes over to handle movement of the machine, the arms, etc.