There are some robotics researchers trying to frame robotics tasks as sequences of action and sensor tokens, giving the model all the data collected by the robo

There are some robotics researchers trying to frame robotics tasks as sequences of action and sensor tokens, giving the model all the data collected by the robot and the actions taken by the robot, so that the model can learn to predict the action tokens based on sensor info. Here's a blog post from a researcher in this field reviewing a relevant paper: https://evjang.com/2023/06/22/robocat.html https://evjang.com/2023/06/22/robocat.html