3 ms·
This post didn't really convince me that robotics is inherently harder than generating text or images On the one hand we have problems where ~7B humans have be
by lairv 2y ago
This post didn't really convince me that robotics is inherently harder than generating text or images
On the one hand we have problems where ~7B humans have been generating data for 30 years every day (more if you count old books), on the other hand we have a problem where researcher are working with ~1000 human collected trajectories (I think the largest existing dataset is OXE with ~1M trajectories: https://robotics-transformer-x.github.io/ https://robotics-transformer-x.github.io/ )
Web-scale datasets for LLMs benefits from a natural diversity, they're not highly correlated samples generated by contractors or researchers in academic labs. In the largest OXE dataset, what do you think is the likelihood that there is a sample where a robot picks up a rock from the ground and throws it in a lake? Close to zero, because tele-operated data comes from a very constrained data distribution.
Another problem is that robotics doesn't have an easy universal representation for its data. Let's say we were able to collect web-scale dataset for one particular robot A with high diversity, how would it transfer to robot B with a slightly different design? Probably poorly, so not only does the data distribution needs to cover a high range of behavior, it must also cover a high range of embodiment/hardware
With that being said, I think it's fair to say that collecting large scale dataset for general robotics is much harder than collecting text or images (at least in the current state of humanity)