4 ms·

Right but is there any reason that this architecture won't work with more diverse data? Fundamentally it seems like their research is benchmarked around pick an

Right but is there any reason that this architecture won't work with more diverse data? Fundamentally it seems like their research is benchmarked around pick and place, so it makes sense to me that they would want to prove out transformer models could work in this space. Knowing transformers, it's probably safe to say that with more diverse training data, it will be able to scale to more complex controls in more complex embodied robots. While I would love to see them working on robot chefs, I can appreciate that they want to start small. And regardless, it doesn't seem like there are any constraints outside of data for this architecture to work in more and more domains.

polygamous_bat 4y ago
> Knowing transformers, it's probably safe to say that with more diverse training data, it will be able to scale to more complex controls in more complex embodied robots. While I would love to see them working on robot chefs, I can appreciate that they want to start small. Except they are (were?) one of the biggest spenders in the entire field. While it may seem easy to hand-wave away "Just add more data!", robot data is way more expensive to get than language/image data, since people don't generate that naturally as they browse the internet. If their current operation was already too expensive for Google to keep running (as evidenced by Google shutting down this research arm), imagine what would happen if they proposed "let's spend a couple more orders of magnitude to get data!" All I am saying is that they are going for an unambitious project that looks cool to the outsiders, with buzzwords like LLM. There are work by others [1] that are much more impressive in terms of manipulation diversity that would probably be a much better bet to add more data into, since they show a much more promising route to the future. [1] https://twitter.com/chenwang_j/status/1628792565385564160?t=yaqu8YHTb8vTnmqgMRW2dQ&s=19 https://twitter.com/chenwang_j/status/1628792565385564160?t=...
- robitsT6 4y ago
  I can't really comment on the expenditure or the general strategy of Google's robotics teams, I know they closed down every day robotics, but that seemed to be specifically about the hardware. Their software efforts still seem to be going strong, and I don't really think it's fair to say that demonstrating transfer learning in an embodied model none the less, is unambitious - nor does that seem like it's reflected in the results - but to be honest, I'm more or less a layman when it comes to this so I'll defer to you and keep what you're saying in mind. To that end, if you're still feeling up to it, maybe you could tell me what your thoughts are with efforts like this from Google: https://diffusion-rosie.github.io/ https://diffusion-rosie.github.io/ Seems to compare (somewhat) to Mimicplay, in that it is attempting to create more data for "cheap", even if it's not "real" control data.

polygamous_bat 4y ago

> Knowing transformers, it's probably safe to say that with more diverse training data, it will be able to scale to more complex controls in more complex embodied robots. While I would love to see them working on robot chefs, I can appreciate that they want to start small. Except they are (were?) one of the biggest spenders in the entire field. While it may seem easy to hand-wave away "Just add more data!", robot data is way more expensive to get than language/image data, since people don't generate that naturally as they browse the internet. If their current operation was already too expensive for Google to keep running (as evidenced by Google shutting down this research arm), imagine what would happen if they proposed "let's spend a couple more orders of magnitude to get data!" All I am saying is that they are going for an unambitious project that looks cool to the outsiders, with buzzwords like LLM. There are work by others [1] that are much more impressive in terms of manipulation diversity that would probably be a much better bet to add more data into, since they show a much more promising route to the future. [1] https://twitter.com/chenwang_j/status/1628792565385564160?t=yaqu8YHTb8vTnmqgMRW2dQ&s=19 https://twitter.com/chenwang_j/status/1628792565385564160?t=...

robitsT6 4y ago
I can't really comment on the expenditure or the general strategy of Google's robotics teams, I know they closed down every day robotics, but that seemed to be specifically about the hardware. Their software efforts still seem to be going strong, and I don't really think it's fair to say that demonstrating transfer learning in an embodied model none the less, is unambitious - nor does that seem like it's reflected in the results - but to be honest, I'm more or less a layman when it comes to this so I'll defer to you and keep what you're saying in mind. To that end, if you're still feeling up to it, maybe you could tell me what your thoughts are with efforts like this from Google: https://diffusion-rosie.github.io/ https://diffusion-rosie.github.io/ Seems to compare (somewhat) to Mimicplay, in that it is attempting to create more data for "cheap", even if it's not "real" control data.