5 ms·

Figure 01 has learned to make coffee

23 points by hubraumhugo 3y ago

wepple 3y ago
Can anyone speak to the details & limitations here? Can it truly learn via video visualization of humans performing an action and replicate? During “training”, what feedback does it use, and what is the goal function to signify success?
- wepple 3y ago
  Did a quick search for anything at all with more detail than the twitter post, and couldn’t find anything. They sure do have a ton of hip marketing image. But they also seem to have some clued up engineers. So who really knows
steveBK123 3y ago
Isn't this just a slightly more advanced / evolved factory robotic arm / pick&place type mechanism? The robot is not "making coffee". It is placed in front of a Keurig, coffee cup already in place, water already filled, powered on.. with a pod in front. It is doing essentially 3 things - pick&place pod, secure latch, press go.
- graypegg 3y ago
  I think the thing they want to communicate here is the training method. Rather than a pick and place tool path, this is just trained on videos of someone using a coffee machine, and imitating that. Which… now that I think about it… is probably drawing a skeleton over top of the video Kinect-style, and… drawing a tool path… But at least it seems to understand when it’s made a mistake, which is interesting.
  - steveBK123 3y ago
    Right I mean, for B2B industrial robotic arm applications this might actually be a big improvement / breakthrough. I just find it annoying how many of these AI companies are pumping goofy B2C applications. Like they make it look as though they are building a breakthrough humanoid robotic assistant or something, but really that is still very very far away and not at all related to the "video trained robotic arm" breakthrough they are trying to communicate.
  - WiSaGaN 3y ago
    I’m wondering what cost function it’s using. Does it use a simulator?
- fabian2k 3y ago
  I don't find this particularly impressive as well. Anything except a pod machine would likely present the robot with tasks it can't handle like e.g. handling actual beans or ground coffee and filling water. And most household tasks would often involve this kind of task that is not entirely straightforward as your house and its contents aren't designed with limited robots in mind.
  - steveBK123 3y ago
    This Keurig queued up perfectly use case is fundamentally unimpressive because I could expect my 5 year old nephew to pass it with 1 minute of verbal direction or watching me do it once, not 10 hours of deep learning. Now expand the Keurig use case to the real world. Present it with a machine: * filled / not filled with water * used pod bin empty / used pod bin filled * pods available / pods in a box on counter / pods in a box in the cabinet * cup on counter / cup in cabinet / cup in dishwasher / cup in sink We now have ~50 permutations of even the simple Keurig use case and let me know how many years of training to accomplish even this. All of this to make fairly bad coffee. We haven't even gotten into your example of actually making real coffee.
    - xbar 3y ago
      I am more impressed than you. A general purpose robot can learn new tasks in a modest amount of time. As you argue, fluidly chaining sequences of learned tasks together is tricky, but current AI technology is capable of solving nearly all those problems within a handful of years. With centralized learning labs, over-the-air updates, and a 2x or 3x improvement in execution speed of precision tasks, I think this type of solution is going to be ubiquitous within the decade.
cjs_ac 3y ago
I like the fact that it doesn't know what to do with its hands while waiting for the Keurig machine to work.
probablypower 3y ago
If this is making coffee, I don't want to live in this android future.
lopatin 3y ago
How long realistically until we have a commercial robot maid that can do laundry and fold/put away clothes?
- martythemaniak 3y ago
  My opinion is towards the end of this decade. Hardware is mostly there - the collection of actuators, metal bits, batteries, cameras and inference chips means you can probably build this today at scale for the price of mid-range car, but of course it wouldn't do anything useful. But given the rate of progress in AI over the last few years and the increasing rate of investment in the field, I'd be shocked if another 6 years of progress would amount to nothing.
graypegg 3y ago
Why does every hype-funded general purpose robot (Tesla bot, this robot) NEED to be shaped like a human. I get that our built environment is designed for humans, and some sci-fi loving part of me loves it. But it feels like theatrics. Boston dynamics makes utilitarian robots, that all have a seemingly purpose built design to them. Even the bipedal + 2 arms + a head robot Atlas, isn’t forced to constrain itself to human proportions. By comparison this robot seems pretty unstable. I wonder if that has to do with humans and robots being fundamentally made of different materials with very different properties. I probably answered my own question by calling it “hype-funded”.
- cjs_ac 3y ago
  When we talk about the 'Uncanny Valley', we talk about things that are very humanoid, but not quite humanoid enough. But the obvious target market for humanoid robots is retail and hospitality, where there's a high degree of human interaction. In that case, a lot of 'humanness' is needed to suggest to customers that 'yes, you can talk to the robot, and it will do things'. The design of this robot[0], on the other hand, is very un-human (poorly-drawn face aside), telling customers that it has a specific job that doesn't involve them. [0] https://eftm.com/2020/09/woolworths-robot-for-checking-the-shelves-and-the-isles-for-spills-85284 https://eftm.com/2020/09/woolworths-robot-for-checking-the-s...
  - graypegg 3y ago
    But doesn’t that make the value in these human-y robots, simply impressing humans? I’m fine with theatrics and awe being the main purpose, but it’s really just the front end to get humans speaking to a hunk of metal without feeling weird. What difference does it make in utility if it was a screen with some 3D model talking back at you? That’s a lot easier to make feel “human”. And now you’ve freed up the design of the robot behind the scenes that actually does the task to be shaped in whatever format is useful. Is the retail space going to get similar value from the robot once the initial “oh wow robot cool” factor wears off?
  - steveBK123 3y ago
    I mean Japan was a decade ahead of us there https://en.wikipedia.org/wiki/Pepper_(robot) https://en.wikipedia.org/wiki/Pepper_(robot) I don't know that replacing $15/hr at-will labor with a $50k machine that can be vandalized/stolen/etc is going to have a great cost-benefit for most US retailers...
lnxg33k1 3y ago
I mean "learned to make coffee" is a bit exaggerated? It just placed a capsule in the machine and pressed a button? Possibly the coffee machine has contributed much more in making that coffee and doesn't even have an AI system :D
martythemaniak 3y ago
To me, there's only one metric that matters for humanoid robots: Can it earn its keep? A $30K robot with 10 years useful lifetime, and a 10 year 5% mortgage is $10.40 per day. Minimum wages range from $7 to $15, so the US national average would be around that. If your robot can do 1 hour of minimum wage labour per day, people will buy it. This is a pretty low bar. Light cleaning (dusting, wiping down surfaces, vacuum), loading the dishwasher, doing laundry, tidying up after kids, etc. These tasks don't demand AGI, agility, adeptness, speed, accuracy, etc.
- graypegg 3y ago
  Do keep in mind that a comparison like that doesn’t suppose the null case, where someone dusts their own home for free, loads the dishwasher, and trains the kid AI to clean up their own messes with thoughtfully deployed apple-sauce-snack-based positive reenforcement training. I do agree these tasks don’t demand AGI. I don’t really want my “robot that walks around and does 11$ worth of tasks I don’t want to deal with” to feel emotions.
Hizonner 3y ago
From 10 hours of video, it learned to pick up a pod carefully placed on a counter in front of it, insert said pod into a Keurig machine also carefully placed on the counter in front of it, and push the button. From the hype, I was expecting it to pass the full Woz test: 1. Walk into a strange house and find the kitchen. 2. Find whatever coffee is available, including knowing likely places to look. Choose the best among multiple options based on the best available information about the drinker's likely preferences. 3. Find whatever coffee maker is available, get it out without screwing anything else up, assess its condition, clean it if-and-only-if necessary, plug it in, figure out its controls. 4. Similarly find filters or other supplies if necessary. 5. Grind the coffee if necessary to match the coffee maker and the desired results. 6. "Make the coffee" in the sense these guys are trying to claim is a breakthrough. 7. Serve it, or at least put it it in a cup and tell the user. 8. Put everything away and clean up 9. Take any available feedback on making it better next time.
- steveBK123 3y ago
  Yes, this is "making coffee" the same way it is "making dinner" to place an unpacked, frozen Hungry Jack TV dinner in the microwave and hit "start". You could queue up a dozen of kitchen use cases that individually look impressive if you don't think at all. But they are just rhyming variations of "pick&place, then hit the button".
juancn 3y ago
It feels so 80's... the motion is so weird and unnatural, not like Boston Dynamics where it feels like they built a weird animal instead.
freitasm 3y ago
Disappointed. The robot t is not "making coffee". It is feeding a capsule to a machine and pressing a button. I expected the robot would be grinding beans, filling the coffee holder, tamping, fitting the holder to a steam machine and pulling an espresso for a certain time. I could even understand fitting a coffee filter to a holder, dropping ground coffee and dripping hot water until a certain level is reached. Those things require a lot more decisions than "open machine, drop capsule, close machine, press button".

23 points by hubraumhugo 3y ago

Can anyone speak to the details & limitations here? Can it truly learn via video visualization of humans performing an action and replicate? During “training”, what feedback does it use, and what is the goal function to signify success?

wepple 3y ago
Did a quick search for anything at all with more detail than the twitter post, and couldn’t find anything. They sure do have a ton of hip marketing image. But they also seem to have some clued up engineers. So who really knows

steveBK123 3y ago

Isn't this just a slightly more advanced / evolved factory robotic arm / pick&place type mechanism? The robot is not "making coffee". It is placed in front of a Keurig, coffee cup already in place, water already filled, powered on.. with a pod in front. It is doing essentially 3 things - pick&place pod, secure latch, press go.

graypegg 3y ago
I think the thing they want to communicate here is the training method. Rather than a pick and place tool path, this is just trained on videos of someone using a coffee machine, and imitating that. Which… now that I think about it… is probably drawing a skeleton over top of the video Kinect-style, and… drawing a tool path… But at least it seems to understand when it’s made a mistake, which is interesting.
- steveBK123 3y ago
  Right I mean, for B2B industrial robotic arm applications this might actually be a big improvement / breakthrough. I just find it annoying how many of these AI companies are pumping goofy B2C applications. Like they make it look as though they are building a breakthrough humanoid robotic assistant or something, but really that is still very very far away and not at all related to the "video trained robotic arm" breakthrough they are trying to communicate.
- WiSaGaN 3y ago
  I’m wondering what cost function it’s using. Does it use a simulator?
fabian2k 3y ago
I don't find this particularly impressive as well. Anything except a pod machine would likely present the robot with tasks it can't handle like e.g. handling actual beans or ground coffee and filling water. And most household tasks would often involve this kind of task that is not entirely straightforward as your house and its contents aren't designed with limited robots in mind.
- steveBK123 3y ago
  This Keurig queued up perfectly use case is fundamentally unimpressive because I could expect my 5 year old nephew to pass it with 1 minute of verbal direction or watching me do it once, not 10 hours of deep learning. Now expand the Keurig use case to the real world. Present it with a machine: * filled / not filled with water * used pod bin empty / used pod bin filled * pods available / pods in a box on counter / pods in a box in the cabinet * cup on counter / cup in cabinet / cup in dishwasher / cup in sink We now have ~50 permutations of even the simple Keurig use case and let me know how many years of training to accomplish even this. All of this to make fairly bad coffee. We haven't even gotten into your example of actually making real coffee.
  - xbar 3y ago
    I am more impressed than you. A general purpose robot can learn new tasks in a modest amount of time. As you argue, fluidly chaining sequences of learned tasks together is tricky, but current AI technology is capable of solving nearly all those problems within a handful of years. With centralized learning labs, over-the-air updates, and a 2x or 3x improvement in execution speed of precision tasks, I think this type of solution is going to be ubiquitous within the decade.

cjs_ac 3y ago

I like the fact that it doesn't know what to do with its hands while waiting for the Keurig machine to work.

probablypower 3y ago

If this is making coffee, I don't want to live in this android future.

lopatin 3y ago

How long realistically until we have a commercial robot maid that can do laundry and fold/put away clothes?

martythemaniak 3y ago
My opinion is towards the end of this decade. Hardware is mostly there - the collection of actuators, metal bits, batteries, cameras and inference chips means you can probably build this today at scale for the price of mid-range car, but of course it wouldn't do anything useful. But given the rate of progress in AI over the last few years and the increasing rate of investment in the field, I'd be shocked if another 6 years of progress would amount to nothing.

graypegg 3y ago

Why does every hype-funded general purpose robot (Tesla bot, this robot) NEED to be shaped like a human. I get that our built environment is designed for humans, and some sci-fi loving part of me loves it. But it feels like theatrics. Boston dynamics makes utilitarian robots, that all have a seemingly purpose built design to them. Even the bipedal + 2 arms + a head robot Atlas, isn’t forced to constrain itself to human proportions. By comparison this robot seems pretty unstable. I wonder if that has to do with humans and robots being fundamentally made of different materials with very different properties. I probably answered my own question by calling it “hype-funded”.

cjs_ac 3y ago
When we talk about the 'Uncanny Valley', we talk about things that are very humanoid, but not quite humanoid enough. But the obvious target market for humanoid robots is retail and hospitality, where there's a high degree of human interaction. In that case, a lot of 'humanness' is needed to suggest to customers that 'yes, you can talk to the robot, and it will do things'. The design of this robot[0], on the other hand, is very un-human (poorly-drawn face aside), telling customers that it has a specific job that doesn't involve them. [0] https://eftm.com/2020/09/woolworths-robot-for-checking-the-shelves-and-the-isles-for-spills-85284 https://eftm.com/2020/09/woolworths-robot-for-checking-the-s...
- graypegg 3y ago
  But doesn’t that make the value in these human-y robots, simply impressing humans? I’m fine with theatrics and awe being the main purpose, but it’s really just the front end to get humans speaking to a hunk of metal without feeling weird. What difference does it make in utility if it was a screen with some 3D model talking back at you? That’s a lot easier to make feel “human”. And now you’ve freed up the design of the robot behind the scenes that actually does the task to be shaped in whatever format is useful. Is the retail space going to get similar value from the robot once the initial “oh wow robot cool” factor wears off?
- steveBK123 3y ago
  I mean Japan was a decade ahead of us there https://en.wikipedia.org/wiki/Pepper_(robot) https://en.wikipedia.org/wiki/Pepper_(robot) I don't know that replacing $15/hr at-will labor with a $50k machine that can be vandalized/stolen/etc is going to have a great cost-benefit for most US retailers...

lnxg33k1 3y ago

I mean "learned to make coffee" is a bit exaggerated? It just placed a capsule in the machine and pressed a button? Possibly the coffee machine has contributed much more in making that coffee and doesn't even have an AI system :D

martythemaniak 3y ago

To me, there's only one metric that matters for humanoid robots: Can it earn its keep? A $30K robot with 10 years useful lifetime, and a 10 year 5% mortgage is $10.40 per day. Minimum wages range from $7 to $15, so the US national average would be around that. If your robot can do 1 hour of minimum wage labour per day, people will buy it. This is a pretty low bar. Light cleaning (dusting, wiping down surfaces, vacuum), loading the dishwasher, doing laundry, tidying up after kids, etc. These tasks don't demand AGI, agility, adeptness, speed, accuracy, etc.

graypegg 3y ago
Do keep in mind that a comparison like that doesn’t suppose the null case, where someone dusts their own home for free, loads the dishwasher, and trains the kid AI to clean up their own messes with thoughtfully deployed apple-sauce-snack-based positive reenforcement training. I do agree these tasks don’t demand AGI. I don’t really want my “robot that walks around and does 11$ worth of tasks I don’t want to deal with” to feel emotions.

Hizonner 3y ago

From 10 hours of video, it learned to pick up a pod carefully placed on a counter in front of it, insert said pod into a Keurig machine also carefully placed on the counter in front of it, and push the button. From the hype, I was expecting it to pass the full Woz test: 1. Walk into a strange house and find the kitchen. 2. Find whatever coffee is available, including knowing likely places to look. Choose the best among multiple options based on the best available information about the drinker's likely preferences. 3. Find whatever coffee maker is available, get it out without screwing anything else up, assess its condition, clean it if-and-only-if necessary, plug it in, figure out its controls. 4. Similarly find filters or other supplies if necessary. 5. Grind the coffee if necessary to match the coffee maker and the desired results. 6. "Make the coffee" in the sense these guys are trying to claim is a breakthrough. 7. Serve it, or at least put it it in a cup and tell the user. 8. Put everything away and clean up 9. Take any available feedback on making it better next time.

steveBK123 3y ago
Yes, this is "making coffee" the same way it is "making dinner" to place an unpacked, frozen Hungry Jack TV dinner in the microwave and hit "start". You could queue up a dozen of kitchen use cases that individually look impressive if you don't think at all. But they are just rhyming variations of "pick&place, then hit the button".

juancn 3y ago

It feels so 80's... the motion is so weird and unnatural, not like Boston Dynamics where it feels like they built a weird animal instead.

freitasm 3y ago

Disappointed. The robot t is not "making coffee". It is feeding a capsule to a machine and pressing a button. I expected the robot would be grinding beans, filling the coffee holder, tamping, fitting the holder to a steam machine and pulling an espresso for a certain time. I could even understand fitting a coffee filter to a holder, dropping ground coffee and dripping hot water until a certain level is reached. Those things require a lot more decisions than "open machine, drop capsule, close machine, press button".