5 ms·
The speech to servo movement is impressive as others pointed out. What strikes me as amazing is the speed with which it is performing tasks that require dexteri
by daveguy 3y ago
The speech to servo movement is impressive as others pointed out. What strikes me as amazing is the speed with which it is performing tasks that require dexterity. This is the first object manipulation robot demo I have seen that didn't require speeding up the video for it to look "natural".
- huytersd 3y agoHow though. It’s probably just predefined actions that are triggered by the LLM output. At the same time it would be impressive if the LLM determined the right function to call in real time, was able to deal with the ambiguity in placement of the garbage and bonus points if it could do that in a scenario that wasn’t hardcoded to exactly standing behind that table in that spot.
- famouswaffles 3y agoThe robot movements are separate end-to-end neural networks. They're triggered by the LLM but aren't hardcoded.
- margorczynski 3y agoWhy does it then after performing an action or whole plan return to the "default" position with the hands in that strange stance? Looks kinda like there is some "hard coded" flow that simply uses the LLM(s) to perform actions.
- famouswaffles 3y agoThat's a closed loop operation. It's pretty common to train these robots on closed loop demonstrations.
- huytersd 3y agoDo you know this for a fact or are you speculating?
- StarCyan 3y agoOne of the Figure engineers described the system at a high level: https://twitter.com/coreylynch/status/1767927506332721154 https://twitter.com/coreylynch/status/1767927506332721154
- m3kw9 3y agoIs it just me or are people easily impressed by these robot demo that moves nothing like humans and doing the simplistic tasks like passing a large object. The freaking Apple is within his arms reach and he asked to pass the Apple. I almost laughed out loud.
- bigyikes 3y agoEasily impressed? These are difficult tasks! Even being able to grip the apple without crushing it is impressive. Not sure anything here is state of the art, but that doesn’t make it easy.
- darth_avocado 3y agoI am 100% convinced that the demo is partly fabricated, or at least very far from the robot’s capabilities. The inference part was probably true, but the dexterity would put them waaaaay ahead of what robots are capable of, if not programmed finely for a specific task. Industrial robots obviously have that kind of dexterity and precision because they’re specifically programmed for a task. General purpose robots however are nowhere as close to this level of accuracy or fluidity in their movements.
- whatever1 3y agoProbably some sort of imitation learning
- fragmede 3y agoIt is. Have you seen ok robot? https://ok-robot.github.io/ https://ok-robot.github.io/
- binoct 3y agoOK-Robot is super impressive, but there is a huge difference in the manipulation ability OK-Robot shows with a simple 1-DOF gripper and parallel-axis arm to pick up objects and place them largely on flat surfaces, and the human-analog arms and fingers of Figure 01 picking up a plate in both hands and placing it in the right slot in a drying rack, or dropping the apple into a person's moving hand. It would be absolutely amazing if they really are at that level of manipulation in general, and it would put them vastly beyond what anyone has been able to do date. However robotics demos have a great history of being a mix of slight of hand (partial/full tele-op), heavily cherry picked, or tuned to a extremely specific example. Because it's such a leap being implied by this video, it's reasonable to want significantly more evidence before believing they can do this type of interaction and manipulation in a general way. But even if it is heavily leaning say on imitation learning for this exact scenario, there are tons of potential applications for this level of capability.
- fragmede 3y agoYes, there are two half’s to this. This is being able to understand the command which OK robot shows is possible, and then there’s the control of motors and actuators as you pointed out. Not needing a team of engineers to laboriously write gcode to manipulate motors is part two and is the other astounding thing here. I haven’t been following this sort of thing, so I have no idea what previous state of the art was, so I can’t judge how much of an advancement it really is.
- StarCyan 3y agoYeah the motion looks way more fluid then similar systems I've seen before.
- btbuildem 3y agoThe slightly dismissive nature of the toss seemed like a hack or maybe a deliberate way to speed up the [pick up, carry, drop] pathway.