3 ms·
How though. It’s probably just predefined actions that are triggered by the LLM output. At the same time it would be impressive if the LLM determined the right
by huytersd 3y ago
How though. It’s probably just predefined actions that are triggered by the LLM output. At the same time it would be impressive if the LLM determined the right function to call in real time, was able to deal with the ambiguity in placement of the garbage and bonus points if it could do that in a scenario that wasn’t hardcoded to exactly standing behind that table in that spot.
- famouswaffles 3y agoThe robot movements are separate end-to-end neural networks. They're triggered by the LLM but aren't hardcoded.
- margorczynski 3y agoWhy does it then after performing an action or whole plan return to the "default" position with the hands in that strange stance? Looks kinda like there is some "hard coded" flow that simply uses the LLM(s) to perform actions.
- famouswaffles 3y agoThat's a closed loop operation. It's pretty common to train these robots on closed loop demonstrations.
- huytersd 3y agoDo you know this for a fact or are you speculating?
- StarCyan 3y agoOne of the Figure engineers described the system at a high level: https://twitter.com/coreylynch/status/1767927506332721154 https://twitter.com/coreylynch/status/1767927506332721154