3 ms·

> We're probably still far from passing Steve Wozniak's Coffee Test Do you think? At this point I have the impression it's just a problem of dexterity and spee

> We're probably still far from passing Steve Wozniak's Coffee Test Do you think? At this point I have the impression it's just a problem of dexterity and speed. Understanding, planning and navigation seem basically solved.

jvanderbot 2y ago
Is this spoken from experience? My experience in robotics tells me otherwise. It's not so much the issue with repeatability (though that is significant), it is moreso the issue with handling novelty. I do not believe there exists a system, which if given sufficient time (to negate speed), and a perfect inverse/forward kinematics sovler, could walk into my house and make coffee. There's too many challenges of the kind "Seek information about ... " and "adapt a multi-step process to overcome ... ". One industrial process that is only now being automated is connected trailers to trucks. They have to connect a hose and a few lines. Two companies are struggling with this even now. (outrider and isee). Both well funded and staffed by intelligent folks, and have to coax a robotic arm into connecting a hose that we all know is there, but not where, to a port that we all know is there, but might be different than expected.
- throw310822 2y ago
  I was not thinking (much) about robotics but rather about a ChatGPT-style LLM processing video or frames and asked to navigate a random environment and find a way to make coffee. I didn't try but it doesn't sound far from their current capabilities. Then of course the manipulation of objects is still tricky and needs improvements, but the "general intelligence" needed to adapt to a novel environment is already there.
  - jvanderbot 2y ago
    Well, then you weren't thinking about the Wozniak coffee test. > "The Coffee Test has been proposed, and is attributed to Steve Wozniak, one of the founders of Apple Computers. According to Steve, this test would require a robot, not a computer screen. The robot would need to locate the kitchen and brew a pot of coffee in a random house that it had never seen before. And more specifically, I disagree that such a system would work just from video frames. The issue isn't so much "Coming up with text that sounds right", it's more about state and environment representation, novelty, planning multi-step processes, and so on. A good test of if this were easy, is can it be done in game engines, which negate all the speed / repeatability / controls complexity almost entirely. Plug an LLM into a game engine with a hovering arm that can grab and place and push buttons, and have it "make coffee" in procedural generated kitchens, coffee makers, etc. Bonus points for using POV imagery, bonus points for using bags of grains that might be coffee but might not. Similar weight, feel, placement. Hell my coffee, coca, sugar, and flour are right next to each other. That would be a landmark top tier research accomplishment and top HN article for days (the real accomplishment!)

jvanderbot 2y ago

Is this spoken from experience? My experience in robotics tells me otherwise. It's not so much the issue with repeatability (though that is significant), it is moreso the issue with handling novelty. I do not believe there exists a system, which if given sufficient time (to negate speed), and a perfect inverse/forward kinematics sovler, could walk into my house and make coffee. There's too many challenges of the kind "Seek information about ... " and "adapt a multi-step process to overcome ... ". One industrial process that is only now being automated is connected trailers to trucks. They have to connect a hose and a few lines. Two companies are struggling with this even now. (outrider and isee). Both well funded and staffed by intelligent folks, and have to coax a robotic arm into connecting a hose that we all know is there, but not where, to a port that we all know is there, but might be different than expected.

throw310822 2y ago
I was not thinking (much) about robotics but rather about a ChatGPT-style LLM processing video or frames and asked to navigate a random environment and find a way to make coffee. I didn't try but it doesn't sound far from their current capabilities. Then of course the manipulation of objects is still tricky and needs improvements, but the "general intelligence" needed to adapt to a novel environment is already there.
- jvanderbot 2y ago
  Well, then you weren't thinking about the Wozniak coffee test. > "The Coffee Test has been proposed, and is attributed to Steve Wozniak, one of the founders of Apple Computers. According to Steve, this test would require a robot, not a computer screen. The robot would need to locate the kitchen and brew a pot of coffee in a random house that it had never seen before. And more specifically, I disagree that such a system would work just from video frames. The issue isn't so much "Coming up with text that sounds right", it's more about state and environment representation, novelty, planning multi-step processes, and so on. A good test of if this were easy, is can it be done in game engines, which negate all the speed / repeatability / controls complexity almost entirely. Plug an LLM into a game engine with a hovering arm that can grab and place and push buttons, and have it "make coffee" in procedural generated kitchens, coffee makers, etc. Bonus points for using POV imagery, bonus points for using bags of grains that might be coffee but might not. Similar weight, feel, placement. Hell my coffee, coca, sugar, and flour are right next to each other. That would be a landmark top tier research accomplishment and top HN article for days (the real accomplishment!)