13 ms·
Benjie's Humanoid Olympic Games
- robobenjie 1y agoHey Hacker News. Curious if folks feel like I'm missing categories of challenge.
- mjq7 1y agoHow about something with unpacking items from a shopping bag, i suspect the difference in bags (standard plastic, reusable etc) and certain items can really crank the difficulty. It can also create a good time of a story - open the door to get the grocery delivery, unpack the delivery etc.
- deleted 1y ago[deleted]
- meeech 1y agosomething requiring navigating stairs while holding something full like a laundry basket. bronze - straight stairs, silver - one 90deg turn. gold - spiral. something requiring co-ordination between 2 robots. think relay race which the olympics has. So say, moving a couch together. btw love the idea and the silver body suit. good stuff.
- robobenjie 1y agoOoh. I like full body manipulation. Humans use hips & elbows to move laundry baskets. Two robot collaboration is good too. I wonder who I can convince to wear another silver suit.... :)
- meeech 1y agoif you find yourself in montreal, message me - i will do it, for science!
- levocardia 1y agoMaybe careful application of large amounts of force? Opening a jar, peeling garlic, splitting a squash, opening a soda can. This category seems like a good test of "grip" strength + force feedback + sense of touch.
- amirhirsch 1y agoI like these benchmarks and the videos are funny! Consider examples using building tools like screwing in a drywall screw, or hammering a nail, using a paint roller, caulking a sink, minor plumbing repair with a torch and solder. These differ enough in terms of forces, state changes, and combined dexterity/acuity (two-handed proprioception) from the windex, sandwich and key examples Ikea product assembly for gold medal.
- anentropic 1y agoIkea assembly often needs two people, so robot+robot and person+robot co-operation challenges is a whole area to be explored
- nancyminusone 1y agoI'd go for something in the manipulation of ropes or wires. State of the art seems to be that they can untangle a loosely knotted cord. Untying a short rope with a tightly pulled overhand knot in the middle seems like it's decades away. You have to be able to grip it well enough, then twist the rope and push (even though every physicist says pushing a rope is impossible).
- robobenjie 1y agoInteresting. Futurism is super hard, but "decades" too far away to me. I think with strong 2 finger grippers this is probably close to state of the art, especially with a wrist force sensor, like the TRI setup.
- calepayson 1y agoBelaying a climber would be a hilarious (and fraught) gold medal challenge.
- inasio 1y agoThere are already robots that do that (autobelayers)...
- tintor 1y agoNot really. Auto belay device can't hold the climber up on the wall, like human belayer can, so that climber can rest and try again. Auto belay device can't do lead belay.
- jayGlow 1y agoThey're not wide spread but they can do lead belay, I've seen a few videos of it online https://climbingbusinessjournal.com/could-your-gym-be-getting-a-lead-climbing-auto-belay-soon/ https://climbingbusinessjournal.com/could-your-gym-be-gettin...
- 1y ago
- nashashmi 1y agoRobots currently do things easy and faster to do for robot but difficult to do for man.
- tintor 1y agothreading a needle (different thread thickness and hole sizes)
- vasco 1y agoCarving a wooden spoon in hand.
- robotresearcher 1y agoStandard evening family home tidy/reset - toys, books, clothes, shoes away in their places. All over the house. Oh, and load/unload dishwasher. Same with laundry machines. Along with folding laundry, these are the domestic robot equivalents of 'de-mining' and 'search and rescue': the classic motivating use cases for mobile autonomous robots.
- Mistletoe 1y agoI love your list and it makes me think we are so far away from these things ever being feasible/cost effective compared to just hiring a poor person to do it. And the world is making a lot of poor people right now.
- fragmede 1y ago(HN link on Substack points at empty page instead of this one, at least before I made this comment.) What I think is missing is marathon events. Biathalons and Triathalons. We all know LLMs have a rather limited context window. Thus seeing robots do longer chains of events would be interesting to see that they're capable than a possibly rigged demo. Something like: move a stack of boxes from one room to another. The boxes at the end also need to be stacked up. or how about pick up a box, go up some stairs, open a door, and put the box on a shelf on the other side. Also, the real world is sloppy and messy and dirty and, to be real, kinda janky sometimes. Gold for unlocking a door with a key at a well-maintained office complex, (and opening it, and walking through it) is one thing, because facilities is going to replace the lock before it gets old and needs replacing, and we can assume the door fits in the frame properly so it doesn't need to be shoved or lifted up or yanked in order to be opened is easier than. But the real world is messy and sloppy and you gotta jiggle the key in just the right way in order to get it to work. Closing the door (assuming the robots weren't raised in a robot barn) is also harder than it looks if the door is shitty and needs a proper slam in order to be fully closed. Also, the robot locking the door behind itself after it comes in. Scanning a key card and opening a door, but the first try fails. We're a long way from a general robot that can screw a simple screw together like you would to assemble Ikea furniture. Object recognition. Gather only the dishes from a messy coffee table and put them in the dish bin. Pick up only the clothes from a messy floor and bed, and put them in the hamper. Dump a hamper of clothes onto a table, and sort out stuff that doesn't want to go into the washing machine. Terrain traversal. Just walk 500 ft, but theres increasing levels of obstacles in the way. We all saw Boston dynamics robot parkour videos, but what I want to see is a robot make it from the front door of Simpsons house to the kitchen in the back, but it's got to go through the living room, but it's hella messy, with Maggie and Bart and Lisa’s crap strewn all over, Homer’s got some beer bottles, some empty, some full, all over the floor and on the table, and all the robot has to do is walk from one side of the room to the far side of the room without stepping on anything, or knocking anything over. (Simpsons merely being a home layout that's familiar to most people. Doesn't need to actually be them.) Ducking under a low ceiling. Climb over a barrier, of varying shapes and sizes. Other loocomotion. how much weight in its arms in front of it, holding a 5-lb briefcase with one hand while walking. Can it carry something on its back? What's the limit? Can it give piggyback rides? A category for simulated. Let companies show off their robot's kinematics control systems, so have something on the level of CoppeliaSim, so the motors and the gears and the actuators are themselves simulated, vs a simple 3d video game where they are not. Plug their model into the simulated robot and see how well it just walks. If we remember QWOP, it's harder than it looks! Obviously it's not going to be totally 100% accurate to the real world. The benefit of this is it lets people complete from all over world without having to replicate a very specific setup in the physical world, and compete from wherever they live am not have to fly to your facility to test, opening up a whole new world of contestants because they can now compete because they can afford it now. At the end of the day, the most important challenge is, can it pick up a battery from the shelf, swap it with one of the two in its chassis, and put the dead one it just pulled out onto the charger?
- iainmerrick 1y agoMaking a bed. You need to manipulate a large sheet, and you probably need to move around, bend down and lean over to reach all the corners. Bonus points for neat hospital corners on a flat sheet. Putting pillows in pillowcases is another fun one. Usually pretty easy, probably a bronze medal. Gold medal: put a UK super king size duvet inside a duvet cover. It's huge and awkward, there are buttons, and it's almost but not quite square (why??) so there's a good chance you'll get it round the wrong way and have to rotate it 90 degrees.
- robobenjie 1y agoI think this is my favorite suggestion. All of them are super hard, but - Bronze: pillow in pillow case - Silver: Make a bed with fitted sheet, flat sheet and blanket - Gold: put a duvet in a duvet cover Feels like it would be a good option for a round 2...
- duck 1y agoMost humans would be hard pressed to get a gold medal in this one.
- iainmerrick 1y agoExactly, this is one where it might actually be quicker to wait for the robots.
- exasperaited 1y agoYes why!? There seriously needs to be an industry standard for marking up the long side of a king size duvet.
- muvlon 1y agoSomething I still struggle with as a human: Cracking an egg into a bowl. No making a mess and no egg shell in the bowl. Gold Medal: Separate the egg white from the yolk!
- bethekidyouwant 1y agoCrack it on a flat surface not an edge
- thaumasiotes 1y ago> Gold Medal: Separate the egg white from the yolk! This is done now by using a bowl that does the job. Is that supposed to be an additional challenge for the robot?
- iainmerrick 1y agoSeparating the yolk is the easy part once you've got it into the bowl with no mess -- a half-eggshell is the perfect tool to scoop the yolk out.
- tintor 1y agoreplacing an inner tube on bicycle wheel - strong forces / both arms / level use / dexterity / problem solving
- robobenjie 1y agoThis one is really hard for me...
- vessenes 1y agoI'm on the ag robot bandwagon lately. I'd like to see some fruit picking and weeding olympic challenges. Maybe slug removal.
- AndrewKemendo 1y agoThanks for putting this together. This IMO is 1000x better than any other AI challenges to date. ARC-AGI is bullshit and has nothing to do with reality. I’ll likely use some of your tests for our robotics (not just humanoid) testing in the future at least as some baselines. Also I really liked you dressing up as a robot - that’s very fun and really reflects the point of robotics: replace human action for all tasks. My suggestion: Identify a collapsed person in the home and render first aid (I need this because I have epilepsy and live alone) Bronze: ID collapse and call emergency services Silver: Bronze tasks + manipulate person into appropriate recovery position Gold: Communicate details to emergency personnel and playback previous hours of interactions
- Symmetry 1y agoDeforming solid objects. Breaking twigs to set lengths, chopping vegetables, cracking eggs.
- thenthenthen 1y agoTie your shoe laces? Pet a cat. Open one of those scissors that are sold in those fused plastic boxes, requiring a scissor to open. Opening a packet of tissues (wet ones for extra challenge). Cook rice. Throw out the trash.
- dmillard 1y agoInteresting post and great reference to [1] about why laundry hits a sweet spot of capability. Interestingly the repeated critiques in the article are about sensor richness: primarily force feedback and tactility, which indicates lacking hardware. Software only robotics has a long and fraught history, but it really feels to me that current industrial hardware could be driven more intelligently without much change. No doubt the "ideal" robot for any given task requires developments in both. I'm also curious about safety, since generally capable mechanisms need a multilayered safety stack that includes semantics, and cobot certification is likely not enough anymore. Examples: feeding someone the wrong pill, pouring a glass of water into electronics, cutting vegetables vs fingers. [1] https://substack.com/redirect/82d94852-76b6-4b0d-8595-86e46a29bbba https://substack.com/redirect/82d94852-76b6-4b0d-8595-86e46a...
- HarshP123 1y agoThis is amazing!!
- jerrybmarchant 1y agoWhat are your thoughts on this? Where do we stand? https://substack.com/@isitpropaganda/note/c-167073531?utm_source=notes-share-action&r=2flx18 https://substack.com/@isitpropaganda/note/c-167073531?utm_so...
- bee_rider 1y agoRather than teaching your robot to fold inside-out clothes, you should teach it to attack people who put their clothes in the hamper inside out.
- arscan 1y agoI think my shirts just automatically get inside-outted in my washer/dryer? I certainly don’t put them in that way, and it seems like I spend a lot of righting them when putting away laundry.
- bee_rider 1y agoTry putting some in inside-out and seeing if they get turned outside-out. I’m also curious which device does it! We should work this out, and also we will need a lot of robots to go after the manufacturer.
- blauditore 1y agoI agree, but in some cases it makes sense to protect the print. Although I habe no idea if it actually helps.
- dylan604 1y agoIf it's vinyl applied with heat (numbers on a jersey as an example), they recommend turning inside out. That way, if it gets too hot in the dryer, it only sticks to itself instead of to other garments. Losing the one garment is better than multiple.
- toast0 1y agoI've seen suggestions that having your clothes inside out in the wash helps them get cleaner. And if it keeps the AIs under control, you know, benefits.
- nashashmi 1y agoClothe instruction care sometimes requires washing clothes inside out.
- bnjmn 1y agoHere's a use case that seems more science fictional to me (as the parent of a 2yo) than warp drive: a robot that can gently restrain an uncooperative human baby while changing its diaper, with everything that entails: identifying and eliminating all traces of waste from all crevices, applying diaper cream as necessary, unfolding and positioning the new diaper correctly and quickly, always using enough but never too much force... not to mention the nightmare of providing any guarantees about safety at mass-market scale. Even one maimed baby, or even just a baby some robot neglects to prevent from falling off the changing table, is game over for that line of robots. Is there any research program that could claim to tackle this? It's so far beyond folding laundry and doing dishes, which are already quite difficult. I wouldn't bet my life on this tech _never_ materializing, but I would mistrust anyone who claimed it was feasible with today's tech. It calls for an entirely different kind of robotic perception, feedback, and control.
- robobenjie 1y agoThis is a great one. The manipulation is hard, but we're probably on a trajectory to be able to do it in 1-3 years if you were tolerant of some risk to the baby, but, of course, your tolerance for injuring babies is basically zero. I think 'risk & reliability' is a good potential category: there is the bar of 'got it to do a task reliably enough that we got a video' and the bar of 'got it to do a task reliably enough that I'd risk an infant in its grippers.
- dylan604 1y ago> your tolerance for injuring babies is basically zero. Um, no it's not. Is absolutely zero tolerance. There is not weasel words out of this. If a robot was to cause any pain to the baby, there would be no remorse. There would be no front of mind thoughts to not repeat the same thing the next time. There would be no guilt for causing pain to the baby. Why you would "basically" this the way you have is disturbing.
- poly2it 1y agoI don't think the parent comment advocates for hurting babies. It just, probably correctly, states that cherry picked examples won't be representative of roboty safety with infants in the next years, but that true safety will improve over time as well.
- XCSme 1y agoI don't know, for me it looks like the demo robot already does it quite well.
- byearthithatius 1y agoThink about how fast progress is being made now. When I was a kid in the early 2000s we would see some basic robot progress on movements (almost always from Boston Dynamics or sometimes China) and we thought it was incredible. Robot dogs running was amazing and five or so years later a backflip blew our minds. Those robots were specially designed and didn't look humanoid. Now we have bi-pedal humanoid robots and they walk and move fairly capably - even able to get up after falls. Now within the last year I have seen them learn Kung Fu, become really fast at getting up, become resistant to being knocked down by quite a lot of force, and now even doing tasks like those shown here. Just imagine 2050 if the progress continues at this rate. I am both excited and really scared.
- hatthew 1y agoAlmost all progress at doing tasks reliably has been made with 2 massive caveats: 1. Force. Walking, running, fighting, doing backflips, etc. all allow for large amounts of force, without a lot of dynamic precision required. Many common tasks require precise and dynamic force. E.g. for washing a window, pushing too hard breaks the glass while pushing too softly will leave streaks. 2. Environment interaction. Most reliable humanoid robots do minimal environment interaction beyond self-balancing. They walk/run/jump in environments that are largely open, with usually convex blocky obstacles. The real world has lots of tasks that require processing beyond low-resolution maps of solid/open space. E.g. I'd want to see a robot that can walk through a forest: jumping/stepping over thin branches that are hard to see, ducking under fallen logs, pushing though bendy branches without breaking them, avoiding ground that is muddy, and seeing through the current obstacle to determine if the obstacle beyond is traversable. Just to reiterate, I don't see fast progress being made on doing these tasks reliably. It's easy to show 1/N success rate, and much much harder to show ~N/N success rate on these dynamic tasks.
- poly2it 1y agoGreat post! Now somebody with the connections just needs to make it happen. For event five, slippery when wet, you should definitely include drying your hands on a towel, as it serves an important hygienic function.
- ubj 1y agoPlatinum Medal: Complete a gold medal task in the presence of multiple children between the ages of 3 and 8. The children must remain safe throughout (and after) the entire process.
- numpad0 1y agoCan I say something: could you all please stop inserting contaminated tool back into a jar of food? You use a clean tool to take out the amount you eat, and that's it. You can put it back if the tool had touched nothing but: air, the food in the jar, and your hand at the operating end. Otherwise, that butter knife stays on your dish for the rest of the meal. The exception would be if you cleaned the tool, like bare minimum by wiping with a brand new piece of tissue paper(but that's kind of wasteful). Is that an outrageous ask? I know it's probably not a huge deal, like free water and such, and my techniques are that of total amateur being never professionally involved in medical and/or bio science fields, but just, how can you stand possible breadcrumbs IN THE JAR!?
- benjiro 1y ago> but just, how can you stand possible breadcrumbs IN THE JAR!? Bacteria can do a number on people... Kept telling the wife to not cross contaminate jars, and other products. But you know, she knows better... Wife was eating some fish in can, puts spoon back (with lovely saliva bacteria), puts in fridge "i wil eat tomorrow", 2 days later she eats the leftover. She enjoyed a hour+ of "fun" muscles contracting stomach cramps to the point it was almost hospital time. Learned her lesson, well, ... for fish. Its always like "but i do not like to keep using fresh utensils". I am always: "the dishwasher does not care if it 20 or 40 utensils". Its the same amount of wash and cost. So stop trying to recycle utensils!
- numpad0 1y agoMy faith in humanity was lost and now I need an emergency memetic eyewash. Even I as a human dishwasher have no issues doing extra freshly used spoons. but that refrigerated two days old spoon... that is going to have dried stuffs stuck on the surface, and it's going to need to be soaked before washing. And also mildly bio-hazardous? That's horrible. Just why...
- YeGoblynQueenne 1y agoIf the fish was left in the fridge then it's not what caused the stomach cramps. Food generally keeps for more than two days in the fridge. It's pretty much what fridges are for.
- 5- 1y ago> If you sit down and write out what you might want a robot to do for you: a lot of tasks end up being kind of wet. yes, but i thought this wasn't that kind of article.
- exasperaited 1y agoSuper super platinum: a gently-padded humanoid robot that can successfully put on a silver body suit.
- satonakamoto 1y agoA wonderful article, truly insightful. But it feels more like a series of challenges than the Olympics. This is the Humanoid Robot Olympics held in China earlier this year. https://www.youtube.com/watch?v=5Y-tElcmJVE https://www.youtube.com/watch?v=5Y-tElcmJVE Instead of focusing on individual technical, it requires completing comprehensive movement tasks. For example, long-distance running tests battery managment and durable of joint, while boxing tests flexibility and coordination. Many participating teams or manufacturers, after failing, return to tackle the specific issues they encountered. This process drives the progress of the entire industry. In my view, this is worthy of being called an Olympic Game—in both form and spirit.
- k__ 1y agoIs someone building synthetic skin? Like, just a piece of silicone with a few sensors, pressure, heat, etc. that you can "glove" around a robot.
- Symmetry 1y agoIt's a topic of research but the need for durability combined with the amount of sensing we want makes it a hard problem.
- deleted 1y ago[deleted]