17 ms·
Extreme parkour with legged robots
- ouraf 3y agoone part i didn't fully get is if, after the training, the machine can calculate and implemente the same maneuvers while "offline" (not connected with any other device to assist with the navigation) and what would be the computing hardware requirements for smooth navigation without external help or update? (for example, guiding it with a joystick or GPS waypoint on rugged terrain and having it automatically adjust to each unforeseen obstacle in the way)
- 082349872349872 3y agothere ought to be plenty of money in this for delivering explosive payloads... (Edit: although come to think of it, in the animal world, jump height is much more constrained by 9,81 m/s2 than by body size)
- antognini 3y agoPhysics constrains jump height to be about a few meters regardless of size. The basic problem is that the total energy required scales with mass, but the total energy available also scales with mass. The two end up cancelling out and you get a maximum jump height that is independent of size. What really matters is the energy density of your storage medium, which is the same to an order of magnitude for any system based on chemical energy. So both a flea and an elephant (along with battery-powered robots) can jump about a meter or two in height.
- Torkel 3y agoWith special designs you can build up energy over time and then release it all at once, to reach sporadic super jumps. Example: https://www.youtube.com/watch?v=mvHXwTa5-DA https://www.youtube.com/watch?v=mvHXwTa5-DA (But for all but extreme designs I fully agree with you)
- sbierwagen 3y agoArtillery exists. Delivering explosive payloads is a solved problem. Detecting humans is a solved problem. The hard part is reliably distinguishing enemy solders from friendly soldiers or civilians.
- beachy 3y agoIt's possible to know where the humans are but be unable to destroy them with commonly available artillery payloads due to them being well dug in with overhead protection. These could swarm in the back door.
- PoignardAzur 3y agoI don't know, loitering munitions are pretty much bringing a renaissance to the field of blowing people up.
- sbierwagen 3y agoDrones are a solution to the targeting problem: the operator watches through a live video link, identifies the target, then deploys weapons. (Drops grenade/launches Hellfire/navigates into the enemy) This is why, despite being technically feasible, we haven't seen drone swarms yet. You can easily build 1000 drones, but you can't have 1000 channels of live video over radio back to 1000 drone operators.
- jiggawatts 3y agoArtillery is surprisingly inefficient. It takes about 15 rounds of 155mm artillery per battlefield casualty. Each shot costs about $5k. Robots like this might cost the same as a single artillery shot but have the ability to get multiple kills each. It’s a cold arithmetic, but I guarantee you someone at DARPA is thinking and planning the future of warfare in these terms.
- beachy 3y agoI thought you must be wrong - $5K for a dumb arty round? But some googling indicates this is about right, and prices have even gone up since the Russian invasion.
- unblough 3y agoOnly thing missing in that video is some rando walking by on campus in the background yelling “when they come, I hope they come for you first!”
- nightpool 3y agoAs a reminder: CMU and its robotics program is intimately associated with the US Department of Defense, and although there are certainly civilian applications for this technology, one of the most likely short-term applications is improving legged drones for battlefield deployment (often, it should be said, in humanitarian roles, including search and rescue, or support roles, such as ammo, food, and water resupply, but deployment of armed legged drones is also an active area of research). Here's a military news article covering the same umbrella grant program this research was funded under: https://www.c4isrnet.com/2022/06/23/darpa-adding-common-sense-to-battlefield-robots/ https://www.c4isrnet.com/2022/06/23/darpa-adding-common-sens...
- fuelfive 3y agoGlad to see our universities making progress in this important area of national security.
- croes 3y agoHow does this improve national security?
- gumby 3y agoThey are vehicles that can navigate terrain a wheeled vehicle cannot. So they could do anything from operate as pack "animals" for infiltration teams to be armed drones that creep along the ground, harder to detect than a flying one.
- croes 3y agoOnce again, how does this improve national security? That's offense not defense. It worsens the national security of your opponent but doesn't strengthen your own.
- XorNot 3y agoWhat do you think defense actually is in the current technological era? You don't defend territory with static defense's or force fields or walls.
- d--b 3y agoThis doesn't look like an extreme environment at all, yet the robot still has his legs sliding quite a bit on various obstacles, which surprised me since it can do complex stuff. They only show slow-motion on perfect moves, which is a bit disingenuous. I thought SOTA was better than this honestly.
- beachy 3y agoI thought the opposite. Watching it scrabble to get its hind legs up on the ledge - exactly as a dog would do, with the benefit of millions of years of evolution behind it - was very impressive to me.
- mytailorisrich 3y agoThis gives me Black Mirror-fueled nightmares... not least because those videos suggest that a Black Mirror scenario may not be that far off and what probably saves us for now is battery capacity.
- stavros 3y agoWhat's the Black Mirror scenario?
- hanniabu 3y agoGeneral dystopian outcomes. After world powers harness robotic armies, revolts will no longer be possible and citizens will be permanently enslaved and must succumb to whatever oppressions are put on them.
- stavros 3y agoWhy is walking the salient technology that will enable this? Why not wheels, or treads, or flying?
- mytailorisrich 3y agoPsychologically because it's life-like (uncanny valley). Practically, because legs are superior to wheels when it comes to pursue a human target. It may be superior to flying as well indoor, to open doors, etc. And then there is the energy expenditure and additional mass constraint when airborne. Research on legged robots is not just for 'fun', it's because they are better at going through obstacles and rough terrain than wheeled vehicles.
- pixl97 3y agoPart of it is because we already have flying/wheeling drones. Walking is just the last step of all terrain capabilities. Flying can get you almost anywhere, but not quite. It is both very loud and has issues in enclosed spaces. Wheels/tracks are very efficient at moving mass, but have issues crossing some materials such as exploded building internals like you would find in a war zone. Walking/climbing is the last bit to get you inside enclosed 'human' spaces such as buildings and tunnels. The war occurring now in Gaza would be an example area of what future battle would look like.... A target building it hit by a bomb and mostly destroyed. Smaller automated drones quickly fly over the area and assess visible targets and threats for follow up artillery. Multiple mobilized traced/wheeled heavily armored drones would address any residual ground level threats. Upon reaching the target building the larger drone vehicles would release multiple smaller 'walking/crawling' style drones to ascend the rubble, either addressing any remaining targets themselves, or reporting back to a command station for some other group to make a decision on what to do next. The final goal of these walking units is to probe deep into the rubble to ascertain if any tunnels survived the initial bombing, and if so to begin mapping any threats in said tunnels.
- o0banky0o 3y agoIt walks into your room, discovers you, and moves in to make the kill, but for some reason the gun trained at an odd angle. You see it take two shots that veer across to the corner of the room before it hilariously enters some self-correcting routine, seems to rediscover you, and finally blows your head off. You get a chuckle before you die
- p1esk 3y agoYou better have your own robots protecting you.
- qup 3y agoOr become undetectable.
- PoignardAzur 3y agoThe only protection from a bad guy with a robot is a good guy with a time machine and a shotgun.
- p1esk 3y agoWait, who was the bad guy with a robot? I thought there were only bad robots.
- throwaway67743 3y agoWar of the Worlds (recent remake) is finally here!
- philjohn 3y agoThat was my first thought on seeing this - it's eerily similar to the form factor of the Bob's.
- hammock 3y agoIs it just me or was I unimpressed that it can't long jump further than 2x its body length?
- 93po 3y agoHere's a Boston Dynamics one that is less than a foot long, 11 pounds, and can jump 30 feet (easily from ground to roof of one story building) https://www.youtube.com/watch?v=6b4ZZQkcNEo https://www.youtube.com/watch?v=6b4ZZQkcNEo
- namero999 3y agoWhat about the handstand? Was it programmed to do so or was it an artifact of the neural net? I wonder if this relates to the fact that some dogs do in fact perform "handstand" (pawstand?) in seemingly random situations...
- deleted 3y ago[deleted]
- riku_iki 3y agoI am always wondering, why they build such complicated biology inspired robots, while they can build something with wheels and simple levels which allows robot to jump over obstacles and turn upside down if it landed on the back.
- next_xibalba 3y agoMy limited understanding is that wheels really don’t work for all terrain scenarios. For example, craggy mountains in Afghanistan. (Much of this research is funded directly and indirectly by the US DoD)
- mcbutterbunz 3y agoIts my assumption that biology inspired robots would adapt better to a biological world. It might also be easier to solve problems by copying those who have already figured out a solution.
- Mistletoe 3y agoNature can’t really make a nice wheel and free wheeling hub though. Blood vessels would get ripped off, cartilage etc.
- lusus_naturae 3y agoNature does it own version of a "wheel", mainly in flagella: https://en.m.wikipedia.org/wiki/Rotating_locomotion_in_living_systems https://en.m.wikipedia.org/wiki/Rotating_locomotion_in_livin...
- Mistletoe 3y agoThat’s very true but it doesn’t seem to scale up well from that. :)
- bagels 3y agoGives us something tangible to compare.
- noelwelsh 3y agoI'm sure this is impressive for an autonomous robot. However, as a fan of real parkour its kinda annoying to see some modest jumps and walking on a slope labelled "extreme parkour". What the robot demonstrates I'd expect any healthy 10 year old to be able to equal.
- pb060 3y agoGiven the title, the video was indeed a bit disappointing but I imagine that “extreme” is ironic or at least relative.
- flemhans 3y agoI imagined a reference to The Office
- foderking 3y agoits actually hard for robots to compete with 10 year olds
- johnthesecure 3y agoTwo years ago, the Boston Dynamics Atlas could do mid-air somersaults - that's better than most 10-year-olds. https://www.youtube.com/watch?v=tF4DML7FIWk https://www.youtube.com/watch?v=tF4DML7FIWk
- kaba0 3y agoBut worse at running then any able-bodied 10-years-old. Comparatively, a somersault is quite easy/well-controlled movement.
- noelwelsh 3y agoFor sure, but the paper is disingenuous in their description of the challenge they are supposedly tackling compared to what they actual achieve. Here's an excerpt from the introduction of the paper: "Parkour is a popular athletic sport that involves humans traversing obstacles in a highly dynamic manner like running on walls and ramps, long coordinated jumps, and high jumps across obstacles. This involves remarkable eye-muscle coordination since missing a step can be fatal. Further, because of the large torques exerted, human muscles tend to operate at the limits of their ability and limbs must be positioned in such a way as to maximize mechanical advantage. Hence, margins for error are razor thin, and to execute a successful maneuver, the athlete needs to make all the right moves. Understandably, this is a much more challenging task than walking or running and requires years of practice to master. Replicating this ability in robotics poses a massive software as well as hardware challenge as the robot would need to operate at the limits of hardware for extreme parkour." Here's an example of real parkour: https://www.youtube.com/watch?v=5lp1oS0vXg0 https://www.youtube.com/watch?v=5lp1oS0vXg0 They aren't doing parkour, let alone "extreme" parkour.
- smusamashah 3y agoShared this few days ago but didn't get any traction then https://news.ycombinator.com/item?id=37690384 https://news.ycombinator.com/item?id=37690384
- chanind 3y agoThe robot needs a speaker so it can yell "Parkour!" during each jump
- beefman 3y agoRobot they're using is a Unitree A1: https://www.unitree.com/en/a1 https://www.unitree.com/en/a1
- GraceCat123 3y agoReally useful information! Thanks!
- folli 3y agoWhat's the approx. price tag for one of those? (I'm not going to buy one, just curious)
- resolutebat 3y ago$13,900 https://unitreerobotics.net/ https://unitreerobotics.net/
- rasz 3y agoLove ordering stuff from a website with no company information, no address, empty Support page. Totally not a scam.
- alcover 3y ago"The Hound half rose in its kennel and looked at him with green-blue neon light flickering in its suddenly activated eyebulbs. It growled again."
- gpderetta 3y ago"THEY sent A SLAMHOUND on Turner's trail in New Delhi, slotted it to his pheromones and the color of his hair. It caught up with him on a street called Chandni Chauk and came scrambling for his rented BMW through a forest of bare brown legs and pedicab tires. Its core was a kilogram of recrystallized hexogene and flaked TNT. He didn't see it coming. The last he saw of India was the pink stucco facade of a place called the Khush-Oil Hotel."
- the__alchemist 3y ago- "Ng Security Industries Semi-Autonomous Guard Unit #A-367 lives in a pleasant black-and-white Metaverse where porterhouse steaks grow on trees, dangling at head level from low branches, and blood-drenched Frisbees fly through the crisp, cool air for no reason at all, until you catch them."
- Animats 3y ago"A single neural net policy operating directly from a camera image, trained in simulation with large scale RL, can overcome imprecise sensing and actuation to output highly precise control behavior end-to-end." So that's what it takes. That's so much simpler than the way Boston Dynamics does it, working out all the dynamics in simulation first. It's amazing to see this done from vision to actuators in one net. It's now much less of a mystery how animals managed to evolve the ability to do that. (I had a go at this problem years ago. I was trying to work out the dynamics from first principles, knowing it would be somewhat off. Then use some self-tuning (a predecessor to machine learning) to fine-tune the thing. Got as far as running up and downhill in 2D in the early 1990s. About one hour of compute for one second of motion.)
- ShamelessC 3y agoDoesn’t the use of RL here imply some level of simulation?
- gwern 3y agoYes, but the key is the RL simulation can be simple & unrealistic since it's used to train offline, not to plan & control live: a control systems approach like the Boston Dynamics requires excellent physics modeling because that's the only way it can do planning or adjust for errors while controlling the actual robot in real time, while the NN is trained to cover many possible physics during development, and at runtime in an actual robot, it just shrugs and adapts to whatever crazy scenario it finds itself in.
- teraflop 3y agoOf course, the details of how to actually implement something like this are way more complex than "just throw everything into a big neural net with images as the inputs and actuators as the output". You need to provide the right kind of guidance in order to learn a usable policy in any reasonable amount of time. A very recent development (which this work builds on) is the idea of "online adaptation". It essentially involves doing the training in two stages: 1. You add a variety of dynamically-varying environmental effects to your simulator, by randomly altering parameters such as ground friction, payload weight distribution, motor effectiveness, and so on. You give the motion controller perfect knowledge of these parameters at all times, and let it learn how to move in response to them. 2. Then, you remove the oracle that tells the controller about the current environmental parameters, and replace it with another neural network that is trained to estimate (a latent representation of) those parameters, based on a very short window of data about the robot's own motor commands and the actual motion that resulted from it. All of this can be done in simulation, many times faster than real-time. But when you transfer the system to a real robot, it adapts to its environment using the estimated parameters, without any of the networks needing to be re-trained. This ends up making it pretty robust to difficult terrain and perturbations. It also has the benefit of papering over subtle differences that arise between the simulated and real-world dynamics. This paper adds a lot of additional refinements to the same basic idea. In the first stage, the system is given perfect knowledge of its surrounding terrain and the locations of some preselected waypoints, and learns to follow them. The second stage replaces those inputs with estimates derived from an RGB+depth camera.
- bravura 3y agoI was staying in Whitechapel, London recently. There are many religious folks there who are horrified by dogs, don't want dogs to touch them, and shun dogs and make a huge berth for them on the street and inside, like in buses. I was on dog walk duty and always held the lead close, but I knew they were worried I wasn't so conscientious and would let my dog friend run amuk on their garments. So I was trying to imagine an example of a creature that would horrify me similarly. Like perhaps if people had giant BEETLES as pets, and thought it was all grand, and I was like "what the FUCK are you on, my friend?", and I was pretty happy with that. But I think a closer cut to how disconnected this is from what seems appropriate, and normal, and reassuring, would be these robots. Clearly, some people don't really understand how horrifying this is, particularly with elevator music as the background. And if you want to know how upsetting the elevator music is as a background to set tone, I strongly urge you NOT NOT NOT to google "crab club" on youtube.
- notyourwork 3y agoDid anyone else find the stair handstand to look like cheating? Extremely neat to see this stuff.
- scrpl 3y agoThose movements are extremely animal-like, to the point of being unsettling (despite that I want a robot like that for myself)
- nmca 3y agoWhilst a lovely achievement, this title definitely overstates current robot abilities. Here is some human parkour for contrast: https://m.youtube.com/watch?v=QHqAVaQqQWQ&t=106s https://m.youtube.com/watch?v=QHqAVaQqQWQ&t=106s
- lukevp 3y agoThe difference being each human has to spend thousands of hours learning that level of control over their bodies, and work hard to maintain that level of physical fitness. And if they fall off the roof, millions of dollars of potential earnings and GDP die with them. Whereas these robots are only $70k, and once one of them can do this, they all can do it. Just like with Chess and Go. It’s not impressive at first, then a couple years later, it’s better than humans could ever be, and it can be cheaply replicated ad infinitum.
- msikora 3y ago^^^This
- r3trohack3r 3y ago$70K? The one pictured appears to be $14k https://m.unitree.com/en/a1/ https://m.unitree.com/en/a1/
- quickthrower2 3y agoThe most impressive thing about the video is not the skill but the control of their fear.
- ChatGTP 3y agoI think without fear, challenges to over come, ways to improve, there would be little point to living. Experience would be so bland? The feeling over fear when you want to ask someone out, then you do it and it works out, that is the spice of experience. Like wise when it doesn’t work out, it can suck but it’s also part of the spice.
- 3y ago
- superb_dev 3y agoWhy doesn't it use existing momentum to get over that gap or up on the ledge? It slows down before it makes the jump
- 90-00-09 3y agoLooks like it slows down just a tiny bit (needs time to compute?) before making some of the jumps in this video. So not completely losing momentum. Wonder if this will look more fluid once it's able to reach a higher speed.
- wly_cdgr 3y agoHorrifying
- ChatGTP 3y agoCan it unpack the dishwasher ?
- lukevp 3y agoIt’s pretty incredible how animalistic its behaviors are becoming (hesitating for a moment at the edge of a lip, coiling it’s back legs for increased actuation, and when it almost misses the high jump and moves its back leg really fast multiple times to give itself a tiny boost each time until it recovers). Is that because it’s trained on animal behaviors? Or is this emergent behavior and animals just also do it more or less optimally already?
- gwern 3y agoIt trains from scratch (no imitation-learning or hand-written policies to bootstrap off of), so it's all emergent: https://extreme-parkour.github.io/resources/parkour.pdf#page=6 https://extreme-parkour.github.io/resources/parkour.pdf#page...
- loandbehold 3y agoit suggests that animals learn using similar neural nets.
- cnity 3y agoOr it simply suggests that this is advantageous behaviour and that different learning methodologies would result in the same behaviour.
- deleted 3y ago[deleted]
- imchillyb 3y agoQuiet actuators, a proboscis, and a bit of ricin, and the world waits.
- TaylorAlexander 3y ago[dupe] https://news.ycombinator.com/item?id=37823440 https://news.ycombinator.com/item?id=37823440
- modeless 3y agoOh this is automatically reposted from the second chance pool, I originally posted it yesterday before that other story. It's a shame it isn't getting more attention. IMO it deserves to be at the top of HN for hours. This research represents how the methods in robotics are changing quickly. I think people underestimate how capable robots will be in the near future. Also the comments on that other story are just sad. I understand war is in the news but I hope it's not all we can think about! Also I'm surprised by all the people who think this is not impressive because "a 10 year old could do this" or whatever, that's actually ridiculous. I guess HN just isn't the right community to appreciate real hard technology like this anymore. This makes me sad, especially because I don't know of anywhere else that's better.
- dang 3y agoIt was a collision between the SCP (https://news.ycombinator.com/item?id=26998308 https://news.ycombinator.com/item?id=26998308) and a regular submission, but as this one is on the front page now, we'll merge the comments hither.
- 55555 3y agoWhere did they buy this robot dog from? It looks like the Boston Dynamics one.
- modeless 3y agoUnitree. I think it's equivalent to the Go1 which is ~$5k shipped (although I don't think that version gives you the SDK access needed to do these kinds of tricks). It's much smaller than Boston Dynamics' Spot.
- au8er 3y agoIt's the predecessor of Go1. If you buy the Pro/Edu version of Go1 then you get direct low level SDK control, though there are reverse engineering efforts that can do the same on the cheap ~5k version as well
- yieldcrv 3y agoOut of curiosity, is everyone of university age good at clickbait now? Like the whole point of saying “extreme” parkour is to boost engagement from pedantic analytic people like us talking about the hyperbolic title choice
- isoprophlex 3y agoThe mainstream, state broadcaster in my country publishes youtube videos of news topics with SENSATIONAL TITLES and thumbnails with close-up REACTION FACES. On their home page, along with the written articles. I hate what the attention economy has become. Old man yells at emoji...
- 90-00-09 3y agoIt is a bit disappointing. This video did not show anything more "extreme" than various Boston Dynamics videos from years ago. And to be even more pedantic, this is hardly parkour at all. Jumping over a gap and climbing on a box has little resemblance of what we came to expect when we hear this term.
- la64710 3y agoWhen can I have one of these to patrol around my house ?
- kaba0 3y agoToday? But the bigger version is around the price of a higher-end car.
- pjmlp 3y agoOne step closer to ED-209.
- hackernewds 3y agoI'm curious why the leg joints hinge towards the posterior, instead of away from the posterior like they do in dogs. Seems we would want to mimic the nature model as optimal, however does this imply the robot model performs better, in a way that somehow all quadrupeds didn't evolve to? Or is it some nuance of metals vs carbon-based joints?
- singhblom 3y agoDogs have more joints, one for the knee where the femur meets the tibia and one for the ankle where the tibia meets the tarsal bones. If you just have one single joint in the hind legs you probably want to be able to push off as easily as possible, so the lower one makes more sense.
- emsign 3y agonice. now send them to Ukraine so they can clear out Russian trenches
- danmaz74 3y agoI scanned the paper and, if I got it right, during training the robot gets "external" information about its world position and speed compared to pre-set waypoints, something that an animal or person wouldn't have access to. What I didn't understand for sure is if this information is also available to the robot while performing the "parkour": I'm pretty sure that it perceives the obstacles only through its depth camera, but how does it determine where it is and where it should go next? Is this still done through waypoints and global position knowledge? A joystick is mentioned for control; is that being used to set waypoints? Or feed the robot a relative direction and speed?
- waldarbeiter 3y agoWhen the policy is deployed in the real world only the depth camera is used no waypoints etc.. Scan dots and target heading is used in the first Phase of the training to pretrain a policy in simulation. In Phase 2 a policy is trained end-to-end using the pretrained actor network: "First, exteroceptive information is only available in the form of depth images from a front-facing camera instead of scandots. Second, there is no expert to specify waypoints and target directions, these must be inferred from the visible terrain geometry." For policy training in Phase 2 DAgger which is based on Behavior Cloning is used (with the policy from Phase 1 as the expert), they also use some tricks to make sure no actions that are too different from the expert actions are executed during training. In Phase 2 the network learns to extract environment information instead of from the scan dots from the depth camera. Also in Phase 2 they use the pretrained actor network from Phase 1 but the depth embedding must be learned from scratch. This is how I understand it.
- danmaz74 3y agoThank you for your comment. What I don't understand is this: when the robot is in a new environment, how does it know where it's supposed to go? My understanding is that the training teaches the robot how to get to a position, but I didn't see anything about how to choose where to go (in "old AI" parlance, what could have been defined as planning).
- novalis78 3y agoI need that Go1 support asap. Time for my little Unitree friend to evolve and keep up with its GPT4 brain…
- FrustratedMonky 3y agoThis all seemed a lot cooler before that Black Mirror episode, now every time I see a cool video like this, I get scared.
- _ZeD_ 3y agothat's basically a poodle!
- jeffrallen 3y agoBad boys bad boys, watcha gonna do when they come for you?
- m3kw9 3y agoPretty unimpressive given recent advances in AI
- 93po 3y agoHow would you recommend cutting edge robotics research to approach this instead?
- michael1999 3y agoReal feedback via proprioception and hand-eye coordination are so much more powerful that closed control movement planning.