3 ms·
The first game engine for robotics
- aktenlage 5mo agoWhat's the difference of a "robot game engine" to a simulator like <https://gazebosim.org https://gazebosim.org>
- xgulfie 5mo agonothing
- tartoran 5mo agoProbably the name is
- sentdex 5mo agoHey I work @ Lucky Robots. The differences are pretty big, but the simplest way to illustrate is to try to use gazebo, isaac...etc, and then try to build a whole physically interactive kitchen. First off, it's gonna take you 3 months to author that thing, if you don't ragequit along the way. Then, when you go to run, your "50 million steps per second" sim becomes 500 steps per second. The reason we have robots doing backflips and acrobatics instead of actually useful stuff like picking up your house is making the scenes and getting the data is tough. It requires sensors like cameras and rendering, vs purely proprioceptive-only envs with a flat ground plane and no other physics interactions. Right now, the industry is doing manual teleop to collect data because it's straight up easier than trying to build these sorts of things in simulators. This is why we're building Lucky.
- anonydsfsfs 5mo agoThis (and a global pandemic) are central to the plot of The Talos Principle, a 2014 puzzle game. Can't say much more without spoilers, but we'd better hope its other predictions for the future don't come true.
- bogwog 5mo agoIs robotics going to be the next industry to jump on the We'Re sO DaNgErOuS!1!1! marketing bandwagon?
- AndrewKemendo 5mo agoI was the Principal architect for ML-Agents at Unity for a while and this looks like it could be a more elegant version of what we were doing (cause it’s not a sidecar to an engine). I’m going to try it to see if I can make my Go-1 edu do some work around the house finally
- pj_mukh 5mo agoWondering if it would be better to abstract Vulcan away by Unreal, get even better photo-real quality. I guess the better question is how much does photo-realism quality matter for this kind of sim2real work? A lot I would wager.
- Flux159 5mo agoUnreal will definitely get better results out of the box, but it's also possible to do photorealism with significantly less overhead (particularly UE shader compilation overhead) - useful for single purpose platforms. If you don't need to support lots of specific editor or game features, it may be a valuable investment. UE is definitely used to obtain simulation data in other domains (this is coming from first hand experience in big tech), but usually through scripting UE handmade levels in python which also needed convoluted server systems at the time (hopefully this has gotten better now).
- a_t48 5mo agoVulkan is a rendering technology, Unreal is an engine, that can render using Vulkan under the hood. You can absolutely do this in Unreal. I implemented this at Cruise using UE4 for integration testing, and it worked great for inference (we weren't doing training on sim at that point, but I was pushing for it! There was a paper out in 2018 or so that showed mixing in a bit of simulated data had an outside positive impact on the outputted model). There are companies out there right now doing this with even more modern renderers. I can't comment on how much the rendering realism gap matters here. I think there's some people out there using a variant of lower quality rendering + some kind of diffusion to get "better" images without having to do detailed modeling/lighting for their sims (fuzzy memory, I don't have a source on this).
- pj_mukh 5mo agoIs there a good Unreal + Physics simulator OOB now that people can try? I guess Isaac Sim is king?
- dkersten 5mo agoHaven’t Unreal engine and Unity been used for robotics for over a decade? Hasn’t the Bullet physics engine been used for robotics for over a decade? I don’t understand this “first game engine for robotics” messaging. As an aside, this website crashes for me on safari on iOS.
- Legend2440 5mo agoTheir claim is actually: "the world's first game engine purpose-built for robotics." Idk if that's true or not, but it does exclude all the engines you mentioned.
- dkersten 5mo agoIs that true though? From Cherno’s videos it sounded like it was basically the hazel engine, repurposed. So unless he rewrote hazel, purposely for robotics, it’s still not actually the case?
- TheCherno 5mo agoWhile "rewrote hazel" might be a bit of a stretch, we did fundamentally rewrite a lot of the core to make it specifically suitable for robotics simulation, rather than human gamers.
- slopinthebag 5mo agoThis is the company that Yan Chernikov - aka TheCherno has cofounded. This engine is based on the engine (Hazel) he built as part of his game engine series on YouTube. If you're interested there is probably at least 100 hours of devlogs about this engine which is kind of cool. https://www.youtube.com/@TheCherno https://www.youtube.com/@TheCherno
- echoangle 5mo agoIs there a danger of overfitting if you train something on a physics sim? How do you prevent the model to exploit the differences to real life? Surely there are some numerical errors and other idealizations that result in some stuff being a good solution but not working in real life, or is the sim that accurate?
- m0llusk 5mo agoWhat about Karel the robot?
- LoganDark 5mo agoImpressive that they reach 10kHz with C#!
- greenpizza13 5mo agoWhat about IsaacSim [0]? [0] https://developer.nvidia.com/isaac https://developer.nvidia.com/isaac