6 ms·
Newton: physics simulation engine built upon NVIDIA Warp
- bogwog 1y agoIs this related to the Newton Dynamics physics engine? https://newtondynamics.com/ https://newtondynamics.com/
- flohofwoe 1y agoHeh, that was my first thought too and it doesn't look like it's related.? The nvidia dudes could have done some minimal amount of googling to pick a name that causes less confusion.
- forrestthewoods 1y agoNo > Newton extends and generalizes Warp's (deprecated) warp.sim module, and integrates MuJoCo Warp as its primary backend. It’s MuJoco GPU Edition. Nothing new or improved.
- erwincoumans 1y agoWell, MuJoCo initially used JAX for GPU (MJX) and MuJoCo Warp replaces MJX with better performance.
- user____name 1y agoMy first thoughts as well, it's a well established engine.
- swiftcoder 1y agoYeah, this is going to be confusing for sure
- materialpoint 1y agoNo, but the choice of name exudes a certain arrogance that aligns with the authors of MuJoCo. It's a very capable and robust engine, but the authors have been very condescending of other technologies and confusing the terms gaming with game. It certainly won't replace PhysX as they are designed for comparatively small scale simulations, however with instancing. For instance, MuJoCo doesn't have real broadphase that scales with large environments, but sticks with the old tried-and-true SAP. Neither does it have separate friction coefficients for slip, tying it into robotics, except maybe for the VDB solver.
- huflungdung 1y ago[dead]
- 29athrowaway 1y agoYears ago there was PhysX... how does this compare?
- erwincoumans 1y agoThis will eventually replace PhysX, some of its developers are working on Newton Physics. Newton Physics has multiple solvers, including MuJoCo-Warp and is easier to customize and extend.
- zokier 1y agoTheir FAQ explicitly says > Will Newton replace PhysX? > No, the two engines serve different primary goals https://newton-physics.github.io/newton/faq.html#will-newton-replace-physx https://newton-physics.github.io/newton/faq.html#will-newton...
- cyber_kinetist 1y agoProbably the parent commenter has much more insider info than all of us since he's currently at NVIDIA... From what I understand, PhysX has been built primarily as a physics engine middleware for games. So when folks at NVIDIA tried to extend this engine to robotics (for IsaacSim/IsaacLab) it seems they've faced lots of challenges (mainly with subpar multi-env performance and inaccurate solver, but also lots of technical debt over the years). So changing the internal engine to a more robotics-oriented one (Mujoco-warp) doesn't seem far-fetched. Nowadays for game engine development there are much better middleware CPU-based physics engines available (mainly Jolt Physics) - and GPU physics in games aren't that popular anymore due to pragmatic reasons (the GPU -> CPU roundtrip defeats the whole purpose of better performance)
- erwincoumans 1y agoI was assuming the context of robot learning (IsaacLab), where Newton Physics will eventually replace PhysX. Newton Physics doesn't target games or other areas.
- jackling 1y agoI think this is a step in the right direction, but I really dislike the Pythonification of everything. After using IsaacSim/IsaacLab for work, I'm convinced that Python is not the right tool for the job. Developers inevitably write slow, error filled code when dealing with Python and working with the type annotations can be a pain. Happy there's something to replace PhysX for robotics, and I do really like MuJoCo's API, but really wish we could get some good C/C++ APIs. Apart from the language, NVIDIA doesn't seem to be great when dealing with software. IsaacSim and IsaacLab have so many bugs, are incredibly slow, and hard to debug. We spend so many hours on my team findings bugs for IsaacSim, it's just a pain. On version 5.0 and still feels like beta software. Also IsaacSim's relience on USD to hold the scene structure and update prims makes it so hard to program for. USD isn't really performant when trying to generate a large amount of scenes. And the USD interface stops working completly when simulation starts on IsaacLab. I hope Newton goes a different route, and has less of a reliacne on USD. IMO USD should just be used as an interchange format, rather than how you actually represent the scene and properties internally. I much prefer that approach, which Unreal Engine seems to support. Lastly, my god the names in this field are terrible. USD (Googling becomes a pain sometimes), Newton (Already another engine), Warp (literally the name of the architecture and a way to write Python GPU kernels, wtf).
- erwincoumans 1y agoAgreed on most, and naming is terrible. Note that at run-time Python is out-of-the loop, since Newton Physics records a CUDA graph, and executes it, so performance is not impacted (aside from startup JIT time for modified kernels). I'd prefer C/C++ as well, and although you can call Warp-compiled kernels from C++ (without Python, see my https://github.com/erwincoumans/warp_cpp https://github.com/erwincoumans/warp_cpp project), it would be better to have native C/C++ support without requiring a Python interpreter. It just happens that almost all Deep Learning/RL for robotics uses Python.
- jackling 1y agoHopefully I didn't come across as too negative, My entire team and I are really excited for Newton. Hope to get some time this week to try things out!
- whatever1 1y agoHow do they parallelize the sequential actions ?
- erwincoumans 1y agoThe primary use case of Newton Physics is reinforcement learning, with 1000s of similar environments. Even if each environment would have sequential actions, you run many envs in parallel.
- shihab 1y agoOne feedback from someone interested in using this about the examples: I have looked at several and they seem too high level to get a sense of the actual API (i.e. the expected benefit of using this library vs the development complexity of using it). For example, the cloth bending simulation is almost entirely: at __init__, call a function to add a cloth mesh to model builder obj, pass built model to initializer of a solver class; and at each timestep: call a collide model function, then call another function called solver.step. That's really it.
- amelius 1y agoWe're becoming too reliant on libraries made by our hardware vendors (vendor, singular, actually, to make it worse).
- th0ma5 1y agoI wish I could upvote this a bunch. It will only happen if people being to reflect on their gains at the hands of a specific vendor makes them beholden to that vendor. I know it is an idealism but I often think of statements like "source code or didn't happen" or wonder why people entrust such important things to one company an executive order away from being illegal.
- fooblaster 1y agoI think the bigger deal for founders is that Nvidia can decide at a whim to deny you supply or, better yet give 100 billion dollars to your competitor.
- lwhi 1y agoI guess it's a problem of monopoly, what could or should be done to solve it?
- TinkersW 1y agoWhat type of solver is mujoco?
- gattis 1y agoI've been playing around with this for a few weeks. Newton is a pretty thin wrapper around mujoco-warp, which is trying to port mujoco, originally a CPU sim, over to warp on the GPU. There is also mujoco-mjx for this purpose, but using jax instead of warp. I think mjx/jax has the edge on performance because there are mature RL libraries for jax (brax) and big advantages to using Jax for everything, especially with its ability to "vmap" over each layer of abstraction. But I can see why nvidia wants to move away from IsaacLab using physX+pytorch because physX was made for games and interfacing with it through IsaacSim is a bit of a kludge. And apparently mjx isn't so accurate with collisions because of the way it has to be treated in jax. Pytorch RL works decently with newton/warp, at least they can share GPU buffers and you don't have to copy things back and forth to the CPU, however you can't optimize with cuda graphs past the newton/warp boundary because newton/warp have their own cuda graph capture scheme going on at the same time underneath. They already have a newton branch of IsaacLab on github but its pretty early for it. I just came across a dope project today that is a different wrapper around mujoco-warp that already mimics IsaacLab's api and you can run some robot environments on it. Clean code too, very promising: https://github.com/mujocolab/mjlab.git https://github.com/mujocolab/mjlab.git