3 ms·

MuJoCo – Advanced Physics Simulation

116 points by modinfo 6mo ago

4corners4sides 6mo ago
People have made cool racing education simulators with this too: https://github.com/FT-Autonomous/ft_grandprix https://github.com/FT-Autonomous/ft_grandprix.
cachius 6mo ago
This is what StuffMadeHere used in his latest video to simulate a mini-golf course! https://www.youtube.com/watch?v=2OfjZ3ORJfc&t=368s https://www.youtube.com/watch?v=2OfjZ3ORJfc&t=368s The physics engine I'm using is called MuJoCo. And if you're wondering why I didn't write my own physics engine, it's basically because I don't have 20 years.
- prathje 6mo ago
  That the calibration got the simulation so close to reality was quite impressive.
  - cachius 6mo ago
    Though he had to resort to manual calibration. I always find he has interesting problems for domain experts and would like to see him team up with one. Also with programmers for faster programs than self taught python.
    - Izikiel43 6mo ago
      The guy has a masters in compsci and mechanical engineering, he has done both python and c++ afaik for his projects
      - liuliu 6mo ago
        I think the parent comment is specifically addressing why the black box (or stochastic?) optimizer he used not working.
      - ghusbands 6mo ago
        In the video in question, he doesn't seem able to choose a good scoring function for the stochastic solver (even over multiple weeks), seemingly choosing a linear sum of distances (see 8:50) between simulation and reality. That's a mistake that not even an undergraduate should make. He needs some domain experts.
        Izikiel43 6mo ago
        I did comp sci, I sucked at this stuff, and I have a masters. Low level stuff? Os? Distributed systems? Multithreaded code? Now that’s more on my alley
- nestorD 6mo ago
  It's what put MuJoCo on my radar recently! But I was surprised to not see him do any kind of gradient descent to optimize his hyperparameters. MuJoCo has a JAX backend so it should be fairly straightforward.
  - petters 6mo ago
    He is much better at building hardware than he is writing software.
    - KeplerBoy 6mo ago
      He seems pretty damn good at both.
  - lpribis 6mo ago
    I'm pretty sure he has used gradient descent in previous videos to optimize systems, maybe this time it was just easier to hand tune rather than set up an optimization feedback harness around MuJoCo.
prathje 6mo ago
We are using MuJoCo to train a G1 humanoid robot right now. The best thing is that we do not need to fight with NVIDIA software and that it runs on macOS. PS: I just finished a first draft for agentic skills around working with MuJoCo in Python. Feel free to check them out here: https://github.com/prathje/agentic_mujoco_skills https://github.com/prathje/agentic_mujoco_skills
zokier 6mo ago
Mujoco is also key part of nvidias Newton physics system https://github.com/newton-physics/newton https://github.com/newton-physics/newton
xingyi_dev 6mo ago
[flagged]
sheepscreek 6mo ago
This makes me so happy and excited! Often my mind wanders into the unknown, imagining what would happy to X if it did this? Would it have friction, etc? I am looking forward to a way I can easily describe a scenario and have an LLM build a legitimate simulation for it. No more hypothetical talk! Next best thing to actual experimentation (can be a useful tool in convincing others to join you/support you in said real experiment - “see? I tested it in a simulation and it behaves exactly that way! We need to try this..”).
- ai_fry_ur_brain 6mo ago
  Build things yourself. Using LLMs doesnt help you understand anything, they will just give you an annoying case of dunning kruger. Using them will only make you retart-d
- v9v 6mo ago
  One thing to be mindful of is that you can get a simulation to behave in (almost) any way you want if you set the parameters right, so you should take care to understand the assumptions that you're baking into your sim before taking its results as gospel.
- gspr 6mo ago
  > what would happy to X if it did this? Would it have friction, etc? MuJoCo does not simulate at a scale where you can discover whether something has friction or not. Friction is a parameter to MuJoCo's simulations.
tlb 6mo ago
MuJoCo is great. I have it running in the browser for robotics simulation. See for example https://visibot.com/sheet/examples/humanoid_walking.v https://visibot.com/sheet/examples/humanoid_walking.v
Rebuff5007 6mo ago
Not sure why this is hitting the home page right now but people may also be interested in Mujoco Playground [1] which is the latest RL environment wrapper of mujoco, implementing both classic deepmind-control benchmarks, and some very new interesting ones! [1] https://playground.mujoco.org/ https://playground.mujoco.org/
- s0rce 6mo ago
  Probably because of the StuffMadeHere video
saran-t 6mo ago
While we have people's attention here, we're also working on an official browser-based interactive viewer that allows people to specify a URL to their own model and share a link. Very early experimental prototype at the moment, doesn't work super well on phones yet, but you can try it at https://saran-t.github.io/mujoco/?model=https://raw.githubusercontent.com/google-deepmind/mujoco/main/model/cards/cards.xml https://saran-t.github.io/mujoco/?model=https://raw.githubus... It'll be relocated to live.mujoco.org over the next few days.
fleahunter 6mo ago
[dead]
saucesaft 6mo ago
My two cents, I actually worked with getting MuJoCo's physics gradients to train quadruped locomotion policies. Really interesting field. https://github.com/saucesaft/differential_policies/ https://github.com/saucesaft/differential_policies/
whatever1 6mo ago
Wait what's new ? This is an established library

116 points by modinfo 6mo ago

People have made cool racing education simulators with this too: https://github.com/FT-Autonomous/ft_grandprix https://github.com/FT-Autonomous/ft_grandprix.

cachius 6mo ago

This is what StuffMadeHere used in his latest video to simulate a mini-golf course! https://www.youtube.com/watch?v=2OfjZ3ORJfc&t=368s https://www.youtube.com/watch?v=2OfjZ3ORJfc&t=368s The physics engine I'm using is called MuJoCo. And if you're wondering why I didn't write my own physics engine, it's basically because I don't have 20 years.

prathje 6mo ago
That the calibration got the simulation so close to reality was quite impressive.
- cachius 6mo ago
  Though he had to resort to manual calibration. I always find he has interesting problems for domain experts and would like to see him team up with one. Also with programmers for faster programs than self taught python.
  - Izikiel43 6mo ago
    The guy has a masters in compsci and mechanical engineering, he has done both python and c++ afaik for his projects
    - liuliu 6mo ago
      I think the parent comment is specifically addressing why the black box (or stochastic?) optimizer he used not working.
    - ghusbands 6mo ago
      In the video in question, he doesn't seem able to choose a good scoring function for the stochastic solver (even over multiple weeks), seemingly choosing a linear sum of distances (see 8:50) between simulation and reality. That's a mistake that not even an undergraduate should make. He needs some domain experts.
      - Izikiel43 6mo ago
        I did comp sci, I sucked at this stuff, and I have a masters. Low level stuff? Os? Distributed systems? Multithreaded code? Now that’s more on my alley
nestorD 6mo ago
It's what put MuJoCo on my radar recently! But I was surprised to not see him do any kind of gradient descent to optimize his hyperparameters. MuJoCo has a JAX backend so it should be fairly straightforward.
- petters 6mo ago
  He is much better at building hardware than he is writing software.
  - KeplerBoy 6mo ago
    He seems pretty damn good at both.
- lpribis 6mo ago
  I'm pretty sure he has used gradient descent in previous videos to optimize systems, maybe this time it was just easier to hand tune rather than set up an optimization feedback harness around MuJoCo.

prathje 6mo ago

We are using MuJoCo to train a G1 humanoid robot right now. The best thing is that we do not need to fight with NVIDIA software and that it runs on macOS. PS: I just finished a first draft for agentic skills around working with MuJoCo in Python. Feel free to check them out here: https://github.com/prathje/agentic_mujoco_skills https://github.com/prathje/agentic_mujoco_skills

zokier 6mo ago

Mujoco is also key part of nvidias Newton physics system https://github.com/newton-physics/newton https://github.com/newton-physics/newton

xingyi_dev 6mo ago

[flagged]

sheepscreek 6mo ago

This makes me so happy and excited! Often my mind wanders into the unknown, imagining what would happy to X if it did this? Would it have friction, etc? I am looking forward to a way I can easily describe a scenario and have an LLM build a legitimate simulation for it. No more hypothetical talk! Next best thing to actual experimentation (can be a useful tool in convincing others to join you/support you in said real experiment - “see? I tested it in a simulation and it behaves exactly that way! We need to try this..”).

ai_fry_ur_brain 6mo ago
Build things yourself. Using LLMs doesnt help you understand anything, they will just give you an annoying case of dunning kruger. Using them will only make you retart-d
v9v 6mo ago
One thing to be mindful of is that you can get a simulation to behave in (almost) any way you want if you set the parameters right, so you should take care to understand the assumptions that you're baking into your sim before taking its results as gospel.
gspr 6mo ago
> what would happy to X if it did this? Would it have friction, etc? MuJoCo does not simulate at a scale where you can discover whether something has friction or not. Friction is a parameter to MuJoCo's simulations.

tlb 6mo ago

MuJoCo is great. I have it running in the browser for robotics simulation. See for example https://visibot.com/sheet/examples/humanoid_walking.v https://visibot.com/sheet/examples/humanoid_walking.v

Rebuff5007 6mo ago

Not sure why this is hitting the home page right now but people may also be interested in Mujoco Playground [1] which is the latest RL environment wrapper of mujoco, implementing both classic deepmind-control benchmarks, and some very new interesting ones! [1] https://playground.mujoco.org/ https://playground.mujoco.org/

s0rce 6mo ago
Probably because of the StuffMadeHere video

saran-t 6mo ago

While we have people's attention here, we're also working on an official browser-based interactive viewer that allows people to specify a URL to their own model and share a link. Very early experimental prototype at the moment, doesn't work super well on phones yet, but you can try it at https://saran-t.github.io/mujoco/?model=https://raw.githubusercontent.com/google-deepmind/mujoco/main/model/cards/cards.xml https://saran-t.github.io/mujoco/?model=https://raw.githubus... It'll be relocated to live.mujoco.org over the next few days.

fleahunter 6mo ago

[dead]

saucesaft 6mo ago

My two cents, I actually worked with getting MuJoCo's physics gradients to train quadruped locomotion policies. Really interesting field. https://github.com/saucesaft/differential_policies/ https://github.com/saucesaft/differential_policies/

whatever1 6mo ago

Wait what's new ? This is an established library