4 ms·

I suspect that the actual generation and simulation/rendering takes several minutes for each step.

by ilaksh 2y ago

deleted 2y ago
[deleted]
psb217 2y ago
The simulation/rendering is actually pretty fast since it's all done by heavily optimized gpu-based physics and graphics engines. The "generative" part is that they have some LLM stuff that's finetuned for generating configurations/parameters for the physics engine conditioned on some text. Ie, the physics and graphics are classical clockworky simulations, with a generative frontend to make it easier (but less precise) to get a world up and running. The open source release currently provides the clockworky simulator stuff, with the generative frontend to be released some time in the future.
- ilaksh 2y ago
  That's what I meant about the LLM. If you are saying it's a real time simulation and rendering then I did not realize that.
- iamfirdaus1 2y ago
  man i love what u writing even as far of this post ur 2012 post about SVM, i am 2nd year CS student from indonesia. are you a phd on computer science?
  - psb217 2y ago
    Yeah, I finished a PhD in machine learning around 2016 and have been working professionally as a researcher since then, though I'm currently between jobs. It's a fun gig, but the "publish or perish" aspect of academic-ish research roles gets old fast.
stonet2000 2y ago
This is kind of correct. For the demos they show those will simulate quite slowly due to the soft body / fluid simulation. Their current code does not parallelize soft body/fluid simulation like they do with rigid bodies. Running their fluid sim code now with default settings (which are tuned for speed, not accuracy) gives maybe max 10-100x real time speed (their rigid body sim is magnitudes faster but definitely not faster than existing GPU simulators like Isaac or MJX). Rendering at that quality will also take forever but everyone in this area of research always runs the slow high quality ray tracing pipelines for the demo videos (which run in the 10-100 FPS range depending on settings). Another note is they claim realism but this is only visual realism, there doesn’t exist soft body sim solvers that transfer to the real world accurately enough yet for eg robotics. They are good enough for visual data generation which will be pretty cool. (full disclosure I work on ManiSkill/SAPIEN which runs on PhysX from NVIDIA, Genesis authors claim they are essentially faster than PhysX by 10-80x).

I suspect that the actual generation and simulation/rendering takes several minutes for each step.

by ilaksh 2y ago

I suspect that the actual generation and simulation/rendering takes several minutes for each step.

deleted 2y ago

[deleted]

psb217 2y ago

The simulation/rendering is actually pretty fast since it's all done by heavily optimized gpu-based physics and graphics engines. The "generative" part is that they have some LLM stuff that's finetuned for generating configurations/parameters for the physics engine conditioned on some text. Ie, the physics and graphics are classical clockworky simulations, with a generative frontend to make it easier (but less precise) to get a world up and running. The open source release currently provides the clockworky simulator stuff, with the generative frontend to be released some time in the future.

ilaksh 2y ago
That's what I meant about the LLM. If you are saying it's a real time simulation and rendering then I did not realize that.
iamfirdaus1 2y ago
man i love what u writing even as far of this post ur 2012 post about SVM, i am 2nd year CS student from indonesia. are you a phd on computer science?
- psb217 2y ago
  Yeah, I finished a PhD in machine learning around 2016 and have been working professionally as a researcher since then, though I'm currently between jobs. It's a fun gig, but the "publish or perish" aspect of academic-ish research roles gets old fast.

stonet2000 2y ago

This is kind of correct. For the demos they show those will simulate quite slowly due to the soft body / fluid simulation. Their current code does not parallelize soft body/fluid simulation like they do with rigid bodies. Running their fluid sim code now with default settings (which are tuned for speed, not accuracy) gives maybe max 10-100x real time speed (their rigid body sim is magnitudes faster but definitely not faster than existing GPU simulators like Isaac or MJX). Rendering at that quality will also take forever but everyone in this area of research always runs the slow high quality ray tracing pipelines for the demo videos (which run in the 10-100 FPS range depending on settings). Another note is they claim realism but this is only visual realism, there doesn’t exist soft body sim solvers that transfer to the real world accurately enough yet for eg robotics. They are good enough for visual data generation which will be pretty cool. (full disclosure I work on ManiSkill/SAPIEN which runs on PhysX from NVIDIA, Genesis authors claim they are essentially faster than PhysX by 10-80x).