4 ms·
This might be a vague question, but what kind of intuition or knowledge do you need to work with these kind of things, say if you want to make your own model? I
by Valk3_ 1y ago
This might be a vague question, but what kind of intuition or knowledge do you need to work with these kind of things, say if you want to make your own model? Is it just having experience with image generation and trying to incorporate relevant inputs that you would expect in a 3D world, like the control information you added for instance?
- ollin 1y agoI think https://diamond-wm.github.io https://diamond-wm.github.io is a reasonable place to start (they have public world-model training code, and people have successfully adapted their codebase to other games e.g. https://derewah.dev/projects/ai-mariokart https://derewah.dev/projects/ai-mariokart). Most modern world models are essentially image generators with additional inputs (past-frames + controls) added on, so understanding how Diffusion/IADB/Flow Matching work would definitely help.
- Valk3_ 1y agoThanks!