I have only vague idea about this (I worked on small 3d games many many years ago), but I imagined something similar to what you described. Basically you use S

I have only vague idea about this (I worked on small 3d games many many years ago), but I imagined something similar to what you described. Basically you use Sora to generate a promising scene, then you ask it (or another model) to turn that scene into a scene graph in a text file. It will make mistakes, but it could work similarly to the Python interpreter in ChatGPT--it can iterate until everything is OK. Maybe there could even be some adversarial stuff where the scene graph is rendered on the fly to compare it to the generated clip, etc. And then you can use you standard toolset to edit it, probably enhanced with a copilot model to automate as much as possible.