3 ms·
The semantic understanding feels much richer than diffusion based modeling, e.g. the trees on the shore growing to match the manipulated reflection, the sun cha
by vagabund 3y ago
The semantic understanding feels much richer than diffusion based modeling, e.g. the trees on the shore growing to match the manipulated reflection, the sun changing shape as it's moved up on the horizon, the horse's leg following proper biomechanics as its position is changed. I haven't gotten such a cohesive world model when doing text-guided in-painting with stable diffusion etc. This feels like it could very conceivably be guided by an animation rig with temporally consistent results.
- orbital-decay 3y agoTemporal consistency for a guided scene is a separate problem. It's been kind of solved a couple years ago. [0] It can be used with animation rigs and simplistic tagged geometries, and it even works in near real-time. "Kind of" because training a model from scratch from a large dataset is not something you want to do for the actual job; what you want is a good style transfer mechanism that can extract features from as few references as possible. [0] https://isl-org.github.io/PhotorealismEnhancement/ https://isl-org.github.io/PhotorealismEnhancement/