5 ms·
There are lots of inconsistencies in these clips of the type you would never find even in a hastily put together amateur film. I wonder how you would even add c
by chrsw 2y ago
There are lots of inconsistencies in these clips of the type you would never find even in a hastily put together amateur film. I wonder how you would even add continuity support into a generative video model. It's got its training data, its model, its algorithms for generating data... but could you say "make sure this shirt always has 6 buttons in this scene"? Does it even understand what a button is? Or a shirt? Or a thing?
It seems to me that eventually these systems are going to have to be grounded in some hard truths about our world. Like, there are things called objects, objects can be distinct, objects can have relationships between other objects, etc. Then the generative network would have to generate data around these priors. Or maybe they already have that, I don't know how they work.
- jononor 2y agoHopefully continuity (of relevant features) would be the results of the training process, eventually. In videos from the wild, the number of buttons on a shirt basically never changes during a scene. That kind of information is in the training data already. So it is theoretically possible for a model to learn that should stay consistent, in contrast to other properties that affect the shirt, like lighting or pose. But we are still in the very early days of kinda-working video generation, and certainly in terms of temporal consistency.