4 ms·
I think the bigger question is would it be stable enough. Many SD like models struggle with consistency across multiple images (i.e. frames) even when content d
by NBJack 3y ago
I think the bigger question is would it be stable enough. Many SD like models struggle with consistency across multiple images (i.e. frames) even when content doesn't change much. Would he a cool problem to see tackled.
- erwannmillon 3y agotemporal coherence is def an issue with these types of models, though I haven't tested it out with ColorDiffusion. Assuming you're not doing anything autoregressive (from frame to frame) to do temporal coherence, you can also parallelize the colorization of each frame, which would affect cost. Tbh most cost effective would be a conditional GAN though
- lajamerr 3y agoChange up the model. That allows it to see previous frames and 1-2 future frames. Then train the model on movies that are color and then turn them black and white. That way you can train temporal coherence.