3 ms·
Again, I understand what they are doing very well. They noticed that colorization tends to track things in video. Of course colorization tracks to objects, it
by AboutTheWhisles 8y ago
Again, I understand what they are doing very well. They noticed that colorization tends to track things in video.
Of course colorization tracks to objects, it wouldn't work if it didn't.
This is essentially automatic video image segmentation, which itself is heavily derived from and related to natural image matting.
Natural image matting could even be seen to be a combination of clustering and somehow solving (or minimizing) the error in the matting equation described by porter duffman compositing algebra.
So, automatic video segmentation can be seen as clustering over 3 dimensions of pixels - x, y and time, with some loose expectations of coherency over time.
There are many ways to achieve this, which should be obvious if you watch some of the videos or glance at some of the papers I've linked.
One simple way is with a bilateral filter to iterate over the volume of pixels, which gradually clusters them together. One of the papers shows this technique.
Everything I linked gives much better results. They don't require 'deep learning' and the idea that colorization follows objects is so trivial it's nonsense to make a paper out of it. This is more a case of visibility and most people not knowing the research that has already been done. That's understandable for people here, but the authors of this paper should have known better.