3 ms·
Very clever, and "obvious" only in hindsight: Training a deep convnet to colorize all frames in a grayscale video clip from a single color frame taken from the
by cs702 8y ago
Very clever, and "obvious" only in hindsight: Training a deep convnet to colorize all frames in a grayscale video clip from a single color frame taken from the same clip induces the neural net to learn to track all objects in the video, with robustness to occlusions, change of viewing angles, etc. Labels are not required; only a color frame from each clip. Most impressively, the embeddings learned by the convnet (i.e., the representations learned by the next-to-last layer) are linearly separable by object. Very nice!
- Eridrus 8y agoSimple to explain, yes, but I feel like this isn't even really "obvious" even in hindsight. This whole thing is very clever. The only complaint I have is that it's not better than supervised object tracking, so I wonder if this idea is too late? To draw a parallel to image classification, at one point in time neural nets were trained with a bunch of unsupervised pre-training using reconstruction loss, but that technique has basically fallen by the wayside as we've gotten larger datasets and found a pile of tricks for training them from scratch.
- cs702 8y agoLabeling object locations in all frames of a large number of video clips is significantly more expensive than labeling a comparably large number of images.
- Eridrus 8y agoSure, but like Imagenet, these datasets already exist. So unless these models are quite brittle to the objects being tracked, this is likely not going to be an issue.
- vanderZwan 8y ago> Sure, but like Imagenet, these datasets already exist. I'm pretty sure that making it cheap to use new datasets is very valuable in the long run
- vanderZwan 8y agoNow I want to know if using two input videos to mimic our two eyes makes this even more robust, or even gives depths sense for free?