I think it would be cool if you could have some kind of combo human+ML system, where you use ML to actually detect the different objects and track them between

I think it would be cool if you could have some kind of combo human+ML system, where you use ML to actually detect the different objects and track them between frames, and a human can choose what color to use for each one. In a couple shots at least the Hindenburg showed as fully red instead of silver, and I don't think it is always possible for an algorithm to actually know the correct color without external data. If the algorithm could just say "I've identified this object between frames", and then the human can choose the correct color, could be best of both worlds. Or maybe you could feed in some reference photos with the same objects, but already colorized. And then the algorithm could match objects from the reference photos to those in the black and white ones to get the colors.