Thing you are talking about is similar to scene labeling (google the term and look at the pictures). Only you'd be doing it by hand, instead of letting NN find

Thing you are talking about is similar to scene labeling (google the term and look at the pictures). Only you'd be doing it by hand, instead of letting NN find it's own representation. This is good because you'd be creating a dataset full of regions that we could use to automatically segment the pictures into useful regions later. There's even a language mapping that you want, you want it to recognize image parts and associate it with a word, which also isn't simple, because you'd have to have a lot of labels (candles, hands, hairs, cars, trees, licence plates etc.) It is a harder problem than the one in the article. State-of-the-art scene labeling is still not good enough (close to 80% accuracy) but I believe it's due to lack of data because algorithms used combine neural networks with joint learning approaches such as conditional random fields to extract the regions.