2 ms·
I'd be curious to pair up word2vec and these new image classification algorithms. It would be neat for a program to identify objects and then reason about heir
by mrfusion 10y ago
I'd be curious to pair up word2vec and these new image classification algorithms. It would be neat for a program to identify objects and then reason about heir relationship to other objects.
In fact the word associations could give it clues about what's in the image. For example it sees a picture of woman standing next to a king. So it uses word2vec to guess King + woman = queen.
What do you guys think? Anyone want to build a little prototype?
- nicklo 10y agoDeep image captioning is essentially what you are describing. Check out: http://cs.stanford.edu/people/karpathy/deepimagesent/ http://cs.stanford.edu/people/karpathy/deepimagesent/ http://cs.stanford.edu/people/karpathy/densecap/ http://cs.stanford.edu/people/karpathy/densecap/
- dumitrue 10y agoApart from the captioning work already mentioned, there's also visual analogy making work by Reed et al. http://www-personal.umich.edu/~reedscot/nips2015.pdf http://www-personal.umich.edu/~reedscot/nips2015.pdf which is pretty exciting (sort of a converse to what you're proposing).
- Matumio 10y agoTry looking for papers about image caption generation. For machine translation you can squeeze the information of a sentence into a vector (using a LSTM). You can then reconstruct the sentence in a different language. There was a paper how to generate such a "sentence representation vector" from an image instead, translating from "image" to "english".