Caffe (and other frameworks) provide exactly this. It's basically: 1) To setup, load a pre-trained AlexNet/Overfeat/other architecture model (e.g. trained on

Caffe (and other frameworks) provide exactly this. It's basically: 1) To setup, load a pre-trained AlexNet/Overfeat/other architecture model (e.g. trained on ILSVRC2012) 2) To get a vector from an image, run a forward pass on the images, and extract the activations at a given layer (e.g. fc7) as the output vector. http://caffe.berkeleyvision.org/gathered/examples/feature_extraction.html http://caffe.berkeleyvision.org/gathered/examples/feature_ex... is a step-by-step walkthrough. There's a lot of mystique around deep learning and these kind of problems, but it's not _that_ difficult to use these tools.