3 ms·

Isn't using pretrained CNN models make embedding biased towards extracting features that were relevant from the dataset (i.e. ImageNet) it learned from? Fine-gr

Isn't using pretrained CNN models make embedding biased towards extracting features that were relevant from the dataset (i.e. ImageNet) it learned from? Fine-grained classification uses triplet and siamese networks to learn an embedding based on semantic similarity, but you have to define what is semantically similar in the dataset. I am curious how well pretrain networks generalize for indexing for image search. I think finding papers how pinterest and ebay apply visual search at scale may shed some light on this.

eggie5 8y ago
The successive convolutional layers in popular CNN architectures learn representations from very general (edges) to very specific (dog breeds) from the input to the last conv. layer respectively. Depending on how different your new domain is will dictate on what layer you will take the CNN representation from (early layers or later layers) and whether a generic ImageNet model will work at all. Of course if your new domain is very different than the distribution of the original training data, it is a good technique to fine-tune the network a bit with your data.
- perone 8y ago
  Just an extra note, that is the key point why EuclidesDB support multiple models, so you can have for instance a ResNet trained on ImageNet for some images and another ResNet (same architecture) fine-tuned on your data (domain adapted) for another different semantic space. A concrete example is to think in the example of a fashion company who has fine-tuned different models for different product categories: Model A = fine-tuned to classify between different types of shoes; Model B = fine-tuned to classify between different t-shirt types; EuclidesDB can have these two models and you can add/query items into each one of these different models (hence the concept of "model/module space" that is used by EuclidesDB).

eggie5 8y ago

The successive convolutional layers in popular CNN architectures learn representations from very general (edges) to very specific (dog breeds) from the input to the last conv. layer respectively. Depending on how different your new domain is will dictate on what layer you will take the CNN representation from (early layers or later layers) and whether a generic ImageNet model will work at all. Of course if your new domain is very different than the distribution of the original training data, it is a good technique to fine-tune the network a bit with your data.

perone 8y ago
Just an extra note, that is the key point why EuclidesDB support multiple models, so you can have for instance a ResNet trained on ImageNet for some images and another ResNet (same architecture) fine-tuned on your data (domain adapted) for another different semantic space. A concrete example is to think in the example of a fashion company who has fine-tuned different models for different product categories: Model A = fine-tuned to classify between different types of shoes; Model B = fine-tuned to classify between different t-shirt types; EuclidesDB can have these two models and you can add/query items into each one of these different models (hence the concept of "model/module space" that is used by EuclidesDB).