3 ms·
This is a text to image search using deep learning, vector similarity search. Ask me anything.
by okaydeveloper 5y ago
This is a text to image search using deep learning, vector similarity search. Ask me anything.
- yobbo 5y agoWhat in your system is doing the text-to-vector encoding, and how did you train it?
- alexcg1 5y agoWe're using Transformers with `sentence-transformers/paraphrase-distilroberta-base-v1` model. The framework is Jina (https://github.com/jina-ai/jina/ https://github.com/jina-ai/jina/) so it's pretty high-level. You can see the indexing/search Flow on lines 37-52 of https://github.com/alexcg1/jina-meme-search-example/blob/main/app.py https://github.com/alexcg1/jina-meme-search-example/blob/mai...
- alexcg1 5y agoWe rely on pre-trained models at the moment, since Jina supports loads of them out of the box. For image search we use Big Transfer Encoder (https://github.com/jina-ai/executors/tree/main/jinahub/encoders/image/BigTransferEncoder https://github.com/jina-ai/executors/tree/main/jinahub/encod...) but may switch to CLIPImage encoder at some point
- jstx1 5y agoWhat's the point of the deep learning model? Why not just search the metadata of the images?