3 ms·

The demo index puts Google Images to shame with its search by content feature.

by visarga 5y ago

colordrops 5y ago
Google images used to work very well. It seems it was hobbled at some point either intentionally or unintentionally.
- visarga 5y ago
  Another good image search is Yandex. You can do semantic search by image, as opposed to mere duplicate search.
wongarsu 5y ago
Not only does it produce really good results, the interface with it's ability to start another search with either the description or image of any of the results is great for complex searches.
- mkl 5y ago
  Those "search by description" links reveal a strange problem with the index: 1. Search for "monitor": https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2Fclip.rom1504.fr&index=laion_400m&query=monitor https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2... 2. Note that there are several results with AOC model E2260SWDN. 3. Click the search button beside one of them. 4. Note that none of the results contain "E2260SWDN". Searching for "E2260SWDN" by itself is the same. The back button doesn't work in this interface either. The URL changes, but it doesn't actually show that previous search.
  - rom1504 5y ago
    That's interesting indeed! Note that the description search as well as the image search are both using a knn index on embeddings and not exact search. That helps for finding semantically close by items but indeed for exact reference match it might not be the best solution. Re-indexing the dataset with something like elastic search would give the reference search results you expect.
gwern 5y ago
You can look at https://same.energy/ https://same.energy/ for an ex-OAer showing what CLIP-like NNs can do for image search. It's impressive but if Google Images doesn't do it yet, it's almost certainly more to do with the realities of corporate policy, and scaling up & deploying it at GI scale than as a small toy demo on a static set of a few hundred million (as opposed to countless trillions of adversarially changing images). It's certainly not for lack of good NNs. Remember, Google has much better NNs than CLIP already! ALIGN was over half a year ago, and they have MUM and "Pathways" already superseding that. As always, "the future is already here, it's just unevenly distributed"; it's just practical realities about lag - same way that the RNN machine translation models were massively better for years before Google Translate could afford to switch away from its old n-grams approach, or the speech transcription NNs were great long before they ever showed up on Google YouTube auto-captions, or BERT was kicking ass in NLP long before it became a major signal in Google Search, etc.

The demo index puts Google Images to shame with its search by content feature.

colordrops 5y ago

Google images used to work very well. It seems it was hobbled at some point either intentionally or unintentionally.

visarga 5y ago
Another good image search is Yandex. You can do semantic search by image, as opposed to mere duplicate search.

wongarsu 5y ago

Not only does it produce really good results, the interface with it's ability to start another search with either the description or image of any of the results is great for complex searches.

mkl 5y ago
Those "search by description" links reveal a strange problem with the index: 1. Search for "monitor": https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2Fclip.rom1504.fr&index=laion_400m&query=monitor https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2... 2. Note that there are several results with AOC model E2260SWDN. 3. Click the search button beside one of them. 4. Note that none of the results contain "E2260SWDN". Searching for "E2260SWDN" by itself is the same. The back button doesn't work in this interface either. The URL changes, but it doesn't actually show that previous search.
- rom1504 5y ago
  That's interesting indeed! Note that the description search as well as the image search are both using a knn index on embeddings and not exact search. That helps for finding semantically close by items but indeed for exact reference match it might not be the best solution. Re-indexing the dataset with something like elastic search would give the reference search results you expect.

gwern 5y ago

You can look at https://same.energy/ https://same.energy/ for an ex-OAer showing what CLIP-like NNs can do for image search. It's impressive but if Google Images doesn't do it yet, it's almost certainly more to do with the realities of corporate policy, and scaling up & deploying it at GI scale than as a small toy demo on a static set of a few hundred million (as opposed to countless trillions of adversarially changing images). It's certainly not for lack of good NNs. Remember, Google has much better NNs than CLIP already! ALIGN was over half a year ago, and they have MUM and "Pathways" already superseding that. As always, "the future is already here, it's just unevenly distributed"; it's just practical realities about lag - same way that the RNN machine translation models were massively better for years before Google Translate could afford to switch away from its old n-grams approach, or the speech transcription NNs were great long before they ever showed up on Google YouTube auto-captions, or BERT was kicking ass in NLP long before it became a major signal in Google Search, etc.