4 ms·
Are there details on how it works? How is CLIP being used here?
by abidlabs 5y ago
Are there details on how it works? How is CLIP being used here?
- osanseviero 5y agoThe code is available in the repo: https://huggingface.co/spaces/osanseviero/draw_to_search/blob/main/app.py https://huggingface.co/spaces/osanseviero/draw_to_search/blo... It first uses a simple CNN to get one of the 100 classes. It then uses that class to retrieve images using CLIP. The original CLIP Space that helped for inspiration is https://huggingface.co/spaces/vivien/clip https://huggingface.co/spaces/vivien/clip.
- NavinF 5y agoOof. That 100 class limit is less practical than just asking the user for arbitrary text input and using CLIP as a zero-shot classifier. You would have gotten better responses if the limitations and motivations were explained on the page.
- osanseviero 5y agoThanks for the feedback! This was just a demo I shared in Twitter and didn't expect it to be shared so widely by others. We're training a new CNN for more classes and we might switch to another dataset for the image retrieval part. It's really a very small pet project :D