5 ms·
Find images from movies based on what you draw
- abidlabs 5y agoAre there details on how it works? How is CLIP being used here?
- osanseviero 5y agoThe code is available in the repo: https://huggingface.co/spaces/osanseviero/draw_to_search/blob/main/app.py https://huggingface.co/spaces/osanseviero/draw_to_search/blo... It first uses a simple CNN to get one of the 100 classes. It then uses that class to retrieve images using CLIP. The original CLIP Space that helped for inspiration is https://huggingface.co/spaces/vivien/clip https://huggingface.co/spaces/vivien/clip.
- NavinF 5y agoOof. That 100 class limit is less practical than just asking the user for arbitrary text input and using CLIP as a zero-shot classifier. You would have gotten better responses if the limitations and motivations were explained on the page.
- osanseviero 5y agoThanks for the feedback! This was just a demo I shared in Twitter and didn't expect it to be shared so widely by others. We're training a new CNN for more classes and we might switch to another dataset for the image retrieval part. It's really a very small pet project :D
- deleted 5y ago[deleted]
- flixic 5y agoIt somewhat decently identifies the object drawn, but fails badly to actually find the identified object.
- earthbee 5y agoIt seems to keep suggesting the same few movies no matter what I draw
- Closi 5y agoIt doesn't seem to work for me - you draw a shape and then it returns some completely unrelated images. You then pick one of those images and try to draw it as closely as possible and then it returns a completely unrelated image again.
- catillac 5y agoI thought the weird thing was that with that second step, if I drew a broom exactly for example, and it guessed “broom” correctly, it returns images with no brooms in them at all?
- zuhayeer 5y agoHere’s a demo of it in action: https://twitter.com/osanseviero/status/1447830148225589251?s=21 https://twitter.com/osanseviero/status/1447830148225589251?s...
- gus_massa 5y ago[It takes a long time to load, so be patient.] Nice idea, but I was never able to find the movie I was thinking of. Perhaps I'm too bad drawing.
- pcurve 5y agoI’m trying to draw the silence of lamb poster with Jodie foster but it keeps returning pictures of bearded men. Lol. Let me know if anyone is successful with that poster.
- jakespencer 5y agoI had to come back to comment - I tried to draw the Batman symbol and the first thing that came back was The Silence of the Lambs poster.
- bellyfullofbac 5y agoTried to draw sheep, but it sees cat...
- po1nt 5y agoLiterally anything -> syringe
- ASalazarMX 5y agoCurrent state of A.I. in a few words. I drew a closed fist, it says moon / alarm clock.
- catillac 5y agoDoesn’t work when I try to use it, always returns the same thing, and it seems like everyone else is having the same problem. Sounds cool though and I’ll be excited about the day we progress enough where something like this works! GANs are very cool.
- Kiro 5y agoI don't experience your problems. It returns new stuff for me.
- deleted 5y ago[deleted]
- aliabd 5y agoTry sketching one of these: - bench (my favorite) - drums - airplane - basketball
- deleted 5y ago[deleted]
- darepublic 5y agoFrom what I gather this does object detection on the picture and then from those detected items compares to movie stills that have been similarly labelled. On paper sounds like it could work but in practice .. results are a bit underwhelming when I tried
- wpasc 5y agoTried to draw the Deathly Hallows symbol with no luck :( Cool idea though
- antegamisou 5y agoFirst of all, most results are anime. I tried a few easy-to-draw, pretty popular symbols like Harry Potter's forehead scar, The Incredibles logo but got nothing close to them. It's a very cool idea but still needs a lot of work.
- shadyrudy 5y agoDrew Batman in the Animated Series style. Kept getting a cat and Deadpool in the Flashdance pose. Cool, but still has a ways to go.
- cmenge 5y agoWas trying to draw the "Cross of Coronado" from Indy Jones, got 2001 - A Space Odyssey... There goes my artist career :)
- daveslash 5y agoI got a Chainsaw....
- dexterdog 5y agoThis is not ready for prime time. This is not even ready to go up against infomercials at 4am on a Sunday morning.
- PeterBarrett 5y agoLooks like it's just guessing the object and then finding images tagged with that object. I was hoping for some sort of scene detection to find specific scenes from films.
- sigg3 5y agoAh shucks! Well, you saved me a click. Was hoping for some AI visual processing thing and was doing the mental gymnastics of how to store all that data.
- angryGhost 5y agomy mona lisa drawing is apparently a tree
- fennecfoxen 5y agoGiven the tiny drawing area and the insanely thick brush, you're lucky you got something as coherent as 'tree'
- garyfirestorm 5y agoI tried to draw a stick figure and it didn’t work for me.