4 ms·
The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies
- ljf 10y agoGreat to site but the title is all wrong, these aren't photo realistic images from drawings, this is taking a sketch, classifying it and trying to find a similar photo. From the site Abstract We present the Sketchy database, the first large-scale collection of sketch-photo pairs. We ask crowd workers to sketch particular photographic objects sampled from 125 categories and acquire 75,471 sketches of 12,500 objects. The Sketchy database gives us fine-grained associations between particular photos and sketches, and we use this to train cross-domain convolutional networks which embed sketches and photographs in a common feature space. We use our database as a benchmark for fine-grained retrieval and show that our learned representation significantly outperforms both hand-crafted features as well as deep features trained for sketch or photo classification. Beyond image retrieval, we believe the Sketchy database opens up new opportunities for sketch and image understanding and synthesis.
- ljf 10y agoIn fact the original title of the page is: The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies Patsorn Sangkloy Nathan Burnell Cusuh Ham James Hays
- nicklo 10y agoWould be super cool if someone applied deep generative models to synthesize new natural images from input drawings using this dataset.
- xentripetal 10y agoI believe one came out recently, I saw it on GitHub a few days ago. It mainly did landscape and searched for similar images then combined them to form a new one. It also supported image manipulation, eg turning a brown square purse into a green rounded one. I'll see if I can find it and edit the link in. EDIT: https://github.com/junyanz/iGAN https://github.com/junyanz/iGAN Heres a good video showing it off https://www.youtube.com/watch?v=9c4z6YsBGQ0&feature=youtu.be https://www.youtube.com/watch?v=9c4z6YsBGQ0&feature=youtu.be
- DanielleMolloy 10y agoThis is a recent paper that actually creates photos from sketches of faces, using deep nets: https://arxiv.org/abs/1606.03073 https://arxiv.org/abs/1606.03073 Figure 4 does this for self portraits of some famous artists.
- aab0 10y agoYou could probably apply the same code. The dataset ("acquire 75,471 sketches of 12,500 objects") sounds adequate, and if not, can be boosted by first training a CNN to do photo->sketch (throwing away information is usually easier than imagining it) and using that to boost the dataset for sketch->photo.
- sctb 10y agoWe've updated the submission title from “Photorealistic images from drawings”.
- donpark 10y agonice although keywords and photos are more practical query methods than sketches which are harder to input and thus less expressive (i.e. sketch of a bear returning teddy bears and not real bears).
- cesis 10y agoThere definately are a lot of applications for scetch based retrievals as some features are hard to describe eg. a landmark from a specific angle etc.
- rokhayakebe 10y agoCommunication between people who speak different languages that aren't commonly known.
- norikki 10y agoSo if someone sketched something that hadnt been sketched before it wouldnt find an image for it. Not nearly as interesting at it seemed at first.
- ageofwant 10y agoMy vagina drawing skills, while not effective, did not disappoint.