4 ms·
The "sketches to handbags" example, which is buried toward the bottom, is really cool. It's basically an extension of the "edges to handbags," but with hand-dr
by jawns 10y ago
The "sketches to handbags" example, which is buried toward the bottom, is really cool. It's basically an extension of the "edges to handbags," but with hand-drawn sketches.
Even though the sketches are fairly crude, with no shading and a low level of detail, many of the generated images look like they could, in fact, be real handbags. They still have the mark of a generated image (e.g. weird mottling) but they're totally recognizable as the thing they're meant to be.
The "sketches to shoes" example, on the other hand, reveals some of the limitations. Most of the sketches use poor perspective, so they wouldn't match up well with edges detected from an actual image of a shoe. Our brains can "get the gist" of the sketches and perform some perspective translation, but the algorithm doesn't appear to perform any translation of the input (e.g. "here's a sketch that appears to represent a shoe, here's what a shoe is actually shaped like, let's fit to that shape before going any further"), so you end up with images where a shoe-like texture is applied to something that doesn't look convincingly like a real shoe.
- ape4 10y agoThis is be a popular shopping website. Sketch your perfect handbag. See an image of the product. Click to buy.
- deleted 10y ago[deleted]
- daveguy 10y ago"Sketch your perfect handbag" may be a bit much to ask of most people.
- gwern 10y agoIt is, but this sort of approach is easily applied to transformations (in fact, an earlier GAN HN submission was "Generative Visual Manipulation on the Natural Image Manifold" https://people.eecs.berkeley.edu/~junyanz/projects/gvm/ https://people.eecs.berkeley.edu/~junyanz/projects/gvm/ , Zhu et al 2016b ). So you can start with a lousy sketch and transform it until it works, or you can start with a pre-populated set of handbags, recognize the one closest to what you want, and tweak/sketch that.
- sharemywin 10y agoDraw your perfect handbag to share with friends. you only need 10 buyers to have it created.
- fudged71 10y agoI doubt friends want the same handbag through! It's an anti-viral feature: Personalization.
- deleted 10y ago[deleted]
- dougabug 10y agoThere was a paper at CVPR 2016 called "Sketch Me That Shoe," which basically converted hand sketches to images using tied embedding networks. https://www.eecs.qmul.ac.uk/~qian/Project_cvpr16.html https://www.eecs.qmul.ac.uk/~qian/Project_cvpr16.html