6 ms·

Thanks for all the upvotes! Since I made this site, people have already started to train on datasets beyond just real faces. Turns out it can disentangle pretty

• x3tm 8y ago
  Cool page, and great job. > Turns out it can disentangle pretty much any set of data. All the example I have seen (including your links) are variants of face generation algorithms. Any ideas on how this could be useful beyond image generation in some style? Specifically for (data) science? Sorry if this is a naive question. Edit: By "variants of face generation algorithms" I mean any image generation really.
  • lucidrains 8y ago
    no not naive at all. this method isn't specific for just extracting features from faces. it can disentangles features from any kind of images. in fact, the next dataset i might train on is on flowers (or birds) https://twitter.com/hardmaru/status/1095639937842638849 https://twitter.com/hardmaru/status/1095639937842638849
    • x3tm 8y ago
      OK, my point is what could be done beyond generating images in some style? Can we generate interesting mock data given a database for instance (of course this is exactly what you did in a way, but I have in mind e.g. a database containing some numerical/categorical features known to a specific accuracy)?
      • gwern 8y ago
        You can use GANs to generate fake data based on stuff like particle accelerator data or electronic health records. Whether you can use StyleGAN specifically is unclear. What's the equivalent of progressive growing on tabular/numeric data? Or style transfer?
  • gwern 8y ago
    The original Karras et al 2018 paper did both cars and cats, which aren't faces. Worked very well, unsurprisingly. (ProGAN also did well on those, though it was the faces everyone paid attention to.) Look at the samples in the paper or the Google Drive dumps, or at the interpolation videos have posted on Twitter. Aside from the original work, on Twitter, people have done Gothic cathedrals very well, graffiti very well, fonts very well, and WikiArt oil portraits not so well. On Danbooru2017 full anime images (linked in my thread), one person has... suggestive blobs but has only put 2-3 GPU-days into it and we aren't expecting much so early into training. skylion has been running StyleGAN on a whole-body anime character dataset he has, and the results overnight (on 4 Titans) are pretty impressive but he hasn't shared anything publicly yet.
    • lucidrains 8y ago
      Great job on the Danbooru training! I've been following you on twitter and machinelearning for the longest time haha
      • gwern 8y ago
        Thanks! The wait on training is killing me, though. I've been doing large minibatch training to try to fix the remaining issues in the anime face StyleGAN and it's frustrating having to wait days to see clear improvement. Checking GAN samples is so addictive and undermines my ability to focus & get anything else done. I'm also eager to get started on full Danbooru image training, which I intend to initialize from skylion's model - whenever that finishes training... (Who says we aren't compute-limited these days?!)
        • lucidrains 8y ago
          Haha, having to work around the computation limits are welcoming! It feels like building web apps back in the late 90's again. These days we have so much memory and disk space at hand it doesn't even feel like a challenge anymore. That is, until Graphcore delivers their IPU.
    • gwern 8y ago
  • myegorov 8y ago
    Could be used to generate building plans or other schematics (pretty sure of no use though). Could certainly be put to good use generating pornographic images.
• disease 8y ago
  That artwork demo has really piqued my interest regarding using techniques like these for creating animation more easily.
• tropshop 8y ago
  So does this make it trivial to input a source portrait, and then visualize different hair styles?
  • lucidrains 8y ago
    you'll have to build an encoder to encode someone's face into the latent space. then you'll have to dive into the latent space and find the dimension(s) that controls for hair style (just fork the colab and start experimenting with interpolations) then yes, it should be possible
• subcosmos 8y ago
  Love the work you guys are doing in the progressive GAN space. Last year I did something similar to make a face-aging network, involving training an encoder to get an initial guess of a latent vector for someones face into the pgan space, and then relied on BFGS optomization to fine-tune the latent vector, followed by further fine-tuning of some intermediary layers of the generator network to really match the input pixels. I also snuck an affine transform layer in there allowing the network to shift the image around to better fit the target. The results were .... eh .... okay, at least on my ugly face. https://twitter.com/RustBot/status/1044120159022022658 https://twitter.com/RustBot/status/1044120159022022658 But overall, Im still tweaking. In the mean time, I've been focusing on static image analysis for aging research, but I hope to find better encoding schemes down the road. https://twitter.com/RustBot/status/1058607878540472320 https://twitter.com/RustBot/status/1058607878540472320
• cr0sh 8y ago
  I love this part of the system req's (from the stylegan repo): "One or more high-end NVIDIA GPUs with at least 11GB of DRAM. We recommend NVIDIA DGX-1 with 8 Tesla V100 GPUs." I'm sure my wife will understand why I took out that second mortgage on our home...no problem.
  • genai 8y ago
    Or use on the cloud, much cheaper https://cloud.google.com/nvidia/ https://cloud.google.com/nvidia/
  • rice_otaku 8y ago
    They recommend the 8 Tesla, but would a 2080Ti work? That's not too bad. I might try this at home with my gaming pc.
• lucidrains 8y ago
  Looks like someone else has already used another neural net to find our president in the latent space. https://github.com/Puzer/stylegan https://github.com/Puzer/stylegan
• kowdermeister 8y ago
  I like the fact that the server returns an image directly without any HTML or other content. It makes the loading experience fun too. How many faces are generated? This can't be real time.
  • lucidrains 8y ago
    Thanks! Initially, I thought about just generating a big batch and cycling through it. Then I thought it would be more dramatic if the machine was "dreaming" up a face every 2 seconds in real time. I went for the dramatic approach just so I could phrase it that way to my non-tech friends!
    • jm547ster 8y ago
      Oh wow so it really is. Great work
    • meruru 8y ago
      Are the old images discarded? It would be interesting to use these as references for hyper-realist drawings. You could have something completely authentic that could never be traced back to its source.
      • deleted 8y ago
        [deleted]
    • dzhiurgis 8y ago
      Google image search must be getting smashed by this. Is there a canonical exif tag, perhaps isBot?
• daurnimator 8y ago
  What license are the generated images under? Could you release them under creative commons?
  • tomcam 8y ago
    They're probably public domain [1,2]. Generally in the USA and Europe, you can't copyright computer generated images or images created by nonhuman entities. "To qualify as a work of 'authorship' a work must be created by a human being": https://www.copyright.gov/comp3/chap300/ch300-copyrightable-authorship.pdf https://www.copyright.gov/comp3/chap300/ch300-copyrightable-... [PDF], see section 313.2 "Works that lack human authorship" Monkey selfie case: https://en.wikipedia.org/wiki/Monkey_selfie_copyright_dispute https://en.wikipedia.org/wiki/Monkey_selfie_copyright_disput...
    • pests 8y ago
      I feel this is wrong. If I make random generative art I instantly lose copyright? Or how about Photoshop? Really I'm asking myself where is the lone drawn.
      • comepradz 8y ago
        You can though own the algorithm you used to generate the art, as in the case of Fractal Flame[0] created by Scott Draves. [0] https://en.wikipedia.org/wiki/Fractal_flame https://en.wikipedia.org/wiki/Fractal_flame
      • vsl 8y ago
        You don’t “loose” copyright, you never had it in the first place. Copyright is (read the law!) a temporary monopoly granted for works meeting certain criteria, being creative is one of them. You’d hold copyright for the code you wrote to generate the “art”. If you download somebody else’s code (as this site uses Nvidia’s), you lack the creative element.
    • porlune 8y ago
      From the Wiki article of the "Monkey Selfie Case" >>> On 23 April, the court issued its ruling in favor of Slater, finding that animals have no legal authority to hold copyright claims [1] [1] https://petapixel.com/2018/04/24/photographer-wins-monkey-selfie-copyright-case-court-slams-peta/ https://petapixel.com/2018/04/24/photographer-wins-monkey-se...
• imoldtoo 8y ago
  Some enterprising developer could use images from Tinder (or better) OkCupid tagged with data coming from the individuals profile data, then interpolate based on abstract factors such as risk taking, gender bias etc. Well... you get the picture. I think this is a very dangerous game we are playing here but I guess it is going to be done.
• indigodaddy 8y ago
  @lucidrains - this is pretty amazing. Every time you refresh the page, is it a real-time generation, or does it draw from a pool/DB of real-time images generated previously? I got the exact same image twice, which is why I am asking, which kind of dampened the "cool" factor just a notch.
• uwbassist 8y ago
  Hey, you might want to consider bert-as-service[0] for deep feature extraction from a BERT model. It will give you a 768 dimensional representation of the description, then you can embed that in the 512-dim latent space? I've been thinking of something similar. It's not that hard to do it yourself, but it's a really clean package, and it gives you nice CLI flags for most things like pooling strategy, and what layer you want to get the activations from. [0] https://github.com/hanxiao/bert-as-service https://github.com/hanxiao/bert-as-service