12 ms·
If you're wondering how it's so fast and cheap and they can generate variants so easily, it's because they're using GANs (see the footer). GANs are way faster t
by gwern 3y ago
If you're wondering how it's so fast and cheap and they can generate variants so easily, it's because they're using GANs (see the footer). GANs are way faster than diffusion models because they generate the image in a single forward pass and their true latent space encoding makes editing a breeze.
(And if you're wondering how it can look so good when 'everyone knows GANs don't work because they're too unstable', a widespread myth, repeated by many DL researchers who ought to know better, GANs can scale to high-quality realistic images on billion-image scale datasets, and become more, not less, stable with scale, like many things in deep reinforcement learning. See for example BigGAN on JFT-300m https://arxiv.org/pdf/1809.11096.pdf#page=8&org=deepmind https://arxiv.org/pdf/1809.11096.pdf#page=8&org=deepmind , GigaGAN https://arxiv.org/abs/2303.05511 https://arxiv.org/abs/2303.05511 , Projected GAN https://arxiv.org/abs/2111.01007 https://arxiv.org/abs/2111.01007 , StyleGAN-XL https://arxiv.org/abs/2202.00273 https://arxiv.org/abs/2202.00273 , or Tensorfork's chaos runs https://gwern.net/gan#tensorfork-chaos-runs https://gwern.net/gan#tensorfork-chaos-runs . 'Computing is a pop culture'...)
- motoboi 3y agoWhat I’m really wondering is how can this be free. What is the business model here? Are they using me to refine the model in some way?
- htrp 3y agoHuman evaluation is super expensive so yes. Seeing which sessions you discard and keep alone is worth the compute time, especially if it's GAN based.
- irrational 3y agoIt says free for non-commercial. If commercial, contact us. I assume they plan on paying the bills with commercial work.
- brucethemoose2 3y agoThere was a whole community around ESRGAN img2img finetuning kinda like Stable Diffusion LORA community... albeit a much, much smaller one.
- lacoolj 3y agomaybe it was fast 58 minutes ago but apparently it is now at peak capacity. even if you don't get rejected, a new image takes minutes
- syntaxing 3y ago> everyone knows GANs don't work because they're too unstable Is that a wide spread myth? I thought it’s widely accepted that GAN is really good generating these artificial pictures (it’s what started DeepFake after all) when you know your model’s “button”. Similar to how this uses GAN since they have a model “boundary condition”. While humans are diverse, we have a set of repeatable features (two legs, two arms, etc). Diffusion models are great because you can control the latent space with something way more generic, like text hence why it’s been so much more mainstream. Edit: actually I might be misremembering, I think Deepfake used VAEs?
- gwern 3y agoIt is very widespread. You will see people in this very thread dismissing GANs as fundamentally failed, and hotly objecting to any kind of parity, even if they have to fall back to 'well ok GANs do scale, but they're more complicated'. I also have some representative quotes in my linked draft essay from various papers & DL Twitter discussions. (Another way to put it would be: when was the last time you saw someone besides me asserting that GANs can scale to high-quality general images and are not dramatically inferior to diffusion? I rest my case.)
- ShamelessC 3y agoSounds very important to you that you don’t have to change the premise of your essay or ever admit you’re wrong. No one is dismissive of GAN’s here without justification. They’re fine. They don’t beat diffusion, but they’re fine. You come across as severely, _severely_ biased and reactionary.
- deleted 3y ago[deleted]
- ShamelessC 3y ago> Another way to put it would be: when was the last time you saw someone besides me asserting that GANs can scale to high-quality general images and are not dramatically inferior to diffusion? I rest my case. Get off of Twitter, maybe? I don't know but I have seen a ton of work done on large scale GANs. The incentives are pretty fucking obvious - they are faster to use. Of _course_ people want them to work for the same usecases as diffusion. Maybe they will someday! But they aren't currently. You're referencing a zeitgeist that I am not familiar with and therefore come across overly defensive.
- nbardy 3y agoYou’re overstating the simplicity of a scaling a GAN well. GigaGAN is the best quality out of those and requires 7 loss functions and is incredibly complicated. Sure GANs can scale, but Diffusion models are drastically easier to scale.
- gwern 3y agoNo, I'm not. BigGAN did fine on scaling up to JFT-300M with basically no changes beyond model size and a simple architecture. This is also what we were observing, even with a buggy BigGAN implementation. GigaGAN is the best quality, but that's mostly because it's also the biggest; as Table 1 shows most of the gains come from various kinds of additional scaling. (And this is moving the goalposts from the usual assertion that "GANs can't scale" to "they're harder to scale"; note the self-fulfilling nature of such assertions. Considering how there is next to no GAN scaling research, these results are remarkable and show how much low-hanging fruit there is.) Diffusion models are only 'drastically easier to scale' because researchers have spent the past 3 years researching pretty much nothing but diffusion models, discovering all sorts of subtle issues with them and how to make them scale, which is why it took them so long to become SOTA, and why massive architectural sweeps like https://arxiv.org/abs/2206.00364#nvidia https://arxiv.org/abs/2206.00364#nvidia were necessary to discover what makes them 'easier to scale'. If this level of brute force and moon math is 'easy', lord save us from any architecture which is 'hard'!
- GaggiX 3y agoResearchers have spent several years try to even create a GAN that can fit well a distribution made of aligned faces (resulted into StyleGAN1/2); with a simple unet with e-objective and cosine schedule you can fit much complex distributions, still using one loss: L1/L2. Reading your comments make me feel like that you believe that just every researchers (even extremely smart dude like Karras) just switch to diffusion models because they are idiots, they should have instead focus on GANs and today we will have GANs that are as powerful or more than the diffusion models we have today and also work one step; this is just a weird delusion. Diffusion models are just simply much easier to train (just a L1/L2 loss in most cases), write (for example your buggy BigGAN implementation), they usually work out-of-box on different resolutions and aspect ratios, you can just finetune them if you want to create an inpainting model; and for what is right now you just need much less compute to reach a good image coherency or maybe just reaching a coherence that as not been achieved by GAN models; like I would be curious even on a small scale experiment what a GAN (with ~55M parameters) would be able to perform after a 1-day/2-day GPU time of training on Icon645 dataset, because my diffusion model I can assure is much better than I could have imagine while being trivial to implement (I just implemented a Unet as I remember one, nothing rigorous and of course no architecture sweep).
- dragonwriter 3y ago> If you're wondering how it's so fast and cheap and they can generate variants so easily I assume its cheap because they are burning money to build a business, its not fast at all, and the quality... sucks. > And if you're wondering how it can look so good I’m not. I'm wondering why they’re trying to get people to use something worse than using a decent photorealistic SD1.5-based checkpoint with some basic prompt templating. Not saying GANs can't be awesome, just that this site isn't what I’d use to make that case.
- mrguyorama 3y agoLooking at the poses, it feels optimized for generating porn, but one example someone showed had a child's face (good god please don't let your "AI" system generate child anything if you want to sell it for porn purposes), and another user noted that their attempt error'd out because it "detected nudity", even though other users get given a nude model by default.
- dotancohen 3y agoI don't know if the thing in the crotch is a penis or a scrotum, but it is definitely NSFW: https://images.generated.photos/0wV1dBnZ15hGneEfqfZT7SdEIillbpDDGY2wX5XkTeI/rs:auto:1024:1024/wm:1:sowe:18:18:0.5/wmid:gp/czM6Ly9odW1hbi1n/ZW5lcmF0b3ItcHJv/ZC83MTkvODVjZTZh/YTctZTU0Yy00Njcx/LThlMGItZTA3OGMy/NGU1ZDAyLnBuZw.jpg https://images.generated.photos/0wV1dBnZ15hGneEfqfZT7SdEIill... My prompt was simply "Standing in front of a rocket.".
- GaggiX 3y ago>they're too unstable', a widespread myth >See for example BigGAN I remember when you try training a BigGAN model on anime images, the quality was bad. Now look at this example, one single GPU, 1.5M images with a diffusion model: https://medium.com/@enryu9000/anifusion-diffusion-models-for-anime-pictures-138cf1af2cbe https://medium.com/@enryu9000/anifusion-diffusion-models-for... ,the difference in quality is absurd, you can say this or that is not true but the quality speak for itself, obtaining good quality on complex distribution is much easier with a diffusion model than a GAN. For example in the case of the site linked they have conditioned the model on poses because you're not going to get anything close to be coherent without them with a simple StyleGAN as they say they're using.
- Tyr42 3y agoBroken link?
- GaggiX 3y agoFixed thx
- gwern 3y ago> I remember when you try training a BigGAN model on anime images, the quality was bad Because there was a bug in the code, in a part unrelated to the GAN itself. > the difference in quality is absurd Yes, it does help to train on anime with code that isn't buggy. (BTW, Skylion was getting good results with GANs on anime similarly restricted to centered figures like those samples, he just refuses to ever publish anything.)
- GaggiX 3y agoSo you believe that without the bug you would be able to come close to the quality of the diffusion model I have linked? I'm not even asking about using the same compute (1 GPU for ~1 month) but if you just believe BigGAN can come close to that in general. Also the bug is probably related to the added complexity of training a GAN model in comparison to a diffusion model.
- sebzim4500 3y ago>And if you're wondering how it can look so good I don't think anyone is wondering this, especially if they are used to playing with diffusion models.
- jug 3y agoYeah, honestly this whole comment thread is a testament of how quickly generative AI has moved the past year. Even in a hacker community like this where we ought to be close in terms of realizing where current tech stands, we're on different pages on what's possible. Photorealism is pretty much already here and this website is a poor example of current generative AI. The most it has going for it is that it has an approachable UI.
- tavavex 3y agoWhile there's discussion on the topic here - are there any resources that can explain the exact mechanism of how a GAN works for image generation? I have a rough idea of how diffusion models work, but I'm still no AI researcher.
- dublin 3y agoHow it can look so good? ROFL!! It just created a guy with a hand coming off his left ankle in place of a foot, and toes or fingers or something poking out the end of the show on his right foot! https://generated.photos/human-generator/64e682308448b8000c557a6f https://generated.photos/human-generator/64e682308448b8000c5...
- notahacker 3y agoFour hands is much better for climbing industrial installations so you can hang in mid air next to them
- wussboy 3y agoIf it isn't my old friend, Willie McGreg. With a leg for an arm and an arm for a leg!
- bravura 3y agoHi gwern, asking earnestly and without snark about this: "And if you're wondering how it can look so good when 'everyone knows GANs don't work because they're too unstable', a widespread myth, repeated by many DL researchers who ought to know better, GANs can scale to high-quality realistic images on billion-image scale datasets, and become more, not less, stable with scale, like many things in deep reinforcement learning." My interpretation from how this is written is that you are saying: "Researchers say that GANs are unstable at inference time." Is that the correct reading? If so, where have you seen this sentiment? More commonly I've heard that people criticize GANs at inference time for mode collapse and monotony (which comes from training), not instability. Or are you saying that researchers say GANs are unstable during training, which is a common criticism. Don't you feel this is the case? A lot of different tricks are used for getting generators that are roughly balanced with more powerful discriminators so that the adversarial game is balanced, like TTUR and encodec's weight balancer etc. In this case, are you saying that GAN training is as straightforward as diffusion training? In my experience, GAN training involves an unholy number of heuristics and best-practices are still quite murky. I am eager to hear your response. [edit: my experience with GANs and diffusion models isn't about taking image problems with existing working models and scaling or refining them, but about applying GANs and diffusion models to different domains (audio) and on novel problems with different kinds of conditioning. I would love to see more controlled experiments comparing weak and sophisticated generative backbones as the experimental control and varying the training regime across different GAN + diffusion flavors.]
- justinclift 3y agoTrying it out just now, it's still getting major anatomical features wrong, and hands/fingers are still a noticeable problem. eg (NSFW!): https://generated.photos/human-generator/64e741e6c4c0f80009ef72e6 https://generated.photos/human-generator/64e741e6c4c0f80009e... Wrong number of limbs, wrong number of fingers on hands, etc. The result is ok sometimes, but that's probably only about 5% of the time. Here's a decent result (very NSFW) without immediately noticeable deformities: https://generated.photos/human-generator/64e7590e482612000b1ba229 https://generated.photos/human-generator/64e7590e482612000b1... Probably 95% of the images generated so far have been clearly deformed though.
- MasterScrat 3y agoI’ll take this opportunity to mention our research scaling StyleGAN 2 to larger datasets (using LAION) on food images, leveraging free TPU compute through the TRC program. We trained for 36 days on a v4-8 on 558k images. https://nyx-ai.github.io/stylegan2-flax-tpu/ https://nyx-ai.github.io/stylegan2-flax-tpu/ We were hopeful GANs would beat diffusion models when trained on specific domains. But we’ve now switched to Stable Diffusion and Dreambooth training which has proven more much efficient for this purpose. I still have hopes for GANs! I miss their insane inference speed.