9 ms·
Nvidia releases Alias-Free GAN code and pre-trained models, naming it StyleGAN3
- timzaman 5y agoI worked with these guys at nvidia Helsinki office. They are super chill and just somehow crank out super research. Very interesting bunch.
- tmabraham 5y agoThe visualizer looks fun: https://twitter.com/minimaxir/status/1447679798822649856 https://twitter.com/minimaxir/status/1447679798822649856
- minimaxir 5y agoI was joking, but it appears that StyleGAN3's new approach allowed it to develop unsupervised 3D maps of faces within the deeper layers, which might result in interesting things when hacked by researchers.
- metagame 5y agoYou can't use any of it commercially. Nothing within is under an acceptable software license (nor an open source license, nor a free software license). Advanced warning.
- throwaway889900 5y agoNobody will be able to tell if you stick it in a tiny thumbnail and give it enough jpeg artifacts.
- metagame 5y agoComputer-generated artifacts are non-copyrightable. This is not the problem. (That said, as the law is written, no binary should be, but we already threw that baby out with the bathwater.) The problem is that the software is not free software, but encourages you to stop using its free predecessors and competition sneakily.
- wruza 5y agoOr it’s just a paranoia and they plan to license this to non-free use after a period of testing, leaving non-commercial use for those who want to play with it. That is basically what every non-free software does except the commercial support is not yet provided. Also, I second the question about free predecessors/competition, not to argue or compare, but out of pure curiosity.
- dahart 5y agoI’m not sure I understand your claim, why is this encouraging anyone to stop using any competition? The license says plainly it’s not for commercial use, what’s so sneaky? What do you define as free software? This software is open source, always free as in beer, and free as in freedom for research and evaluation purposes (and seems fairly permissive to researchers…)
- metagame 5y agoIt by definition is not open source. The term has a definition. This breaks literally the first rule. https://opensource.org/osd https://opensource.org/osd
- dahart 5y agoI used, or maybe misused, the term open source. You used “free”. The license & project used neither, and made no claim to align with opensource.org’s philosophy or definition. Whatever you call it, the source code has been released for anyone to read and “evaluate”, that’s what I meant by ‘open’. You didn’t answer the question - how is this sneaky, and how does it prevent using previous projects?
- metagame 5y agoThe Open Source Initiative coined the term to begin with. Using it incorrectly is harmful, and is how we've ended up with "literally" meaning "figuratively" in modern English. By insisting on the correct definition, I'm trying to prevent the same from happening to open source. It's pretty offensive to act like it's not a big deal to use something so essential to computing freedom in a cavalier way to intentionally lessen freedom.
- smlacy 5y agoOh really? https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-r-ffhq-1024x1024/0307.png https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-...
- throwaway889900 5y agoObviously if you're using the GAN equivalent of a Megamind meme you won't get away with it, but more reasonable ones you can.
- natch 5y agoYou are talking about the models, right? If you train your own model on your own data without transfer learning (or with transfer learning from a liberally licensed third party model once those exist) then you can do whatever you want to with your model, no?
- urthor 5y agoJust run the whole codebase through OpenAI Codex, then regenerate the source code. Can't copyright an inferred artifact ;)
- atty 5y agoFrom the license file: > 3.4 Patent Claims. If you bring or threaten to bring a patent claim against any Licensor (including any claim, cross-claim or counterclaim in a lawsuit) to enforce any patents that you allege are infringed by any Work, then your rights under this License from such Licensor (including the grant in Section 2.1) will terminate immediately. Is such a clause legal? I have basically zero knowledge of such things, but it seems like it should be illegal to punish someone for a good faith patent claim.
- nerdponx 5y agoApache 2.0 has a very similar clause. But there might be some subtle differences in the wording that makes this broader or stricter.
- DannyBee 5y agoYes they are legal, and I'm not sure I follow the argument that they shouldn't be
- metagame 5y agoThe argument against it, presumably, is that "If you try and make us pay for committing crime, you won't get access to our toys anymore" is very strange and seems illegal, since the ability to play with toys should not stop anyone from reporting violations of the law. But at the same time, it's definitely legal, for better or for worse, as is pretty much any stunt you pull with the joke that is US IP law.
- DannyBee 5y agoThis is not illegal in any country I can think of. Your take around what it exists to do is, IMHO, over the top. I'm not aware of a country that requires you let people sue you in this sort of situation, or requires that you not terminate their contract if you do.
- trhway 5y agoIs patent violation a crime? My understanding it is a civil issue like you kick me out of your property, and I'll kick you out too.
- avivo 5y agoI appreciate the section on "Synthetic image detection": "While new generator approaches enable new media synthesis capabilities, they may also present a new challenge for AI forensics algorithms for detection and attribution of synthetic media. In collaboration with digital forensic researchers participating in DARPA's SemaFor program, we curated a synthetic image dataset that allowed the researchers to test and validate the performance of their image detectors in advance of the public release. Please see here for more details on detection" https://github.com/NVlabs/stylegan3-detector https://github.com/NVlabs/stylegan3-detector It's important to see this sort of thing happening more and more.
- captainmuon 5y ago> It's important to see this sort of thing happening more and more. Why? If we insist on the authenticity of images, this is holding on to the old status quo in the same way we apply book and record copyright to digital content. We don't allow what the tech enables to the fullest, but we restrict it by pressing it into the old mold (e.g. by using DRM to make music a commodity). I think "photographic proof" is a historical accident of the 20st century (and it was never perfect, those with resources could always manipulate pictures to some extent). As a thought experiment, it might be interesting to imagine what happens when you "open up the dams" and are able to synthesize any image you can imagine! In the beginning, this will cause a lot of trouble (say with harrasment and fake news), but I believe society will adapt quickly. I think right now there is a real problem with the internet remembering too much (pervasive surveillance on the one hand, and constant risk of moral outrages for stupid things you did in your past). It would be an antidote if nobody could believe in any picture anymore.
- foxfired 5y agoThere are videos that show what they mean by "details glued to image coordinates" in StyleGAN2: https://nvlabs-fi-cdn.nvidia.com/stylegan3/videos/ https://nvlabs-fi-cdn.nvidia.com/stylegan3/videos/
- anon012012 5y agoI know HN doesn't like hype, but as an AI neophyte, I find this incredible. Nvidia is doing it again. This is likely going to help with 3D generation, the next cornerstone. Imagine that we are solving the problems so fast.
- maCDzP 5y agoIt really bums me out that they didn’t name it GANnamStyle.
- fuzzythinker 5y agoAnd it fits pretty good too. GAN Nvidia Alias-free Models Style
- mkl 5y agoThis produces a kind of artefact I haven't seen before, involving little chains of circles and diamonds, e.g. https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-r-ffhq-1024x1024/1805.png https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-..., https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-r-ffhq-1024x1024/0037.png https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-... (hair). I think they follow those glowing coordinate-ish lines from the internal representation. It also seems to have given some faces contact lenses! https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-r-ffhq-1024x1024/1863.png https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-..., https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-r-ffhq-1024x1024/0269.png https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-...
- opwieurposiu 5y agoAlso the some teeth are getting drawn in front of the lips and the pupils are not round. I think the not-round pupils make the eyes point two different directions, kinda unsettling. https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-r-ffhq-1024x1024/0075.png https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-... https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-r-ffhq-1024x1024/1128.png https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-...
- twofornone 5y agoI suspect because there's a lot of entropy in hair and because of the shape of the optimization function, (which might even have a spatial term) a regular pattern in such a noisy and hard to learn region falls into a local minimum while the rest of the image converges to the true minimum. There's a little meat left to optimize here, but you need to do it cleverly because there's no reason for a neural network to learn all the many combinations of hair pixels in this application. That could require as many parameters all the neurons involved in generating the faces, I'd bet.
- twofornone 5y agoThinking more about it, the shape of the solution space is sufficiently different for hair vs faces that any given combination of {optimization function, hyperparameters, training data} is unlikely to optimize for both. You probably need some other sort of special tuning, like a spatially local adaptive gradient for regions of hair.
- knownjorbist 5y agoEveryone is nitpicking the licenses involved in this thread. Is this the right thing to do?
- gjs278 5y agoon HN yes. in the real world nobody cares.
- RNCTX 5y ago> This material is based upon work supported by the US Defense Advanced Research Projects Agency (DARPA) under Contracts No.R00112030005, HR001120C0123, HR001120C0124 and FA8750-20-2-1004 and the Air Force Research Laboratory (AFRL) under Contract No. FA8750-20-2-1004. This is why AI is just a marketing term with no real future. There isn't room for corporations to profit from it out of the gate + into the future indefinitely. No one is going to pay an AWS tax to use their models on every single API hit forever. No one is going to pay nVidia a license fee to use their image recognition tools forever. If the creators of HTML, CSS, and Javascript wanted license fees we wouldn't be using them right now either. There are two groups of people, off the top of my head, who care about all of this: 1) The US Military, because the budget for their murder robots is theoretically infinite. 2) Google and Facebook because the budget for their spyware is theoretically infinite. To everyone else, it's much ado about nothing.
- Scaevolus 5y agoNvidia makes most of their money selling GPUs as computational accelerators-- both generic CUDA and neural network applications. They don't _need_ to profit off their ML models. It's a value added service. Ensuring their hardware has marketshare at the leading edge of the ecosystem is the main point.
- RNCTX 5y agoThey didn't profit off of ML models, they profited from the DARPA contract. See point above about murder robots.
- intricatedetail 5y agoIs minimum 12GB limit on purpose to make people buy new GPUs? It's sad that this growing area is becoming for privileged people only.
- sydthrowaway 5y agoWe need FOSS GPUs + compute!
- anaganisk 5y agoUnless you discover a novel process to make a chip, economics of semi conductor never can make sense in FOSS. You will end up paying way more, than what the current market price is.
- qayxc 5y agoIt's not on purpose but a direct consequence of the computational requirements of the model. I say this often and I cannot stress this enough: this is not magic! High quality results require a lot of computation and with it a lot of other resources like VRAM. This area of research has always been resource intensive and no one was surprised by this back in the 1990s when some research required hardware that wasn't available to mere mortals (looking at you, SGI Indigo and Octane). You should look at it be the other way around: it's astonishing that you can use off-the-shelf consumer hardware (here: mid-range gfx cards) to participate in and benefit from cutting edge research. Even better yet, services like Google CoLab even allow you to meddle with this for free if you don't own the required hardware. BTW, it's not the researchers' fault that the market is f'ed up again due to shortages and crypto mining. Otherwise you'd be able to buy a 12 GB card for around $330 (e.g. the RTX 3060), which doesn't sound particularly outrageous to me.