6 ms·
The relativistic discriminator: a key element missing from standard GAN
- Y_Y 8y agoThis is just confusing misuse of the word "relativistic", being based as it is on _relative_ probability and having little to do with philosophical relativism and nothing to do with physics relativity.
- drewm1980 8y agoMany of the words used by the machine learning community are gratuitously unrelated to what they mean in other fields. If you hear a ML person use a term and you don't know the ML meaning, you probably need to ask for clarification.
- lrei 8y agoThis is true of every field of science and even social science I’ve known. Also the same terms have different meaning in different fields. Also the standard use of a term within a subfield can be different from its usage in another subfield. There are even variations in usage within communities of the same subfield (e.g. inside different ML communities).
- deleted 8y ago[deleted]
- jmmcd 8y agoI agree -- though I'm not sure how to make the right name for this technique. Maybe "relative-probability GAN" is the best. I also wondered before clicking what a relativistic GAN would be: maybe as the activity of a neuron becomes larger and larger, it becomes harder and harder for it to continue? But that's already true of sigmoidal activation functions.
- SubiculumCode 8y agoI was excited to see how the fabric of space and time was being used to generate adversarial examples. =)
- SubiculumCode 8y agoI almost doubt there exists a single noun in the English language that is not yet the name of some git package, platform, programming language, or text-editor.
- AlexiaJM 8y ago"Relative probability" is only correct for standard GAN, not for other GANs which don't estimate a probability. My first name was "Critic's difference" since it's literally C(x_r)-C(x_f) so its the difference in critics, but it felt really unclear, it doesn't give the reader any sense of what it is about. Relativism/Relativistic is better since it's to say that it really doesn't matter if the data looks real, what matters if how realistic is real data relative to fake data (and vice versa). The frame of reference is important here.
- radarsat1 8y agoIt's not even probability in standard GAN is it? Since you are taking the difference before the sigmoid. It can't really be interpreted as probability until it the range is clamped. Critic's difference or Critical Difference would be a better term perhaps.
- sophistication 8y agoIt's logits or log odds or the logarithm of the odds, i.e. log(odds(p)) = log(p / (1 - p)): https://en.wikipedia.org/wiki/Logit https://en.wikipedia.org/wiki/Logit Since sigmoid(C(x_r)) = p(x_r is real) and C(x_f) = p(x_f is real), the sigmoid of the difference expresses some probability that that x_r looks more real than x_f or vice versa (depending on whether it is C(x_r) - C(x_f) or C(x_f) - C(x_r)). Not sure whether there is a probabilistic interpretation of the difference, but it looks so simple that there maybe is one. I couldn't find one in the paper.
- sophistication 8y agoCorrection: sigmoid(C(x_f)) = p(x_f is real)
- AstralStorm 8y agoIt is not quite relative but regularised by another GAN...
- AlexiaJM 8y agoIt's from same GAN though, C(x_r) and C(x_f) come from the same neural network. It's how realistic real data is compared to fake data (and vice-versa) as determined by C (discriminator without activation function).
- SpaceManNabs 8y agoSame... I was expecting a perspective on GANs inspired by differential geometry or something.
- xstephen95x 8y agowhat 'computer scientist' uses wordpress for their blog. Though, I don't have a blog, so...
- tzs 8y agoDick Lipton and Ken Regan, for example: https://rjlipton.wordpress.com/ https://rjlipton.wordpress.com/ Scott Aaronson: https://www.scottaaronson.com/blog/ https://www.scottaaronson.com/blog/ Jeremy Kun: https://jeremykun.com/ https://jeremykun.com/ Also, although he's a mathematician rather than a computer scientist, there is Terry Tao: https://terrytao.wordpress.com/ https://terrytao.wordpress.com/
- anonfunction 8y agoWonder what he thinks (not really) of Rob Pike using blogspot: https://commandcenter.blogspot.com/ https://commandcenter.blogspot.com/
- SubiculumCode 8y agoBusy people are busy, and choosing a 'better' platform or rolling your own site would be a monumental waste of their time.
- SamPutnam 8y agoIf you do get one at some time, and find you enjoy Wordpress, I warmly recommend WP Engine.
- jakoblorz 8y agoI don't see any sense in attacking the credibility by the choice of the blogging infrastructure - the author wanted to convey a message, present his findings and he achieved that. This should be the only thing that counts. I guess the best (even if in a totally opposite field) argument against opinionated choice of web-technologies would be the website of Berkshire Hathaway [1]. They invest in highly sophisticated companies but still use the website that always provided the service they demanded. [1] http://www.berkshirehathaway.com http://www.berkshirehathaway.com
- anonfunction 8y agoSame guy who wrote this has another post called Deep learning with cats[1] which has a great open source repo[2] with multiple GANs. 1. https://ajolicoeur.wordpress.com/cats/ https://ajolicoeur.wordpress.com/cats/ 2. https://github.com/AlexiaJM/Deep-learning-with-cats https://github.com/AlexiaJM/Deep-learning-with-cats
- shrubhub 8y agoSame guy? Try... Not a guy!
- manux 8y agoYeah, this person clearly is from Québec, and here "Alexia" is a woman's name.
- anonfunction 8y agoThanks for correcting me! I commonly call a group of people "guys" even if they are women, this is something I need to break the habit of. Will strive to be gender neutral in the future.
- DanielleMolloy 8y agoInteresting, especially the performance on bigger images, and it looks like a low-effort modification of many standard GAN losses. Seriously, I want to give this a try right now. What do GAN researchers think about this paper? It should also be appreciated that it comes with code and a short blog post.
- radarsat1 8y agoAnd a 24-page paper ;)
- rmgraham 8y agoGAN, Generative Adversarial Network: https://en.wikipedia.org/wiki/Generative_adversarial_network https://en.wikipedia.org/wiki/Generative_adversarial_network
- hyprCoin 8y agoThis is mentioned in the adversarial learned inference paper https://arxiv.org/pdf/1606.00704.pdf https://arxiv.org/pdf/1606.00704.pdf (see Lg in algorithm 1)
- dang 8y agoSince the url is the same we changed the title to the one the author had for https://news.ycombinator.com/item?id=17449143 https://news.ycombinator.com/item?id=17449143.