4 ms·

Hi, author here. To hopefully clarify, our work is in the context of representation learning, which is a bit different from a "standard" classification. For ex

Hi, author here. To hopefully clarify, our work is in the context of representation learning, which is a bit different from a "standard" classification. For example, to classify a hotdog it might be useful to first generate an intermediate representation of the image (think "cylindrical, brown, meaty thing"). Such a representation can then fairly easily be mapped to the concept "hot dog". These representations can be learned from large image datasets alone (they do not require labels!). In our work we show that you don't even need real images, but that images that are generated from noise processes are enough to train such representations, and that these representations are surprisingly good for classification. Hope this clarifies things a bit, and happy to answer any other questions!

aaronblohowiak 5y ago
The part you lost me on is noise processes - what goes in to the noise and how does it help if it is random?
- deleted 5y ago
  [deleted]
- Animats 5y ago
  It's not random noise. Look at the images in the paper. Horizontal lines, vertical lines, snakeskin patterns, Minecraft textures. Examples of miscellaneous surface patterns, in other words. Back before deep learning, people used to make recognizers for features like that as a lower level of feature recognition. Now it's expected that features will be derived automatically from real imagery. This is kind of a return to that level. A useful training set might be a big texture library used for game development or animation. Those are easily available.
  - jswulff 5y ago
    That would indeed be an interesting thing to try, use real data, but only in terms of textures - so effects like occlusions, perspective, etc. would not be present. I would expect it to be somewhere in the ballpark of our StyleGAN images, which also look very "textural", but lack these effects that are an result of imaging the 3D world. Interestingly, modelling these effects without realistic textures seems to result in worse performance - this is for example the case for images taken from CLEVR or generated from Minecraft, and both perform worse than the StyleGAN images.
- jswulff 5y ago
  One thing to note is that here noise != Gaussian iid noise, so these are not typical white noise images. I think we were not really clear on that part, but for us noise is basically a random process, which takes a seed as input (plus potentially some very low-level assumptions over image statistics, such as a 1/f spectrum) and produces a synthetic image. It is then possible to generate arbitrary amounts of these images as samples from the stochastic process - these images exhibit certain image-like structures (such as oriented edges), but are as a whole still random and extremely varied, which is good and necessary for the representation learning. In terms of helping, though, it is important to note that we do not achieve state-of-the-art performance yet, and when looking at absolute performance for a task like image classification, using real images is still better. That being said, something that is in the paper but generally seems to get lost is that our representations work very well when analyzing data that is very different from normal images, such as medical images or satellite images.
  - a_e_k 5y ago
    It could be very interesting to try Gabor noise.
    - locuscoeruleus 5y ago
      Why would that be interesting?
  - vanderZwan 5y ago
    But then you aren't really throwing "random noise" at it are you? It's more like you are throwing generated data sets with abstract structures at it, and use the randomization part to ensure that it does not overfit on other accidental structures that might be in an individual image, because the randomization ensures that there are no other structures to speak of in the "average" (which does sound like a very sensible way to train a network on abstract structures). Or do I misunderstand the method here?
    - l33tman 5y ago
      I'm pretty sure the title is clickbait, it's not uncorrelated random per-pixel noise as it gives the impression of.
sorenjan 5y ago
Is this basically teaching the network how to do pattern processing, like line and corner detection, and then using that trained network as a starting point when training on real images?
ravel-bar-foo 5y ago
This is a fascinating paper. Does training on generated noisy images make the resultant classifier more resistant to standard adversarial examples?
YeGoblynQueenne 5y ago
Hi, out of curiosity, what is the expected performance of a random classifier on your test datasets?

aaronblohowiak 5y ago

The part you lost me on is noise processes - what goes in to the noise and how does it help if it is random?

deleted 5y ago
[deleted]
Animats 5y ago
It's not random noise. Look at the images in the paper. Horizontal lines, vertical lines, snakeskin patterns, Minecraft textures. Examples of miscellaneous surface patterns, in other words. Back before deep learning, people used to make recognizers for features like that as a lower level of feature recognition. Now it's expected that features will be derived automatically from real imagery. This is kind of a return to that level. A useful training set might be a big texture library used for game development or animation. Those are easily available.
- jswulff 5y ago
  That would indeed be an interesting thing to try, use real data, but only in terms of textures - so effects like occlusions, perspective, etc. would not be present. I would expect it to be somewhere in the ballpark of our StyleGAN images, which also look very "textural", but lack these effects that are an result of imaging the 3D world. Interestingly, modelling these effects without realistic textures seems to result in worse performance - this is for example the case for images taken from CLEVR or generated from Minecraft, and both perform worse than the StyleGAN images.
jswulff 5y ago
One thing to note is that here noise != Gaussian iid noise, so these are not typical white noise images. I think we were not really clear on that part, but for us noise is basically a random process, which takes a seed as input (plus potentially some very low-level assumptions over image statistics, such as a 1/f spectrum) and produces a synthetic image. It is then possible to generate arbitrary amounts of these images as samples from the stochastic process - these images exhibit certain image-like structures (such as oriented edges), but are as a whole still random and extremely varied, which is good and necessary for the representation learning. In terms of helping, though, it is important to note that we do not achieve state-of-the-art performance yet, and when looking at absolute performance for a task like image classification, using real images is still better. That being said, something that is in the paper but generally seems to get lost is that our representations work very well when analyzing data that is very different from normal images, such as medical images or satellite images.
- a_e_k 5y ago
  It could be very interesting to try Gabor noise.
  - locuscoeruleus 5y ago
    Why would that be interesting?
- vanderZwan 5y ago
  But then you aren't really throwing "random noise" at it are you? It's more like you are throwing generated data sets with abstract structures at it, and use the randomization part to ensure that it does not overfit on other accidental structures that might be in an individual image, because the randomization ensures that there are no other structures to speak of in the "average" (which does sound like a very sensible way to train a network on abstract structures). Or do I misunderstand the method here?
  - l33tman 5y ago
    I'm pretty sure the title is clickbait, it's not uncorrelated random per-pixel noise as it gives the impression of.

sorenjan 5y ago

Is this basically teaching the network how to do pattern processing, like line and corner detection, and then using that trained network as a starting point when training on real images?

ravel-bar-foo 5y ago

This is a fascinating paper. Does training on generated noisy images make the resultant classifier more resistant to standard adversarial examples?

YeGoblynQueenne 5y ago

Hi, out of curiosity, what is the expected performance of a random classifier on your test datasets?