3 ms·

PIRL: Learn Image Representations Immune to Geometric Transformation

50 points by amitness 7y ago

alisterburt 7y ago
This looks a lot to me like learned image descriptors from the computer vision community. Some examples are Discriminative learning of local image descriptors (http://matthewalunbrown.com/papers/pami2010.pdf http://matthewalunbrown.com/papers/pami2010.pdf) DeepDesc (https://icwww.epfl.ch/~trulls/pdf/iccv-2015-deepdesc.pdf https://icwww.epfl.ch/~trulls/pdf/iccv-2015-deepdesc.pdf) L2-net (http://www.nlpr.ia.ac.cn/fanbin/pub/L2-Net_CVPR17.pdf http://www.nlpr.ia.ac.cn/fanbin/pub/L2-Net_CVPR17.pdf)
im3w1l 7y ago
The issue with this is that you (most of the time) don't want your image representations to be immune to geometric transformation. A rotated p should be recognized as a d. In nature pictures you want to recognize blue at the top as sky and blue at the bottom as water.
- bobbylarrybobby 7y ago
  I think that's taking a dim view of what machine vision should be capable of. If a picture is rotated 180 degrees, upside down text should be recognized as its flipped version, i.e., the word "6op" should be read as "dog" having been rotated. Similarly a machine should be able to tell if it's looking at the reflection of the sky in a lake. And disregarding entire scenes, any object upside down (upside down chair, upside down dog, upside down car, ...) is clearly recognizable as such to a human, and hence should be to a machine as well.
  - im3w1l 7y ago
    Upside down objects and scenes look alien to us. We can recognize them but not without effort. It's a combination of having seen objects upside down before and being able to reason about what we are seeing. It does not come for free from invariance.
    - dr_zoidberg 7y ago
      The classic Thatcher illusion[0]. [0] https://en.wikipedia.org/wiki/Thatcher_effect https://en.wikipedia.org/wiki/Thatcher_effect Edited for clarity, started writing then wiped half and didn't check first half.
fxtentacle 7y ago
Thank you @amitness for this wonderful website with graphical explanations :) The PIRL technique in question here seems useless to me, because its loss deliberately trains it to behave differently from what a human would do. But that overview page https://amitness.com/2020/02/illustrated-self-supervised-learning/ https://amitness.com/2020/02/illustrated-self-supervised-lea... is gold.

50 points by amitness 7y ago

This looks a lot to me like learned image descriptors from the computer vision community. Some examples are Discriminative learning of local image descriptors (http://matthewalunbrown.com/papers/pami2010.pdf http://matthewalunbrown.com/papers/pami2010.pdf) DeepDesc (https://icwww.epfl.ch/~trulls/pdf/iccv-2015-deepdesc.pdf https://icwww.epfl.ch/~trulls/pdf/iccv-2015-deepdesc.pdf) L2-net (http://www.nlpr.ia.ac.cn/fanbin/pub/L2-Net_CVPR17.pdf http://www.nlpr.ia.ac.cn/fanbin/pub/L2-Net_CVPR17.pdf)

im3w1l 7y ago

The issue with this is that you (most of the time) don't want your image representations to be immune to geometric transformation. A rotated p should be recognized as a d. In nature pictures you want to recognize blue at the top as sky and blue at the bottom as water.

bobbylarrybobby 7y ago
I think that's taking a dim view of what machine vision should be capable of. If a picture is rotated 180 degrees, upside down text should be recognized as its flipped version, i.e., the word "6op" should be read as "dog" having been rotated. Similarly a machine should be able to tell if it's looking at the reflection of the sky in a lake. And disregarding entire scenes, any object upside down (upside down chair, upside down dog, upside down car, ...) is clearly recognizable as such to a human, and hence should be to a machine as well.
- im3w1l 7y ago
  Upside down objects and scenes look alien to us. We can recognize them but not without effort. It's a combination of having seen objects upside down before and being able to reason about what we are seeing. It does not come for free from invariance.
  - dr_zoidberg 7y ago
    The classic Thatcher illusion[0]. [0] https://en.wikipedia.org/wiki/Thatcher_effect https://en.wikipedia.org/wiki/Thatcher_effect Edited for clarity, started writing then wiped half and didn't check first half.

fxtentacle 7y ago

Thank you @amitness for this wonderful website with graphical explanations :) The PIRL technique in question here seems useless to me, because its loss deliberately trains it to behave differently from what a human would do. But that overview page https://amitness.com/2020/02/illustrated-self-supervised-learning/ https://amitness.com/2020/02/illustrated-self-supervised-lea... is gold.