5 ms·

When the output switches to rabbit the picture actually resembles a rabbit. I am unsure if this experiment was supposed to be a “haha look how stupid AI is” ty

When the output switches to rabbit the picture actually resembles a rabbit. I am unsure if this experiment was supposed to be a “haha look how stupid AI is” type thing or not, but it seems like the cloud vision api is performing as intended.

mikejb 8y ago
The interesting part (IMO) isn't that the AI classifies it as both a rabbit and a duck, but that the classification is dependent on the rotation of the picture. I can somewhat understand how that happens, but I find it's in an interesting observation (rather than a criticism on the system, though the title is somewhat unclear and had me expect something else).
- tanilama 8y ago
  But rotation does contain information. 6 and 9 can be considered as a case where rotation SHOULD change the classifier's output. AI does makes dumb predictions from time to time, but I my opinion, this isn't that strong a case. When it rotate upside down, it does look like a rabbit even to me. The more interesting 'failure' here to me, is that while the rotation is smooth, the prediction is not, instead it is flickering, which does raise some interesting questions about what the model's internal distribution surface looks like.
  - mikejb 8y ago
    > AI does makes dumb predictions from time to time, but I my opinion, this isn't that strong a case. I don't think it's meant as an example of a 'dumb prediction'. I think it's an interesting example that inspires (me at least) to think about how this recognition and classification works, and what could cause this effect.
  - devin 8y ago
    Thank you for this comment. To me, duck v. rabbit on rotation just proves the system works as intended.
    - piva00 8y ago
      Exactly, I also see a duck and a rabbit depending on rotation.
  - lifthrasiir 8y ago
    In the reddit comment the author noted that the training set is frequently rotated to help the system generalize better (maybe that's why people is fascinated by this animation at the first place). Possible explanations thus include: the Vision API is not well trained in this regard, or it has (correctly) learned that some images are sensitive to the rotation.
  - salty_biscuits 8y ago
    But only with context right? It is completely ambiguous if the character is 6 or 9 without some other clue, like which way up the paper is, or from the 4 next to it (if you assume that an arbitrary rotation may have occurred). It is just a sign to me that doing some rotations as data augmentation is not good enough. Rotation invariance needs to be built into the architecture of the network (like translation invariance sort of gets in via max pooling). I think it should be giving a 50/50 classification of duck and rabbit at all rotations if it was working as expected.
    - devin 8y ago
      How often do you read upside down?
      - salty_biscuits 8y ago
        All the time, especially when I read to my kids.
        swixmix 8y ago
        Your kids might enjoy this cartoon. https://youtu.be/9-k5J4RxQdE https://youtu.be/9-k5J4RxQdE
      - ozzyman700 8y ago
        every meeting
    - FakeComments 8y ago
      > I think it should be giving a 50/50 classification of duck and rabbit at all rotations if it was working as expected. This would likely give incorrect or worse results when, eg, classifying ducks and rabbits in wildlife photos — animals come in orientations, and you’ll do a better job classifying them in practice if you respect that. It’s also not the case that a human would classify it 50/50 in all rotations — it certainly looks more or less like one or the other as you rotate it. Humans are even program,Ed for that compromise: we see faces in one orientation much better than rotated 180 degrees.
- Sharlin 8y ago
  The distinction is present in the training data. So why wouldn't the classifier use the information? A completely rotation-invariant algorithm would most likely be more difficult to create, and would generate worse classifications because orientation is meaningful information.
gbersac 8y ago
Since human would do the same error, I think it's kind of a praise of how close to human is the AI.
- Sharlin 8y ago
  There's no error. The image is simply ambiguous. Depending on orientation our first interpretation is also either a duck or a rabbit, because our vision is obviously biased to interpret things in the orientation they are most likely to occur based on our priors that have evolved in the presence of "up" and "down" directions. The AI correctly takes the orientation into account because it also matters in its priors, having been fed training data that captures those human priors. Now, most computer vision algorithms do strive for some degree of rotation (and translation etc) invariance, because a classifier that gets confused when you rotate the input by 15 degrees or whatever isn't very useful in the real world. But complete rotation invariance would just be a case of Artificial Stupidity in an application that attempts to classify like a human would. Input orientation is meaningful information.
SiempreViernes 8y ago
I think this shows how poorly many neural networks are at handling ambiguity. The picture is constructed to be ambiguous, and this property is preserved by the rotation: you can still easily see the duck by slightly shifting where you focus. One mode might be more prominent at some orientation, but the ambiguity is always there so to confidently assign labels and then switch what you assign is an error. So you should be constantly switching as the two classes end up with very similar scores and the noise decides. Either that, or you decide one label is the most appropriate and then it correctly handle the trivial rotation.
- FakeComments 8y ago
  The neural network is likely handling it just fine: A classifier generally outputs a vector of weights, so it’s likely classifying, say [0.8, 0.75] and then the output is selecting the highest and saying “bunny”. Then you rotate it, and the classifier says [0.75, 0.8] and the output says “duck”. This is completely reasonable on the part of the network: all things being equal, animals generally appear in certain orientations and we should prefer the interpretation of the amigbuity which respects this alignment, slightly. Example: “bill” down, it looks more like a duck because rabbits rarely have their head in that alignment, while “ears up” it looks more like a rabbit since ducks rarely hold their bills that way. The problem is actually in how we represent probabilistic information to humans, aka “why the weather man is always wrong”, so it seems like the classifier is randomly flapping when it’s actually perfectly correctly adjusting its distribution of answers based on information we give it.
  - mannykannot 8y ago
    It works that way for me when I play it. To see this, it seems to help to look at the picture but have half an eye on the changing NN evaluation. I don't think my perception is simply responding to the changing NN output, as there are times when I disagree with it. My guess is that, by dividing my attention, I reevaluate the picture more frequently.
  - twanvl 8y ago
    > A classifier generally outputs a vector of weights, so it’s likely classifying, say [0.8, 0.75] and then the output is selecting the highest and saying “bunny”. Then you rotate it, and the classifier says [0.75, 0.8] and the output says “duck”. The original post actually included the predicted probabilities, which are around 80% for duck or rabbit and 0% for the other class. So the neural network really is overconfident.

mikejb 8y ago

The interesting part (IMO) isn't that the AI classifies it as both a rabbit and a duck, but that the classification is dependent on the rotation of the picture. I can somewhat understand how that happens, but I find it's in an interesting observation (rather than a criticism on the system, though the title is somewhat unclear and had me expect something else).

tanilama 8y ago
But rotation does contain information. 6 and 9 can be considered as a case where rotation SHOULD change the classifier's output. AI does makes dumb predictions from time to time, but I my opinion, this isn't that strong a case. When it rotate upside down, it does look like a rabbit even to me. The more interesting 'failure' here to me, is that while the rotation is smooth, the prediction is not, instead it is flickering, which does raise some interesting questions about what the model's internal distribution surface looks like.
- mikejb 8y ago
  > AI does makes dumb predictions from time to time, but I my opinion, this isn't that strong a case. I don't think it's meant as an example of a 'dumb prediction'. I think it's an interesting example that inspires (me at least) to think about how this recognition and classification works, and what could cause this effect.
- devin 8y ago
  Thank you for this comment. To me, duck v. rabbit on rotation just proves the system works as intended.
  - piva00 8y ago
    Exactly, I also see a duck and a rabbit depending on rotation.
- lifthrasiir 8y ago
  In the reddit comment the author noted that the training set is frequently rotated to help the system generalize better (maybe that's why people is fascinated by this animation at the first place). Possible explanations thus include: the Vision API is not well trained in this regard, or it has (correctly) learned that some images are sensitive to the rotation.
- salty_biscuits 8y ago
  But only with context right? It is completely ambiguous if the character is 6 or 9 without some other clue, like which way up the paper is, or from the 4 next to it (if you assume that an arbitrary rotation may have occurred). It is just a sign to me that doing some rotations as data augmentation is not good enough. Rotation invariance needs to be built into the architecture of the network (like translation invariance sort of gets in via max pooling). I think it should be giving a 50/50 classification of duck and rabbit at all rotations if it was working as expected.
  - devin 8y ago
    How often do you read upside down?
    - salty_biscuits 8y ago
      All the time, especially when I read to my kids.
      - swixmix 8y ago
        Your kids might enjoy this cartoon. https://youtu.be/9-k5J4RxQdE https://youtu.be/9-k5J4RxQdE
    - ozzyman700 8y ago
      every meeting
  - FakeComments 8y ago
    > I think it should be giving a 50/50 classification of duck and rabbit at all rotations if it was working as expected. This would likely give incorrect or worse results when, eg, classifying ducks and rabbits in wildlife photos — animals come in orientations, and you’ll do a better job classifying them in practice if you respect that. It’s also not the case that a human would classify it 50/50 in all rotations — it certainly looks more or less like one or the other as you rotate it. Humans are even program,Ed for that compromise: we see faces in one orientation much better than rotated 180 degrees.
Sharlin 8y ago
The distinction is present in the training data. So why wouldn't the classifier use the information? A completely rotation-invariant algorithm would most likely be more difficult to create, and would generate worse classifications because orientation is meaningful information.

gbersac 8y ago

Since human would do the same error, I think it's kind of a praise of how close to human is the AI.

Sharlin 8y ago
There's no error. The image is simply ambiguous. Depending on orientation our first interpretation is also either a duck or a rabbit, because our vision is obviously biased to interpret things in the orientation they are most likely to occur based on our priors that have evolved in the presence of "up" and "down" directions. The AI correctly takes the orientation into account because it also matters in its priors, having been fed training data that captures those human priors. Now, most computer vision algorithms do strive for some degree of rotation (and translation etc) invariance, because a classifier that gets confused when you rotate the input by 15 degrees or whatever isn't very useful in the real world. But complete rotation invariance would just be a case of Artificial Stupidity in an application that attempts to classify like a human would. Input orientation is meaningful information.

SiempreViernes 8y ago

I think this shows how poorly many neural networks are at handling ambiguity. The picture is constructed to be ambiguous, and this property is preserved by the rotation: you can still easily see the duck by slightly shifting where you focus. One mode might be more prominent at some orientation, but the ambiguity is always there so to confidently assign labels and then switch what you assign is an error. So you should be constantly switching as the two classes end up with very similar scores and the noise decides. Either that, or you decide one label is the most appropriate and then it correctly handle the trivial rotation.

FakeComments 8y ago
The neural network is likely handling it just fine: A classifier generally outputs a vector of weights, so it’s likely classifying, say [0.8, 0.75] and then the output is selecting the highest and saying “bunny”. Then you rotate it, and the classifier says [0.75, 0.8] and the output says “duck”. This is completely reasonable on the part of the network: all things being equal, animals generally appear in certain orientations and we should prefer the interpretation of the amigbuity which respects this alignment, slightly. Example: “bill” down, it looks more like a duck because rabbits rarely have their head in that alignment, while “ears up” it looks more like a rabbit since ducks rarely hold their bills that way. The problem is actually in how we represent probabilistic information to humans, aka “why the weather man is always wrong”, so it seems like the classifier is randomly flapping when it’s actually perfectly correctly adjusting its distribution of answers based on information we give it.
- mannykannot 8y ago
  It works that way for me when I play it. To see this, it seems to help to look at the picture but have half an eye on the changing NN evaluation. I don't think my perception is simply responding to the changing NN output, as there are times when I disagree with it. My guess is that, by dividing my attention, I reevaluate the picture more frequently.
- twanvl 8y ago
  > A classifier generally outputs a vector of weights, so it’s likely classifying, say [0.8, 0.75] and then the output is selecting the highest and saying “bunny”. Then you rotate it, and the classifier says [0.75, 0.8] and the output says “duck”. The original post actually included the predicted probabilities, which are around 80% for duck or rabbit and 0% for the other class. So the neural network really is overconfident.