4 ms·

Please, correct me if I'm wrong, but isn't the whole machine learning testing approach based on the recognition the whole image at once in full resolution in on

Please, correct me if I'm wrong, but isn't the whole machine learning testing approach based on the recognition the whole image at once in full resolution in one attempt? If so, it's not how animals see and observe the world and this could be a key difference between AI and living beings. What if an algorithm actually had a focused sight? Look at the brightest feature, blur the rest, then try to find another distinctive details, look closer into the parts of the image until the most of it becomes clear. Let other neural network control the scanning process and loop until it develops some confidence in the result. Can it work that way?

sp332 8y ago
Yes, check out "attention" mechanisms. The network can use wide convolutions for the image as a whole, effectively mixing data kind of like a blur function, and also use finer convolutions for a small piece of the image that it chooses to focus on.
- ivan_gammel 8y ago
  Does the problem of elephants still exist with such approach? It doesn't seem right that an unexpected feature in a corner of the image would have a dramatic impact on the recognition of the features elsewhere.
  - sp332 8y ago
    Well if there's a thing/elephant/chair floating over someone's head in one part of the room, that does change the context for interpreting other things in the room.
  - CodesInChaos 8y ago
    I don't see any evidence of "dramatic" impact. The objects visible in both images only show minor changes in the confidence score . The objects missing in one image are close to the visualization threshold (50% confidence) in the other. And for the "chair turning into a couch", we don't see the likelihood of the chair, because it's hidden behind the box for the couch. So the dramatic changes seen in the image seem to be artifacts of the visualization and not of the network's performance.
ThePhysicist 8y ago
Yes but children can pass these tests when presented with a single photo as well (if I understand the setup correctly), so I'd say it's a fair comparison. The algorithm can "look" at the picture as long as it wants as well, e.g. using many different convolutional filters that can "see" features in different parts of the image and then combine these in later stages of the analysis.
- ivan_gammel 8y ago
  How close are those convolutional filters to the physical sight of animals? Were they designed specifically to mimic nature or they are artificial concept with similar purpose?
  - singularity2001 8y ago
    First they were designed to mimic nature, then these patterns (gabor filters) astonishingly automatically developed themselves via backpropagation (which gave rise to the recent enthusiasm). Modern networks are instantiated with random noise but end up pretty much the same as previously hardcoded filters.
  - ThePhysicist 8y ago
    Actually the artificial neural networks used for vision tasks are not so different from our visual cortex, there's an interesting Nature paper on this: https://www.nature.com/articles/s42003-018-0110-y https://www.nature.com/articles/s42003-018-0110-y Of course the neurons in our brain work in a very different way, but at an architecture level the strategies of natural and artificial neural networks seem to match.
    - ivan_gammel 8y ago
      Thank you for the link! Indeed, that's very interesting reading.
  - TeMPOraL 8y ago
    They may be close to first stages of sight pipeline. Maybe even close to full pipe for some simple animals. For humans, they're pretty much input processing stage. Human vision (and presumably animal vision too) has runtime feedback that uses the the output of the vision process to fix up the input. That's what makes you suddenly see things if someone tells you what to look for, or suddenly unsee things. That's why you didn't notice I repeated the the word "the" in few places in this comment. As the intro to the the article says (you now noticed, didn't you?), the way DNN vision systems are different is that "unlike humans, they can’t do a double take". And those "double takes" is what our vision pipeline does all the time.
gok 8y ago
This is what the article says but it's not really correct. Modern CV systems perform "segmenting" before identification.

sp332 8y ago

Yes, check out "attention" mechanisms. The network can use wide convolutions for the image as a whole, effectively mixing data kind of like a blur function, and also use finer convolutions for a small piece of the image that it chooses to focus on.

ivan_gammel 8y ago
Does the problem of elephants still exist with such approach? It doesn't seem right that an unexpected feature in a corner of the image would have a dramatic impact on the recognition of the features elsewhere.
- sp332 8y ago
  Well if there's a thing/elephant/chair floating over someone's head in one part of the room, that does change the context for interpreting other things in the room.
- CodesInChaos 8y ago
  I don't see any evidence of "dramatic" impact. The objects visible in both images only show minor changes in the confidence score . The objects missing in one image are close to the visualization threshold (50% confidence) in the other. And for the "chair turning into a couch", we don't see the likelihood of the chair, because it's hidden behind the box for the couch. So the dramatic changes seen in the image seem to be artifacts of the visualization and not of the network's performance.

ThePhysicist 8y ago

Yes but children can pass these tests when presented with a single photo as well (if I understand the setup correctly), so I'd say it's a fair comparison. The algorithm can "look" at the picture as long as it wants as well, e.g. using many different convolutional filters that can "see" features in different parts of the image and then combine these in later stages of the analysis.

ivan_gammel 8y ago
How close are those convolutional filters to the physical sight of animals? Were they designed specifically to mimic nature or they are artificial concept with similar purpose?
- singularity2001 8y ago
  First they were designed to mimic nature, then these patterns (gabor filters) astonishingly automatically developed themselves via backpropagation (which gave rise to the recent enthusiasm). Modern networks are instantiated with random noise but end up pretty much the same as previously hardcoded filters.
- ThePhysicist 8y ago
  Actually the artificial neural networks used for vision tasks are not so different from our visual cortex, there's an interesting Nature paper on this: https://www.nature.com/articles/s42003-018-0110-y https://www.nature.com/articles/s42003-018-0110-y Of course the neurons in our brain work in a very different way, but at an architecture level the strategies of natural and artificial neural networks seem to match.
  - ivan_gammel 8y ago
    Thank you for the link! Indeed, that's very interesting reading.
- TeMPOraL 8y ago
  They may be close to first stages of sight pipeline. Maybe even close to full pipe for some simple animals. For humans, they're pretty much input processing stage. Human vision (and presumably animal vision too) has runtime feedback that uses the the output of the vision process to fix up the input. That's what makes you suddenly see things if someone tells you what to look for, or suddenly unsee things. That's why you didn't notice I repeated the the word "the" in few places in this comment. As the intro to the the article says (you now noticed, didn't you?), the way DNN vision systems are different is that "unlike humans, they can’t do a double take". And those "double takes" is what our vision pipeline does all the time.

gok 8y ago

This is what the article says but it's not really correct. Modern CV systems perform "segmenting" before identification.