7 ms·

Let me remind you of sofas being classified as cats[0] and people being classified as gorillas[1]. You're overestimating the guesswork convnets are able to do,

Let me remind you of sofas being classified as cats[0] and people being classified as gorillas[1]. You're overestimating the guesswork convnets are able to do, based on fragile training (which is still better guesswork than what previous models did). [0] http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.html http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.ht... [1] https://www.theverge.com/2015/7/1/8880363/google-apologizes-photos-app-tags-two-black-people-gorillas https://www.theverge.com/2015/7/1/8880363/google-apologizes-...

yorwba 9y ago
People being classified as gorillas was actually what I was thinking about regarding the lynx/cat example. The model might have been unsure about the kind of ape it was looking at, but clustering them together is its own kind of achievement.
- dr_zoidberg 9y ago
  The thing is that the convnets are unable to learn about "macro structures" (or structures in general). A cat has ~4 legs, a tail and pointy ears. Gorillas are black, have a primate-y face and fur. The sofa is lacking the tail, head and pointy ears. People were missing the fur. Yet those things did not prevent the net from missclassffication (because those features weren't detected in the learning phase). Once again, children are able to see a cat and extract all that relevant information: four legs, head, tail, eyes & nose & ears with a particular shape, different than dogs, most cats fur (except for those alien-looking furless cats, of course).
  - nkoren 9y ago
    If you ask a child to draw a hand, they will almost always draw it with five fingers stuck straight out, widely separated. This is a view of a hand that one almost never actually sees; generally you'll have fingers clustered together, occluding each other, foreshortened, etc. So why do they draw it like that? It's because they're drawing a conceptual, represntational model of a hand, not a distilation of visual "hand" characteristics. That's the difference with human learning: it's based on representational model-making, which is not at all the same thing as pattern matching.
    - goldenkey 9y ago
      The representation is the distillation of pattern matching. They are isomorphic.
      - nkoren 9y ago
        Is that a belief or a fact? I believe, but cannot prove, that symbolic representation is not isomorphic to pattern matching. If you reverse the output of a CNN "hand" classification, it'll give you images that resemble the geometry and shading of fingers, palms, nails, knuckles, etc. -- these, I submit, are the distillation pattern matching for the actuality of "hands". Under no circumstances will it give you the five widely-separated fingers which a child draws. That's because the child-drawn hand is based on literal visual stimuli, but rather on an abstract logical model of a hand. That logical model is fully integrated with a similarly abstract model of the world, and includes functional relationships between abstractions, like the knowledge that "hands" can open "jars". The value of these being logical models rather than matched patterns is that they can then be extended to include never-before-seen objects. Confronted with a strange but roughly jar-sized object, a child can surmise that maybe it, too, can be opened with hands. That isn't pattern-matching: it's algebra.
        yorwba 9y ago
        If you look at network activations in higher levels of a convolutional neural network, you will find some units that are activated by the individual parts of the hand. (There is a video out there where they show that for parts of the face, I'll be looking for a link.) Those activations are a representation of the matched patterns at a similar level of abstraction as the compositional model humans might come up with. They are not exactly the same as what a child would draw, but mostly because the neural network isn't trained to draw with hands. With a bit of work, I'm sure a bunch of PhDs could make a neural network model generate child-like drawings from realistic images.
        goldenkey 9y ago
        Algebra is pattern matching of a set of operation rules with regard to a space. Your jar example just extends the domain to physicality. And I agree - until these sorts of learning mechanisms have a wide range of quality realisms to pattern match from - they will not be able to form the type of cross visual/physical knowledge that is a much deeper and abstract undestanding of reality. But don't fool yourself. Humans and well..life..are just input output machines with incredible pattern matching capabilities. Algebraic representations are the structural result of that pattern matching.

yorwba 9y ago

People being classified as gorillas was actually what I was thinking about regarding the lynx/cat example. The model might have been unsure about the kind of ape it was looking at, but clustering them together is its own kind of achievement.

dr_zoidberg 9y ago
The thing is that the convnets are unable to learn about "macro structures" (or structures in general). A cat has ~4 legs, a tail and pointy ears. Gorillas are black, have a primate-y face and fur. The sofa is lacking the tail, head and pointy ears. People were missing the fur. Yet those things did not prevent the net from missclassffication (because those features weren't detected in the learning phase). Once again, children are able to see a cat and extract all that relevant information: four legs, head, tail, eyes & nose & ears with a particular shape, different than dogs, most cats fur (except for those alien-looking furless cats, of course).
- nkoren 9y ago
  If you ask a child to draw a hand, they will almost always draw it with five fingers stuck straight out, widely separated. This is a view of a hand that one almost never actually sees; generally you'll have fingers clustered together, occluding each other, foreshortened, etc. So why do they draw it like that? It's because they're drawing a conceptual, represntational model of a hand, not a distilation of visual "hand" characteristics. That's the difference with human learning: it's based on representational model-making, which is not at all the same thing as pattern matching.
  - goldenkey 9y ago
    The representation is the distillation of pattern matching. They are isomorphic.
    - nkoren 9y ago
      Is that a belief or a fact? I believe, but cannot prove, that symbolic representation is not isomorphic to pattern matching. If you reverse the output of a CNN "hand" classification, it'll give you images that resemble the geometry and shading of fingers, palms, nails, knuckles, etc. -- these, I submit, are the distillation pattern matching for the actuality of "hands". Under no circumstances will it give you the five widely-separated fingers which a child draws. That's because the child-drawn hand is based on literal visual stimuli, but rather on an abstract logical model of a hand. That logical model is fully integrated with a similarly abstract model of the world, and includes functional relationships between abstractions, like the knowledge that "hands" can open "jars". The value of these being logical models rather than matched patterns is that they can then be extended to include never-before-seen objects. Confronted with a strange but roughly jar-sized object, a child can surmise that maybe it, too, can be opened with hands. That isn't pattern-matching: it's algebra.
      - yorwba 9y ago
        If you look at network activations in higher levels of a convolutional neural network, you will find some units that are activated by the individual parts of the hand. (There is a video out there where they show that for parts of the face, I'll be looking for a link.) Those activations are a representation of the matched patterns at a similar level of abstraction as the compositional model humans might come up with. They are not exactly the same as what a child would draw, but mostly because the neural network isn't trained to draw with hands. With a bit of work, I'm sure a bunch of PhDs could make a neural network model generate child-like drawings from realistic images.
      - goldenkey 9y ago
        Algebra is pattern matching of a set of operation rules with regard to a space. Your jar example just extends the domain to physicality. And I agree - until these sorts of learning mechanisms have a wide range of quality realisms to pattern match from - they will not be able to form the type of cross visual/physical knowledge that is a much deeper and abstract undestanding of reality. But don't fool yourself. Humans and well..life..are just input output machines with incredible pattern matching capabilities. Algebraic representations are the structural result of that pattern matching.