5 ms·

I'm pretty sure that was the reasoning to build the ImageNet (about a million labeled images) in the first place. But labeling images is expensive, and there ar

I'm pretty sure that was the reasoning to build the ImageNet (about a million labeled images) in the first place. But labeling images is expensive, and there are hints there's more at play with human cognition. If you see a black cat and a white cat, and someone tells you there are striped colored cats, you can imagine it. And if you were to come across it, you'd instantly recognize it as a cat. Neurals nets can't do that. You can also see a lynx and recognize it as "some kind of cat". Again, neural nets are not there yet. Which is why there are people researching to find new, better algorithms that better mimic what we recognize as intelligence.

yorwba 9y ago
Are you sure about your examples of things neural nets can't do? I think GANs might be able to "imagine" striped cats, provided they have been trained on enough images to capture the space of black/white/striped objects. And a lynx being classified as a cat doesn't seem so outlandish. It has to be classified as something and cats are likely the closest in appearance. Of course these are just based on my intuition of what neural nets are capable of, so if you have examples of cases where these specific tasks were attempted unsuccessfully, I'm interested.
- mannykannot 9y ago
  I would be interested regardless of the outcome.
- dr_zoidberg 9y ago
  Let me remind you of sofas being classified as cats[0] and people being classified as gorillas[1]. You're overestimating the guesswork convnets are able to do, based on fragile training (which is still better guesswork than what previous models did). [0] http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.html http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.ht... [1] https://www.theverge.com/2015/7/1/8880363/google-apologizes-photos-app-tags-two-black-people-gorillas https://www.theverge.com/2015/7/1/8880363/google-apologizes-...
  - yorwba 9y ago
    People being classified as gorillas was actually what I was thinking about regarding the lynx/cat example. The model might have been unsure about the kind of ape it was looking at, but clustering them together is its own kind of achievement.
    - dr_zoidberg 9y ago
      The thing is that the convnets are unable to learn about "macro structures" (or structures in general). A cat has ~4 legs, a tail and pointy ears. Gorillas are black, have a primate-y face and fur. The sofa is lacking the tail, head and pointy ears. People were missing the fur. Yet those things did not prevent the net from missclassffication (because those features weren't detected in the learning phase). Once again, children are able to see a cat and extract all that relevant information: four legs, head, tail, eyes & nose & ears with a particular shape, different than dogs, most cats fur (except for those alien-looking furless cats, of course).
      - nkoren 9y ago
        If you ask a child to draw a hand, they will almost always draw it with five fingers stuck straight out, widely separated. This is a view of a hand that one almost never actually sees; generally you'll have fingers clustered together, occluding each other, foreshortened, etc. So why do they draw it like that? It's because they're drawing a conceptual, represntational model of a hand, not a distilation of visual "hand" characteristics. That's the difference with human learning: it's based on representational model-making, which is not at all the same thing as pattern matching.
        goldenkey 9y ago
        The representation is the distillation of pattern matching. They are isomorphic.
        nkoren 9y ago
MrQuincle 9y ago
https://github.com/junyanz/CycleGAN https://github.com/junyanz/CycleGAN
- dr_zoidberg 9y ago
  GANs and style transfer are not the same as being able to recognize and imagine changes on the representation you just learned. Also, look at the GAN examples: even the water in the background is being affected by the "horse to zebra" transfer. You can perfectly imagine a brown horse, standing in the beach, and then being told "now iamgine the horse is white" without than "instruction" affecting how you imagine the beach.
  - MrQuincle 9y ago
    Perhaps explain what your preferred dataset is. + Brown horses on beaches + Some way to indicate "white" "horse". If you have a good idea about how we can train for your problem, I'm not so convinced that it cannot be solved.
    - dr_zoidberg 9y ago
      No, the problem I was pointing at is that you want to change a part of the image (horse into zebra), but the style transfer GAN learned to map pixels from one space into another. So it knows it has to change the colour of some things, and add stripes here and there, and also probably there too. But it isn't consistent with the stripes (you can see in some gifs how the pattern suddenly changes and adjusts), and it doesn't recognize (segment) the horse as the only relevant thing that has to be changed. But that's not bad per se about style transfer. It's an interesting technique, but if you want to convert all horses to zebras in an image, that seems to be a bit too general for current-generation GAN architectures. Maybe it can be improved upon, or a different, novel architecture is required, and not just something you can solve by throwing more data at it.
      - MrQuincle 9y ago
        Yes, the segmentation could be better. I still think you're not pointing out fundamental issues though. :-)

yorwba 9y ago

Are you sure about your examples of things neural nets can't do? I think GANs might be able to "imagine" striped cats, provided they have been trained on enough images to capture the space of black/white/striped objects. And a lynx being classified as a cat doesn't seem so outlandish. It has to be classified as something and cats are likely the closest in appearance. Of course these are just based on my intuition of what neural nets are capable of, so if you have examples of cases where these specific tasks were attempted unsuccessfully, I'm interested.

mannykannot 9y ago
I would be interested regardless of the outcome.
dr_zoidberg 9y ago
Let me remind you of sofas being classified as cats[0] and people being classified as gorillas[1]. You're overestimating the guesswork convnets are able to do, based on fragile training (which is still better guesswork than what previous models did). [0] http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.html http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.ht... [1] https://www.theverge.com/2015/7/1/8880363/google-apologizes-photos-app-tags-two-black-people-gorillas https://www.theverge.com/2015/7/1/8880363/google-apologizes-...
- yorwba 9y ago
  People being classified as gorillas was actually what I was thinking about regarding the lynx/cat example. The model might have been unsure about the kind of ape it was looking at, but clustering them together is its own kind of achievement.
  - dr_zoidberg 9y ago
    The thing is that the convnets are unable to learn about "macro structures" (or structures in general). A cat has ~4 legs, a tail and pointy ears. Gorillas are black, have a primate-y face and fur. The sofa is lacking the tail, head and pointy ears. People were missing the fur. Yet those things did not prevent the net from missclassffication (because those features weren't detected in the learning phase). Once again, children are able to see a cat and extract all that relevant information: four legs, head, tail, eyes & nose & ears with a particular shape, different than dogs, most cats fur (except for those alien-looking furless cats, of course).
    - nkoren 9y ago
      If you ask a child to draw a hand, they will almost always draw it with five fingers stuck straight out, widely separated. This is a view of a hand that one almost never actually sees; generally you'll have fingers clustered together, occluding each other, foreshortened, etc. So why do they draw it like that? It's because they're drawing a conceptual, represntational model of a hand, not a distilation of visual "hand" characteristics. That's the difference with human learning: it's based on representational model-making, which is not at all the same thing as pattern matching.
      - goldenkey 9y ago
        The representation is the distillation of pattern matching. They are isomorphic.
        nkoren 9y ago

MrQuincle 9y ago

https://github.com/junyanz/CycleGAN https://github.com/junyanz/CycleGAN

dr_zoidberg 9y ago
GANs and style transfer are not the same as being able to recognize and imagine changes on the representation you just learned. Also, look at the GAN examples: even the water in the background is being affected by the "horse to zebra" transfer. You can perfectly imagine a brown horse, standing in the beach, and then being told "now iamgine the horse is white" without than "instruction" affecting how you imagine the beach.
- MrQuincle 9y ago
  Perhaps explain what your preferred dataset is. + Brown horses on beaches + Some way to indicate "white" "horse". If you have a good idea about how we can train for your problem, I'm not so convinced that it cannot be solved.
  - dr_zoidberg 9y ago
    No, the problem I was pointing at is that you want to change a part of the image (horse into zebra), but the style transfer GAN learned to map pixels from one space into another. So it knows it has to change the colour of some things, and add stripes here and there, and also probably there too. But it isn't consistent with the stripes (you can see in some gifs how the pattern suddenly changes and adjusts), and it doesn't recognize (segment) the horse as the only relevant thing that has to be changed. But that's not bad per se about style transfer. It's an interesting technique, but if you want to convert all horses to zebras in an image, that seems to be a bit too general for current-generation GAN architectures. Maybe it can be improved upon, or a different, novel architecture is required, and not just something you can solve by throwing more data at it.
    - MrQuincle 9y ago
      Yes, the segmentation could be better. I still think you're not pointing out fundamental issues though. :-)