4 ms·

One of the main challenges in Deep Learning is that it requires massive amounts of data, orders of magnitude more data than a human toddler to detect a cat. It

• rando18423 10y ago
  In a way, I think you just described the fundamental difference between "intelligence" and "very advanced applied statistical analysis." :)
• pawelwentpawel 10y ago
  Interesting! Made me wonder - how would we compare the data that neural net receives with the data that toddler's visual cortex is getting? On a purely visual level - let's say we have 10k of static 32x32 images defining a class of cat. Or even more of them plus some negative examples. Each image is a different cat, in a different position (they're incredibly flexible creatures). Having so many cases we should be trying to make some kind of a generalization of what 1024 pixels of a cat should look like. A family with a toddler has only one cat. From that one example, he learns the concept of what a cat is and is able to generalize it when in different situations. But a toddler has 2 eyes, his visual input is stereo. Even if he sees just one cat, it's not a static image interaction. The input is temporal, he can see the cat moving and interacting with the environment.
  • camikazeg 10y ago
    So instead of 10k static 32x32 random images of cats for training data, why not do 10k frames of cat videos that add up to seeing the cat moving and interacting with the environment?
  • dharma1 10y ago
    I think stereo vision isn't super important for this - people blind in one eye don't do noticeably worse. Not to say depth doesn't help for segmentation or learning concepts but I don't think it's the key
• rryan 10y ago
  > It could be a great area of research on how to reduce the amount of data it takes to train the network. This is typically referred to as "one-shot learning" in the literature -- and people certainly work on it!
• lkozma 10y ago
  To make that comparison fair, maybe you should also count the amount of data used during the human toddler's evolution when the structure of the brain became what it is.
• mattnewton 10y ago
  Wouldn't we just need dolphins and guitars separately with something like https://github.com/awentzonline/image-analogies https://github.com/awentzonline/image-analogies ?
  • 31reasons 10y ago
    Thats amazing. Yes something like that but for concepts. Merging Dolphin + Guitar Playing. Both have visual elements to it but also has conceptual elements.
    • mattnewton 10y ago
      To me though, this proves that it is merely a difference of degree and not of kind. You could do this with short stories about guitar dolphins, as many images as you'd like, maybe a song, and even videos someday with enough compute.
• visarga 10y ago
  > orders of magnitude more data than a human toddler to detect a cat Perception works at 10-20 frames per second, all day long. That means 0.5 million perceptions per day. Why would a small neural net that has 1/1000 the experience of a child and 1/1000 the size of a brain (assuming it is 100 million neurons which is huge!) be able to be more accurate? What you are referring to is called "one shot learning" and is the ability to learn from a single example, which is being studied in the literature.
• lars 10y ago
  > One main thing it lacks is imagination. [...] For example, if I ask you to imagine a Guitar playing Dolphin, you could imagine it and even recognize it from a cartoon even though you have never seen it in your life before. Not so for Deep Learning These days one should be careful when claiming things deep learning can't do. There are in fact systems that can imagine things it has never seen before. Here's an example. It takes any text, and imagines an image for it: https://github.com/emansim/text2image https://github.com/emansim/text2image Here's a more recent example, with even higher quality images: http://arxiv.org/pdf/1605.05396v2.pdf http://arxiv.org/pdf/1605.05396v2.pdf