5 ms·
> ...very different from human learning. A child often needs to see just a few examples of an object, or even only one, before being able to recognize it for li
by breck 6y ago
> ...very different from human learning. A child often needs to see just a few examples of an object, or even only one, before being able to recognize it for life.
I see this a lot. It's completely wrong. I'm not trying to pick on the author here, I think 95%+ of people share this misunderstanding of deep learning.
If you see "only one" horse, say for even a second, you really are seeing a huge number of horses, from various angles, with various shades of lighting. The motions of the horse; the motions of your head (even if slight); the undulations of the light; are generating a much larger number of basically augmented training data. If you look at a horse for a minute it could be the equivalent of training on 1 million images of a horse. I'm not sure the exact OOM, but it's certainly orders of magnitude more than "one" horse.
(Relatedly: Some people say there is an experiment you can conduct at home to see the actual images your brain is training on).
- ummonk 6y agoNot really. A child can see a single static picture of an unfamiliar animal and learn to recognize the animal from that picture alone.
- breck 6y agoNo, they can’t. Flash an image of an animal for a microsecond in front of a child. They won’t “see” anything. One needs to “see” many, many versions of something before you “see” anything at all, nevermind learn something.
- ummonk 6y agoThat isn’t seeing many versions of the animal. It is merely processing the one training sample over several seconds, because it takes a few hundred milliseconds for the human brain and visual system to process an image, and several more seconds for the brain to analyze and memorize key features of the sample. Standard machine learning systems don’t work like this. No matter how much time they’re allowed to process a single sample image, they are unable to learn to classify it. There are of course machine learning systems that work more like humans (facial recognition is a notable example), but they’re the exception not the norm.
- breck 6y agoAll machine learning models that I've trained have involved a data augmentation step. These are artificial augmentations though, so not as effective as more real data. Looking at the same horse live from 100 different perspectives might be better for learning what a horse is than training on 100 static images of different horses.
- ummonk 6y agoYes, data augmentation is standard. It isn't new data though, just different variations of the same data. You also tend to do multiple training passes over the data, even after augmentation. None of that is the point though: a child can look at a single static image (not 100 different perspectives - just one perspective) of a horse, and learn to recognize a horse. A standard machine learning model cannot, no matter how much you augment the image.
- breck 6y ago> a child can look at a single static image But in an important way, they can't. You need to define "look". How many nanoseconds? How much is the lighting changing during that time? How still is the photo? How still is the person's head? In DL a training example in a batch is perfectly defined in bits. Once you try to define a training example for a human you see that a "single static image" means totally different things. A human seeing a static image is the equivalent of training a model on at least thousands, if not millions+ of training images.
- YeGoblynQueenne 6y ago>> If you look at a horse for a minute it could be the equivalent of training on 1 million images of a horse. If you trained a neural net with 1 million images of the same horse, it would learn to recognise that horse... but no other horse. Neural net datasets try to include as many variants of the target concept as possible in order to capture as many of the common features of instances of that concept as possible. A single horse would not suffice, e.g. 1 million images of a white horse would teach a neural net that horses are only white, etc. Also, a child can learn to recognise horses from a caricature of a horse- which is a single image of a horse, not 1 million images. Whatever human minds do when they learn to recognise objects, that's not training with big data in real time.
- breck 6y ago> Also, a child can learn to recognise horses from a caricature of a horse The way the chain of connection goes from the eye through the brain is like a tree, and when a child "sees" a horse, a number of those pathways are being activated, some of which are more general, and some of which are specific. > which is a single image of a horse, Although it may be a "single" image in the sense that is a single file on disk or a single printed image, a child is not seeing a single image. They are seeing a rushing river of horses, even if from just that 1 static image. Think of the 60hz refresh rate of your monitor. A child is seeing at least 60 "images per second", and likely many, many times more.
- YeGoblynQueenne 6y agoBut those 60 images per second are the same static, unchanging caricature. It makes more sense to say that the child sees the same image multiple times than to say she sees multiple images. So how does the child learn to recognise a fully 3-dimensional real-life horse from a single caricature of a horse seen any number of times? Can you explain? Also, if you were to train a nerual net with a single insance of a caricature of a horse, even if you copied it a million times to create an example set of a million copies, the neural net would still only be able to elarn how to recognise that particular caricature, if that- it would certainly not be able to extract any useful features to recognise real-life horses from that single caricature.