4 ms·
>> If you look at a horse for a minute it could be the equivalent of training on 1 million images of a horse. If you trained a neural net with 1 million images
by YeGoblynQueenne 6y ago
>> If you look at a horse for a minute it could be the equivalent of training on 1 million images of a horse.
If you trained a neural net with 1 million images of the same horse, it would learn to recognise that horse... but no other horse. Neural net datasets try to include as many variants of the target concept as possible in order to capture as many of the common features of instances of that concept as possible. A single horse would not suffice, e.g. 1 million images of a white horse would teach a neural net that horses are only white, etc.
Also, a child can learn to recognise horses from a caricature of a horse- which is a single image of a horse, not 1 million images. Whatever human minds do when they learn to recognise objects, that's not training with big data in real time.
- breck 6y ago> Also, a child can learn to recognise horses from a caricature of a horse The way the chain of connection goes from the eye through the brain is like a tree, and when a child "sees" a horse, a number of those pathways are being activated, some of which are more general, and some of which are specific. > which is a single image of a horse, Although it may be a "single" image in the sense that is a single file on disk or a single printed image, a child is not seeing a single image. They are seeing a rushing river of horses, even if from just that 1 static image. Think of the 60hz refresh rate of your monitor. A child is seeing at least 60 "images per second", and likely many, many times more.
- YeGoblynQueenne 6y agoBut those 60 images per second are the same static, unchanging caricature. It makes more sense to say that the child sees the same image multiple times than to say she sees multiple images. So how does the child learn to recognise a fully 3-dimensional real-life horse from a single caricature of a horse seen any number of times? Can you explain? Also, if you were to train a nerual net with a single insance of a caricature of a horse, even if you copied it a million times to create an example set of a million copies, the neural net would still only be able to elarn how to recognise that particular caricature, if that- it would certainly not be able to extract any useful features to recognise real-life horses from that single caricature.
- dmurray 6y agoTransfer learning is a thing though. If you train your network with thousands of pictures of cats, dogs, chickens, birds, goats but no horses, it's possible you can teach it to recognise a horse with a single additional image. Not sure exactly how good the state of the art is compared to human children on that task, but it can't be too far off.
- breck 6y agoAh, the transfer learning is the more important point, dominating my point on augmentation. I can show a 2 year old a single picture of a horse and have them learn it, but only after training them on 5 billion pictures of non-horses.
- YeGoblynQueenne 6y agoBut how exactly do you "train" a 2-year old on 5 billion pictuers of non-horses? And at what point does the child learn to recognise images of anything? I mean, how do you know it takes 5 billion images of non-horses before the child can learn to recognise a horse? Even if the child sees 5 billion images of non-horses, who says that they haven't learned to recognise the non-horses and the horses from the first of those 5 billion images? After all, I don't know that children stay transfixed for hours staring at horses and non-horses while their brains process all those billions of images until they finally get it. I think you are making an effort to explain away something that is not very well understood by modern science, by analogy with machine learning, even though there is no good reason to suppose that the two are connected in any way.
- breck 6y ago24 hours a day times 3600 seconds per hour times 60hz times 1000 = approximately 5 billion. Just a ball park figure for how many “pictures” they have seen. Probably off by a few OOM, but point holds. I think there is a very good reason to think machine learning and human learning are connected. First, I’d say it’s largely true the people who have made the most impact in the former have all studied the way the mind works (as far as science allows), and tried to emulate that in machinery. Second, experimentally the phenomena observed are increasingly similar (deep dream, for example).