3 ms·

I've only got a minute, but I think it was helpful for me to look at ML achievements in terms of input, output, and difficulty. A human brain can learn to reco

by sounds 7y ago

I've only got a minute, but I think it was helpful for me to look at ML achievements in terms of input, output, and difficulty. A human brain can learn to recognize cat pictures given a relatively small amount of input versus a convolutional neural net. A human brain hits a limit somewhere in terms of attention span, though - I used this to understand AlphaGo's accomplishment. AlphaGo is not inherently better at solving complex problems compared to a human brain, but it is demonstrably better at focusing on one complex problem until it has amassed the equivalent of thousands of years of training. I'd guess that the human inability to concentrate on one problem that long is actually a clue. Humans come up with new go and chess insights continually. Maybe it's because we're bad at evaluating the new ideas objectively for such complex problems, but it may also be a clue that the human brain has a more efficient method to develop new ways of analyzing complex problems. The best machine-based deep learning approaches still need to crunch through an enormous data set. Especially so for unsupervised learning.

thfuran 7y ago
>A human brain can learn to recognize cat pictures given a relatively small amount of input versus a convolutional neural net To give the ANN a fair go, I think you should pretrain on another dataset while evolving the architecture. Let that chug for, say, 500 million years. Then refine its performance on your cat dataset and see how long it takes for that step.
- russdill 7y ago
  I think there's another level of representation that needs to happen for NNs to learn the appearance of an object in a similar way that humans do. I've seen a lot of research recently on compressed representations, I think applying that to object recognition might yield some interesting results.
PeterisP 7y ago
"A human brain can learn to recognize cat pictures given a relatively small amount of input versus a convolutional neural net." is not really accurate, this is an apples and oranges comparison. A human brain that's already learned to see and recognize objects can learn to recognize cat pictures from a small amount of input data, and a neural network that's trained to see and recognize objects (which explicitly didn't include cats) can also learn to recognize cat pictures from the same amount of input data. One-shot learning or few-shot learning is a thing and gets reasonable results. And a human brain that's learning to see "from scratch" (i.e. infants) can't learn from a few pictures, it needs huge amounts (many months) of rich image data - i.e. not passive but interactive/experimental, which is known to be neccessary for visual system development in mammals (I recall a bunch of cat experiments) and is known to have much, much better sample efficiency for artificial systems. So as far as I know there's no good reason to assume that human brain can learn cat pictures from less input than neural networks. What we're seeing and misattributing to the "learning capabilities" here is actually the effectiveness of transfer learning, transferring knowledge and skills everyone's learned as a kid to new (but actually very related) problems.

by sounds 7y ago

thfuran 7y ago

>A human brain can learn to recognize cat pictures given a relatively small amount of input versus a convolutional neural net To give the ANN a fair go, I think you should pretrain on another dataset while evolving the architecture. Let that chug for, say, 500 million years. Then refine its performance on your cat dataset and see how long it takes for that step.

russdill 7y ago
I think there's another level of representation that needs to happen for NNs to learn the appearance of an object in a similar way that humans do. I've seen a lot of research recently on compressed representations, I think applying that to object recognition might yield some interesting results.

PeterisP 7y ago

"A human brain can learn to recognize cat pictures given a relatively small amount of input versus a convolutional neural net." is not really accurate, this is an apples and oranges comparison. A human brain that's already learned to see and recognize objects can learn to recognize cat pictures from a small amount of input data, and a neural network that's trained to see and recognize objects (which explicitly didn't include cats) can also learn to recognize cat pictures from the same amount of input data. One-shot learning or few-shot learning is a thing and gets reasonable results. And a human brain that's learning to see "from scratch" (i.e. infants) can't learn from a few pictures, it needs huge amounts (many months) of rich image data - i.e. not passive but interactive/experimental, which is known to be neccessary for visual system development in mammals (I recall a bunch of cat experiments) and is known to have much, much better sample efficiency for artificial systems. So as far as I know there's no good reason to assume that human brain can learn cat pictures from less input than neural networks. What we're seeing and misattributing to the "learning capabilities" here is actually the effectiveness of transfer learning, transferring knowledge and skills everyone's learned as a kid to new (but actually very related) problems.