3 ms·
Human priors are a feature, not a bug. The reason deep neural networks need so much data to train them (and still do not handle languag nearly as well as humans
by YeGoblynQueenne 5y ago
Human priors are a feature, not a bug. The reason deep neural networks need so much data to train them (and still do not handle languag nearly as well as humans) is precisely that humans have "background knowledge", things that we already know and don't have to learn all over again from scratch. We bring this knowledge to bear in our ability to learn and understand language. Deep neural networks on the other hand have only a very, very limited ability to represent and therefore use background knowledge- and so they have to make up for it with astounding amounts of data.
The OP's criticism is valid. Being forced to learn everything end-to-end, from scratch, is a severe limitation.
- bane 5y agoI think what's very interesting is that most of the answers to my question sort of boil down (if you squint a little at the answers) to "this is hard, but general AI will make it easy".
- YeGoblynQueenne 5y agoMy apologies if the wording of my comment was confusing but what you say is not at all what I meant.
- whimsicalism 5y ago> Human priors are a feature, not a bug. ... Being forced to learn everything end-to-end, from scratch, is a severe limitation. They're neither. When performing very human-adjacent tasks, it will certainly put the ML algorithm at a disadvantage compared to us. But for non-human adjacent tasks, say interpreting what a sequence of amino acids actually means, we can expect the computer to absolutely crush us because our stupid human heuristics take us absolutely nowhere, cause us to see patterns that aren't there, etc. etc. Regardless, this is irrelevant to the original point that I was making, which is that comparing the performance of DL on human adjacent tasks to the amount of time it takes a human to learn the same task is misguided because you are ignoring the million-year long optimization process to get there.