3 ms·

The other big limit to DL is its inability to learn a new concept from a single training example, AKA one-shot learning. Given that humans do most of their sup

The other big limit to DL is its inability to learn a new concept from a single training example, AKA one-shot learning. Given that humans do most of their supervised learning this way underscores that, even though DL nets intriguingly resemble the architecture of the brain, in fact, DL's current learning process clearly departs from the one used by the brain. (Unless the way a baby fiddles with each new object is in fact a form of data augmentation, increasing the visual sample space by changing the object's angles. But I digress...) Better to drop the "Neural Net as Brain" metaphor before we Procrustes it to death.

randyrand 8y ago
is that a limit of DL? or is that a limit of how we currently do our training?
hnick 8y ago
It also has no imagination, unless I've missed something (I admit I really need to catch up in this area). Humans are experts at imagining 'what if' scenarios and learning from things that never even happened. "If this was bright red, is it still a cat? Or do all cats have a limited range of colours on a fixed cat-colour-spectrum?" I'm not sure if any DL systems do that?
- randcraw 8y ago
  Proposing what-if scenarios and asking questions of your own and using your imagination generally falls under the name 'counterfactuals' -- a big area of AI research. But to date, the priorities of deep learning have focused elsewhere (mostly in perception of images and speech via discrete classification). Likewise, 99% of the computing strategy used by DL practitioners and researchers continues to use large scale supervised learning -- because it continues to make advances in many problem spaces just as it is. Is it possible to escape DL's apparent limits and use it differently to solve problems that lie in the counterfactual domain? Perhaps. But so far, there's been relatively little work or modest results along those lines. Given the architectural model used by DL (large matrix multiplications and iterations using SIMD operations on GPUs), the prospect of applying it to alternative AI objectives that are more conventionally symbolic and whose methods are typically algebraic (and unnatural to share the same computational model as DL) do not seem to be especially promising places for DL to thrive. Not so far anyway.

randyrand 8y ago

is that a limit of DL? or is that a limit of how we currently do our training?

hnick 8y ago

It also has no imagination, unless I've missed something (I admit I really need to catch up in this area). Humans are experts at imagining 'what if' scenarios and learning from things that never even happened. "If this was bright red, is it still a cat? Or do all cats have a limited range of colours on a fixed cat-colour-spectrum?" I'm not sure if any DL systems do that?

randcraw 8y ago
Proposing what-if scenarios and asking questions of your own and using your imagination generally falls under the name 'counterfactuals' -- a big area of AI research. But to date, the priorities of deep learning have focused elsewhere (mostly in perception of images and speech via discrete classification). Likewise, 99% of the computing strategy used by DL practitioners and researchers continues to use large scale supervised learning -- because it continues to make advances in many problem spaces just as it is. Is it possible to escape DL's apparent limits and use it differently to solve problems that lie in the counterfactual domain? Perhaps. But so far, there's been relatively little work or modest results along those lines. Given the architectural model used by DL (large matrix multiplications and iterations using SIMD operations on GPUs), the prospect of applying it to alternative AI objectives that are more conventionally symbolic and whose methods are typically algebraic (and unnatural to share the same computational model as DL) do not seem to be especially promising places for DL to thrive. Not so far anyway.