4 ms·

Here's the bit about cats: If this [huge google] network had been fed thousands of images labelled as ‘contains cats’ or ‘doesn’t contain cats’ and trained to

Here's the bit about cats: If this [huge google] network had been fed thousands of images labelled as ‘contains cats’ or ‘doesn’t contain cats’ and trained to work out the difference for itself by iteratively tweeking its 1.7 billion parameters until it had found a classification rule, that would have been impressive enough, given the scale of the task involved in mapping from pixels to low-level image features and then to something as varied and complex as a cat’s face. What Google actually achieved is much more extraordinary, and slightly chilling. The input images weren’t labelled in any way: the network distilled the concept of ‘cat face’ out of the data without any guidance.

pavlov 10y ago
The problem is that this network-contained concept of "cat face" is still a symbolic representation. It's a much more complex algorithmic symbol than the rules found in something like 1960s Eliza, but its understanding of the world is on the same level. You can't ask the "cat face" neural network anything about cats. It has no idea what they actually are in relation to the world. A two-year-old human can usually tell you more about cats than you'd care to listen.
- Vraxx 10y ago
  I think the important part is that it "learned" to distinguish this concept of cat face without that being the direct intent provided through labeling. Given the set of images, it learned that "concept" which we can then retroactively label to make it useful in the typical human sense by associating it with cats. If we had to compose computer "knowledge" by training it in everything by manually specifying what it was training for, that process would surely be insurmountable in a general sense. Yet this seems to provide the possibility of automating this process. A separate network of connections between "knowledge" bits could maybe be used to associate related knowledge like you talk about in terms of their relation to the world. This could also probably be formed in a similar manner giving the algorithm the ability to distinguish "important" concepts contained in the data. The thing I find most odd and interesting about this is that the network tends to identify different important concepts than humans do.
- empath75 10y ago
  You can't ask a child's visual cortex to tell you anything about cats, either. But connect that visual recognition ai with something like Watson, and you have what you're looking for, no?
  - pavlov 10y ago
    I just don't know. I'd be happy to see that be the case... But I'm afraid it's going to be the equivalent of this cake recipe: "Take an egg and a packet of sugar. Break the egg over the packet." You certainly need eggs and sugar to make a cake, but on their own and combined without understanding of the whole, you're not getting very far.
    - empath75 10y ago
      The human brain is mostly a bunch of ugly wetware hacks, not a single coherent 'intelligence' that does all the thinking. If you attach enough single purpose AIs together you might get something much more human like than trying to create a single neural network that does everything.
- wmf 10y ago
  If word embedding can produce things like "king - man + woman = queen", then concept embedding on images might be able to achieve a similar level of "understanding" (i.e. very low, but probably better than nothing).
gertef 10y ago
I don't see what's "chilling" about it. The neural net looks for correlations among the data, and it noticed that cat faces have a correlation to each other.
Question1101 10y ago
How exactly is a neural network called that classifies unlabeled data by itself? Also could they turn it around somehow to produce cat faces?
- khedoros 10y ago
  Google has some software called DeepDream. Basically, you give it images, and it amplifies any features that it thinks it recognizes. If the network is designed to recognize cat faces, then it will take an arbitrary picture and strengthen parts of the images to look more cat-face-like. It's essentially an image-recognition neural network run in reverse.
obastani 10y ago
I don't think that's as surprising/chilling as people make it out to be. Let's say I run a compression algorithm (e.g., Huffman coding) on a billion strings, and a very common substring is "hello, world". I wouldn't be surprised if the compression algorithm "learned" to compress "hello, world" to a single bit 0. There are a few big jumps to get to cat faces (e.g., the representation has to be approximate), but I don't see why the idea is fundamentally different.

pavlov 10y ago

The problem is that this network-contained concept of "cat face" is still a symbolic representation. It's a much more complex algorithmic symbol than the rules found in something like 1960s Eliza, but its understanding of the world is on the same level. You can't ask the "cat face" neural network anything about cats. It has no idea what they actually are in relation to the world. A two-year-old human can usually tell you more about cats than you'd care to listen.

Vraxx 10y ago
I think the important part is that it "learned" to distinguish this concept of cat face without that being the direct intent provided through labeling. Given the set of images, it learned that "concept" which we can then retroactively label to make it useful in the typical human sense by associating it with cats. If we had to compose computer "knowledge" by training it in everything by manually specifying what it was training for, that process would surely be insurmountable in a general sense. Yet this seems to provide the possibility of automating this process. A separate network of connections between "knowledge" bits could maybe be used to associate related knowledge like you talk about in terms of their relation to the world. This could also probably be formed in a similar manner giving the algorithm the ability to distinguish "important" concepts contained in the data. The thing I find most odd and interesting about this is that the network tends to identify different important concepts than humans do.
empath75 10y ago
You can't ask a child's visual cortex to tell you anything about cats, either. But connect that visual recognition ai with something like Watson, and you have what you're looking for, no?
- pavlov 10y ago
  I just don't know. I'd be happy to see that be the case... But I'm afraid it's going to be the equivalent of this cake recipe: "Take an egg and a packet of sugar. Break the egg over the packet." You certainly need eggs and sugar to make a cake, but on their own and combined without understanding of the whole, you're not getting very far.
  - empath75 10y ago
    The human brain is mostly a bunch of ugly wetware hacks, not a single coherent 'intelligence' that does all the thinking. If you attach enough single purpose AIs together you might get something much more human like than trying to create a single neural network that does everything.
wmf 10y ago
If word embedding can produce things like "king - man + woman = queen", then concept embedding on images might be able to achieve a similar level of "understanding" (i.e. very low, but probably better than nothing).

gertef 10y ago

I don't see what's "chilling" about it. The neural net looks for correlations among the data, and it noticed that cat faces have a correlation to each other.

Question1101 10y ago

How exactly is a neural network called that classifies unlabeled data by itself? Also could they turn it around somehow to produce cat faces?

khedoros 10y ago
Google has some software called DeepDream. Basically, you give it images, and it amplifies any features that it thinks it recognizes. If the network is designed to recognize cat faces, then it will take an arbitrary picture and strengthen parts of the images to look more cat-face-like. It's essentially an image-recognition neural network run in reverse.

obastani 10y ago

I don't think that's as surprising/chilling as people make it out to be. Let's say I run a compression algorithm (e.g., Huffman coding) on a billion strings, and a very common substring is "hello, world". I wouldn't be surprised if the compression algorithm "learned" to compress "hello, world" to a single bit 0. There are a few big jumps to get to cat faces (e.g., the representation has to be approximate), but I don't see why the idea is fundamentally different.