3 ms·

I’m not an AI researcher so my answer won’t be satisfying, but I’m pretty confident that meaning is stored in the embeddings, and gun to my head I would say und

by fwlr 4y ago

I’m not an AI researcher so my answer won’t be satisfying, but I’m pretty confident that meaning is stored in the embeddings, and gun to my head I would say understanding is hiding in the weights. My handwavey argument is that the map of activation weights between layers of neurons can store understanding, in the same way that a complete log of every transaction can store bank account balances (even though there’s no actual “this amount is in this account at this time” information anywhere in that log).

YeGoblynQueenne 4y ago
Well, the idea with embeddings is, indeed, that they are a representation of meaning, specifically a spatial representation of the meaning of words. The theory is that if two words are close together in "embedding space", their meaning is also close together. The problem with that has always been that embeddings can be said to encode relative meaning, but not absolute meaning. That is, to understand the meaning of a word A with reference to a word B, you need to know the meaning of B. And the only entities that can do that -tell you what a word means - are humans. So a neural net may learn all the embeddings it likes, it's missing a foundation of meaning on which to base them. To give an analogy, imagine trying to understand what a word means, in a language you don't know, given only a dictionary of that language. Every word in the dictionary is explained, except it's explained in a language you don't know. You can see that some words are explained in terms of other words, but that doesn't really help much because you don't know the meaning of the other words in the first place. It's lemmas all the way down. Or suppose I gave you an encrypted string. You can learn the string by heart and repeat it as many times you like, but you won't know what it says until you can decrypt it. So maybe embeddings encode meaning, but where is it that they decode it? The same goes with neural net weights. The problem with all those techniques -and, to be fair, with every other approach to AI so far- is that we're missing a foundational encoding of meaning that we can relate other representations to. Some people think we can get over this by training neural nets on real-world data, or make robots and let them experiment with the world etc, but I don't know that anyone has really shown that this works yet. So, maybe neural nets can store understanding in their weights, but we don't know how to put it there, yet.

by fwlr 4y ago

YeGoblynQueenne 4y ago

Well, the idea with embeddings is, indeed, that they are a representation of meaning, specifically a spatial representation of the meaning of words. The theory is that if two words are close together in "embedding space", their meaning is also close together. The problem with that has always been that embeddings can be said to encode relative meaning, but not absolute meaning. That is, to understand the meaning of a word A with reference to a word B, you need to know the meaning of B. And the only entities that can do that -tell you what a word means - are humans. So a neural net may learn all the embeddings it likes, it's missing a foundation of meaning on which to base them. To give an analogy, imagine trying to understand what a word means, in a language you don't know, given only a dictionary of that language. Every word in the dictionary is explained, except it's explained in a language you don't know. You can see that some words are explained in terms of other words, but that doesn't really help much because you don't know the meaning of the other words in the first place. It's lemmas all the way down. Or suppose I gave you an encrypted string. You can learn the string by heart and repeat it as many times you like, but you won't know what it says until you can decrypt it. So maybe embeddings encode meaning, but where is it that they decode it? The same goes with neural net weights. The problem with all those techniques -and, to be fair, with every other approach to AI so far- is that we're missing a foundational encoding of meaning that we can relate other representations to. Some people think we can get over this by training neural nets on real-world data, or make robots and let them experiment with the world etc, but I don't know that anyone has really shown that this works yet. So, maybe neural nets can store understanding in their weights, but we don't know how to put it there, yet.