3 ms·

Just to clarify, I am an atheist and I don't mind taht people believe in god. Sorry, _God_. I respect the decision to believe and to worship. It's a fundamental

Just to clarify, I am an atheist and I don't mind taht people believe in god. Sorry, _God_. I respect the decision to believe and to worship. It's a fundamental element of humanity. I'm not against it at all. I just don't think that those beliefs are real, that's all. The last thing I would ever try to do is to convince someone that God doesn't exist. Why would I ever do that? As to LLMs, they are not supernatural beings that are the subject of belief, or even disbelief. They are technological artifacts created by humans, machines that perform a task they were created to perform. We created the machines, we know how they are made, and we know what they can do. For example, if you were to argue that a car can understand transport, or an airplane, flight, just because they are very complex technological artifacts and they behave in ways that are not easy to predict, or even explain, then I'd have to say, similar to what I say about LLMs: "an airplane is a machine designed to fly, not understand flight, so if you think it understands flight it's up to you to show me how its mechanisms allow it to do that". In particular I would ask you this because I am just now polishing up a PhD in AI (although not the kind you read in the news) so this is my field of expertise and I believe it is reasonable for me to have questions, when someone makes assertions about the functioning of systems that I have studied. In short, it's my job to know and to understand how those systems work, and if you say they 're working somehow otherwise, then of course I'd like to know why you say that. So, yes, I'm going to want to know where "understanding" has been hiding from me all this time, and I'd like very precise instructions to find it myself, please, otherwise I'm going to assume you're making it up as you go along. No offense meant. I have to do that as part of my job (i.e. reviews). Perhaps it's a "deformation professionelle", eh?

fwlr 4y ago
I’m not an AI researcher so my answer won’t be satisfying, but I’m pretty confident that meaning is stored in the embeddings, and gun to my head I would say understanding is hiding in the weights. My handwavey argument is that the map of activation weights between layers of neurons can store understanding, in the same way that a complete log of every transaction can store bank account balances (even though there’s no actual “this amount is in this account at this time” information anywhere in that log).
- YeGoblynQueenne 4y ago
  Well, the idea with embeddings is, indeed, that they are a representation of meaning, specifically a spatial representation of the meaning of words. The theory is that if two words are close together in "embedding space", their meaning is also close together. The problem with that has always been that embeddings can be said to encode relative meaning, but not absolute meaning. That is, to understand the meaning of a word A with reference to a word B, you need to know the meaning of B. And the only entities that can do that -tell you what a word means - are humans. So a neural net may learn all the embeddings it likes, it's missing a foundation of meaning on which to base them. To give an analogy, imagine trying to understand what a word means, in a language you don't know, given only a dictionary of that language. Every word in the dictionary is explained, except it's explained in a language you don't know. You can see that some words are explained in terms of other words, but that doesn't really help much because you don't know the meaning of the other words in the first place. It's lemmas all the way down. Or suppose I gave you an encrypted string. You can learn the string by heart and repeat it as many times you like, but you won't know what it says until you can decrypt it. So maybe embeddings encode meaning, but where is it that they decode it? The same goes with neural net weights. The problem with all those techniques -and, to be fair, with every other approach to AI so far- is that we're missing a foundational encoding of meaning that we can relate other representations to. Some people think we can get over this by training neural nets on real-world data, or make robots and let them experiment with the world etc, but I don't know that anyone has really shown that this works yet. So, maybe neural nets can store understanding in their weights, but we don't know how to put it there, yet.

fwlr 4y ago

I’m not an AI researcher so my answer won’t be satisfying, but I’m pretty confident that meaning is stored in the embeddings, and gun to my head I would say understanding is hiding in the weights. My handwavey argument is that the map of activation weights between layers of neurons can store understanding, in the same way that a complete log of every transaction can store bank account balances (even though there’s no actual “this amount is in this account at this time” information anywhere in that log).

YeGoblynQueenne 4y ago
Well, the idea with embeddings is, indeed, that they are a representation of meaning, specifically a spatial representation of the meaning of words. The theory is that if two words are close together in "embedding space", their meaning is also close together. The problem with that has always been that embeddings can be said to encode relative meaning, but not absolute meaning. That is, to understand the meaning of a word A with reference to a word B, you need to know the meaning of B. And the only entities that can do that -tell you what a word means - are humans. So a neural net may learn all the embeddings it likes, it's missing a foundation of meaning on which to base them. To give an analogy, imagine trying to understand what a word means, in a language you don't know, given only a dictionary of that language. Every word in the dictionary is explained, except it's explained in a language you don't know. You can see that some words are explained in terms of other words, but that doesn't really help much because you don't know the meaning of the other words in the first place. It's lemmas all the way down. Or suppose I gave you an encrypted string. You can learn the string by heart and repeat it as many times you like, but you won't know what it says until you can decrypt it. So maybe embeddings encode meaning, but where is it that they decode it? The same goes with neural net weights. The problem with all those techniques -and, to be fair, with every other approach to AI so far- is that we're missing a foundational encoding of meaning that we can relate other representations to. Some people think we can get over this by training neural nets on real-world data, or make robots and let them experiment with the world etc, but I don't know that anyone has really shown that this works yet. So, maybe neural nets can store understanding in their weights, but we don't know how to put it there, yet.