6 ms·

Comparing this to atheism is very illuminating, actually. In some ways this topic is very different to atheism/theism. In the ‘belief in god’ debate both sides

by fwlr 4y ago

Comparing this to atheism is very illuminating, actually. In some ways this topic is very different to atheism/theism. In the ‘belief in god’ debate both sides are trying to take away the other side’s belief; it’s important to atheists to stop people believing in God, while to believers it’s important to stop people disbelieving in God, so there’s a lot of convincing being attempted, people seeking each other out to tell them that they cannot or must not hold a certain belief, and so forth. This topic is very different! If I want to go off and believe LLMs can understand and reason and think, you don’t really care; likewise, when you believe that, although it might be producing a novel rephrasing, the information in that rephrasing is merely reproduced from the training corpus, I don’t really care. (Beyond active engagements in discussion threads, anyhow - not trying to imply either of us don’t care about the discussion we’re having right now!) But in other ways it is very similar to the atheism/theism debate. For a time I used to argue God exists because the idea of God controls and causes very specific behaviors from a great many people; a naive diagram of the situation has this God figure and He’s telling these little people figures to do certain things and they do exactly those things (which is why “pointing out that certain figures are hypocritically not doing those things” feels important and like it might be evidence against), and the atheist argument that you can’t actually detect God’s existence is like erasing the God figure from the diagram: it leaves a conspicuous hole in the diagram, and the hole is shaped like God, and if you were to naively fill in the hole in the diagram to explain what’s causing the people to behave that way, the precise shape of the hole forces you to end up re-drawing God (this argument was very unconvincing to atheists for multiple reasons and I’m not trying to re-litigate it here). It became clear to me that atheists were not interested in this level of proof, it had to be physical existence; some ‘God-stuff’ that you could grab a handful of and hold up to the light and say “here it is”. This topic is very similar! For me, it is enough that the diagram of an LLM’s capabilities has a hole shaped exactly like “it understands” and I’m happy to fill in the obvious blank; for you, there needs be an actual process of understanding somewhere in the code and the weights that we could hold up to the light and say “here is the understanding” (and that is how you can claim that no matter how good LLMs get they will never be understanding - all of their processes are already fully described and explained, there’s nowhere for a process of understanding to hide).

YeGoblynQueenne 4y ago
Just to clarify, I am an atheist and I don't mind taht people believe in god. Sorry, _God_. I respect the decision to believe and to worship. It's a fundamental element of humanity. I'm not against it at all. I just don't think that those beliefs are real, that's all. The last thing I would ever try to do is to convince someone that God doesn't exist. Why would I ever do that? As to LLMs, they are not supernatural beings that are the subject of belief, or even disbelief. They are technological artifacts created by humans, machines that perform a task they were created to perform. We created the machines, we know how they are made, and we know what they can do. For example, if you were to argue that a car can understand transport, or an airplane, flight, just because they are very complex technological artifacts and they behave in ways that are not easy to predict, or even explain, then I'd have to say, similar to what I say about LLMs: "an airplane is a machine designed to fly, not understand flight, so if you think it understands flight it's up to you to show me how its mechanisms allow it to do that". In particular I would ask you this because I am just now polishing up a PhD in AI (although not the kind you read in the news) so this is my field of expertise and I believe it is reasonable for me to have questions, when someone makes assertions about the functioning of systems that I have studied. In short, it's my job to know and to understand how those systems work, and if you say they 're working somehow otherwise, then of course I'd like to know why you say that. So, yes, I'm going to want to know where "understanding" has been hiding from me all this time, and I'd like very precise instructions to find it myself, please, otherwise I'm going to assume you're making it up as you go along. No offense meant. I have to do that as part of my job (i.e. reviews). Perhaps it's a "deformation professionelle", eh?
- fwlr 4y ago
  I’m not an AI researcher so my answer won’t be satisfying, but I’m pretty confident that meaning is stored in the embeddings, and gun to my head I would say understanding is hiding in the weights. My handwavey argument is that the map of activation weights between layers of neurons can store understanding, in the same way that a complete log of every transaction can store bank account balances (even though there’s no actual “this amount is in this account at this time” information anywhere in that log).

by fwlr 4y ago

YeGoblynQueenne 4y ago

Just to clarify, I am an atheist and I don't mind taht people believe in god. Sorry, _God_. I respect the decision to believe and to worship. It's a fundamental element of humanity. I'm not against it at all. I just don't think that those beliefs are real, that's all. The last thing I would ever try to do is to convince someone that God doesn't exist. Why would I ever do that? As to LLMs, they are not supernatural beings that are the subject of belief, or even disbelief. They are technological artifacts created by humans, machines that perform a task they were created to perform. We created the machines, we know how they are made, and we know what they can do. For example, if you were to argue that a car can understand transport, or an airplane, flight, just because they are very complex technological artifacts and they behave in ways that are not easy to predict, or even explain, then I'd have to say, similar to what I say about LLMs: "an airplane is a machine designed to fly, not understand flight, so if you think it understands flight it's up to you to show me how its mechanisms allow it to do that". In particular I would ask you this because I am just now polishing up a PhD in AI (although not the kind you read in the news) so this is my field of expertise and I believe it is reasonable for me to have questions, when someone makes assertions about the functioning of systems that I have studied. In short, it's my job to know and to understand how those systems work, and if you say they 're working somehow otherwise, then of course I'd like to know why you say that. So, yes, I'm going to want to know where "understanding" has been hiding from me all this time, and I'd like very precise instructions to find it myself, please, otherwise I'm going to assume you're making it up as you go along. No offense meant. I have to do that as part of my job (i.e. reviews). Perhaps it's a "deformation professionelle", eh?

fwlr 4y ago
I’m not an AI researcher so my answer won’t be satisfying, but I’m pretty confident that meaning is stored in the embeddings, and gun to my head I would say understanding is hiding in the weights. My handwavey argument is that the map of activation weights between layers of neurons can store understanding, in the same way that a complete log of every transaction can store bank account balances (even though there’s no actual “this amount is in this account at this time” information anywhere in that log).