4 ms·

> I know what it does under specific circumstances. I don't know what it does elsewhere. No, but if you get given a circumstance, you know how to work out what

> I know what it does under specific circumstances. I don't know what it does elsewhere. No, but if you get given a circumstance, you know how to work out what it does in that circumstance. Are you saying that you need to keep every input -> output mapping in your head to feel you understand a piece of code? I feel like I understand multiplication pretty well, but there are many multiplication calculations you could give me where I wouldn't know the answer without a lot of thinking. There's some I couldn't work out by myself in my lifetime. That doesn't stop me feeling like I understand multiplication pretty well. > Vengeance? Love? Loyalty? Pain? Hate? Emergent behaviours. Sure, and arguably at the level we're talking about, those are descriptive rather than explanatory. 'Vengeance' isn't something a neuron knows about, nor is it a biological mechanism in our cells, it's how we describe high level behavior resulting from the interactions of the cells. It's an abstraction. If you had the accurate model you were talking about earlier, you'd be able to work out that given the right input, a particular behavior is output. That others might call that behavior 'vengeance', makes not a single iota of difference to your ability to predict the behavior of the system. Are you saying that you need to have developed high level descriptions of the behavior of a system in order to feel you understand it? What if there are no high level descriptions? In the hypothetical scenario where we hit on an algorithm randomly, there's no requirement that it translates to any specific high level concepts, there's just input, output and the algorithm, all of which we can understand. Or perhaps you mean that you already have a set of categories for output behavior and to truly understand something you need to be able to categorise the inputs and know which broad input categories result in which output categories? I could probably accept that as a broad definition of understanding, but there's a lot of flex there in terms of exactly what level of granularity you're requiring.

blagie 3y ago
For a multiplication, I can work out any problem, and I have a sense of what it will do under any circumstance. > Are you saying that you need to have developed high level descriptions of the behavior of a system in order to feel you understand it? Yes. That's almost the definition of "understanding." > What if there are no high level descriptions? There are things we don't or can't understand. That's approximately Godel's Theorem. That likely includes some phenomena in fluid mechanics and in quantum mechanics. It may or may not include large-scale deep learning models. It's okay to admit we don't, or can't, understand something. > Or perhaps you mean that you already have a set of categories for output behavior and to truly understand something you need to be able to categorise the inputs and know which broad input categories result in which output categories? There are different levels of understanding. However, with LLMs, I don't have a clear sense of under what conditions one might decide to, for example, eradicate humanity. I'd say that suggests I have a very limited understanding of them. I don't think there are many people with a better understanding than mine, and no one with a good understanding. I feel like I understand a multiplication algorithm well enough to know it won't do that ever, however. If I multiply two numbers, I won't get a humanity-ending answer out. I don't know if deep learning models have some analogue to emotions. I do know multiplication doesn't. And so on.

blagie 3y ago

For a multiplication, I can work out any problem, and I have a sense of what it will do under any circumstance. > Are you saying that you need to have developed high level descriptions of the behavior of a system in order to feel you understand it? Yes. That's almost the definition of "understanding." > What if there are no high level descriptions? There are things we don't or can't understand. That's approximately Godel's Theorem. That likely includes some phenomena in fluid mechanics and in quantum mechanics. It may or may not include large-scale deep learning models. It's okay to admit we don't, or can't, understand something. > Or perhaps you mean that you already have a set of categories for output behavior and to truly understand something you need to be able to categorise the inputs and know which broad input categories result in which output categories? There are different levels of understanding. However, with LLMs, I don't have a clear sense of under what conditions one might decide to, for example, eradicate humanity. I'd say that suggests I have a very limited understanding of them. I don't think there are many people with a better understanding than mine, and no one with a good understanding. I feel like I understand a multiplication algorithm well enough to know it won't do that ever, however. If I multiply two numbers, I won't get a humanity-ending answer out. I don't know if deep learning models have some analogue to emotions. I do know multiplication doesn't. And so on.