3 ms·

I'm probably not understanding fully, but I really struggle to see why we wouldn't believe emergent behaviour is possible, if not likely, with larger models. A

by kypro 3y ago

I'm probably not understanding fully, but I really struggle to see why we wouldn't believe emergent behaviour is possible, if not likely, with larger models. At its core, what larger models enable is deeper abstraction. Imagine a CNN with a single kernel, for example. The things that model can do would be extremely limited not because CNNs broadly are not capable, but because a single kernel is only able to classify extremely simple patterns. As more kernels are added the network's ability to classify more complex patterns grows. So I suppose in some ways you could argue that there is a simple scalable trend here, more size -> more complexity. But on the other hand, if you're looking for a network with a specific ability, say one which can classify an image of cat accurately, then this ability is more binary and will "emerge" at a certain network size. As I've said in other comments, to accurately predict the next word in a complex piece of text you need to be able to reason. I can't say for sure if the architecture of current LLMs are capable of reasoning, but assuming it is then we should at least expect the ability to reason to begin to emerge at a certain network size. Or we could think about this another way... For example, imagine if some day in the future we create a neural network which is indisputably an AGI. If we scaled back that network's size while maintaining its architecture it likely wouldn't be an AGI. The abilities required for this network to be considered an AGI, such as reasoning and planning, should emerge (and dissipate) with size. So the assumption that more complex abilities would continue to emerge with scale – given the architecture permits – seems clear to me. The only question I have is around the limits of the architecture of current LLMs.

lstodd 3y ago
> As I've said in other comments, to accurately predict the next word in a complex piece of text you need to be able to reason. If anything, LLMs have shown us exactly the opposite: that there is no need to be able to reason "to accurately predict the next word in a complex piece of text".
- kypro 3y ago
  I tend to agree. I think a lot of what LLMs do that looks like reasoning is just basic statistical pattern matching. If you see the string `2 + 2 = 4` enough, you don't need to know maths to know the next character in the string, `2 + 2 = ` is going to be, `4`. A lot of the stuff people ask LLMs do to probably falls into this category to be honest. If I were to guess the vast majority of maths on the internet is just generic textbook examples which an LLM might be tempted to brute force. There's likely much better things LLMs can dedicate network capacity to to reduce error than learning high-level math models, given that most text doesn't contain maths, and most of the maths it sees is probably just the generic, `2 + 2 = 4` type stuff. I'd argue humans often do this too despite our general intelligence. Student prepping for exams often read all of the course material the night before and simply try to remember the bits that they need to pass their test rather than spending unnecessary time building a higher level mental models of the material. It's only when you need to learn to predict something novel frequently that the need to comprehend at a higher level becomes necessary. I think given the size of modern LLMs and the amount of data they're trained on it's likely that they are now starting to create these higher level abstractions as the limits of brute force statistical pattern matching are starting to be reached. With GPT-3.5/4 it seems specifically the types of reasoning you might need to follow a piece of written text it can do quite well. And this isn't surprising given the training data probably consists of a lot of news articles and fictional pieces of text, and relatively little unique maths.
danparsonson 3y ago
Why should facility with language be the foundation of general intelligence? Any more than cat recognition is? I would argue that it's the other way around - in living things, development of language is emergent from intelligence, not a prerequisite for it. That desired behaviour 'emerges' from specialised networks trained for a specific purpose should not be too surprising, but it doesn't automatically imply that any other behaviour will appear. I think in the case of ChatGPT, we just didn't really know what to expect; we are therefore amazed by the things it can do, whereas with time and greater understanding perhaps we will come to realise that this is what language models are - and perhaps in the process understand our own minds more deeply. My feeling is that the crucial difference between us and the machines right now is that our software runs on the network, whereas ChatGPT et al use the network as part of their operation, as if the network is some sort of coprocessor. I don't think we'll see "true" AGI until there is nothing but the network receiving inputs and feeding back into itself endlessly, i.e. introspecting - thinking about thinking about thinking, per Douglas Hofstadter. I have no basis for this assertion beyond intuition though.
- RamblingCTO 3y ago
  That's a very old discussion haha. Wasn't this what Chomsky et al were talking about back in the days? I think the answer still is "we don't know".
  - danparsonson 3y ago
    Almost as old as humanity itself I suppose :-)

by kypro 3y ago

lstodd 3y ago

> As I've said in other comments, to accurately predict the next word in a complex piece of text you need to be able to reason. If anything, LLMs have shown us exactly the opposite: that there is no need to be able to reason "to accurately predict the next word in a complex piece of text".

kypro 3y ago
I tend to agree. I think a lot of what LLMs do that looks like reasoning is just basic statistical pattern matching. If you see the string `2 + 2 = 4` enough, you don't need to know maths to know the next character in the string, `2 + 2 = ` is going to be, `4`. A lot of the stuff people ask LLMs do to probably falls into this category to be honest. If I were to guess the vast majority of maths on the internet is just generic textbook examples which an LLM might be tempted to brute force. There's likely much better things LLMs can dedicate network capacity to to reduce error than learning high-level math models, given that most text doesn't contain maths, and most of the maths it sees is probably just the generic, `2 + 2 = 4` type stuff. I'd argue humans often do this too despite our general intelligence. Student prepping for exams often read all of the course material the night before and simply try to remember the bits that they need to pass their test rather than spending unnecessary time building a higher level mental models of the material. It's only when you need to learn to predict something novel frequently that the need to comprehend at a higher level becomes necessary. I think given the size of modern LLMs and the amount of data they're trained on it's likely that they are now starting to create these higher level abstractions as the limits of brute force statistical pattern matching are starting to be reached. With GPT-3.5/4 it seems specifically the types of reasoning you might need to follow a piece of written text it can do quite well. And this isn't surprising given the training data probably consists of a lot of news articles and fictional pieces of text, and relatively little unique maths.

danparsonson 3y ago

Why should facility with language be the foundation of general intelligence? Any more than cat recognition is? I would argue that it's the other way around - in living things, development of language is emergent from intelligence, not a prerequisite for it. That desired behaviour 'emerges' from specialised networks trained for a specific purpose should not be too surprising, but it doesn't automatically imply that any other behaviour will appear. I think in the case of ChatGPT, we just didn't really know what to expect; we are therefore amazed by the things it can do, whereas with time and greater understanding perhaps we will come to realise that this is what language models are - and perhaps in the process understand our own minds more deeply. My feeling is that the crucial difference between us and the machines right now is that our software runs on the network, whereas ChatGPT et al use the network as part of their operation, as if the network is some sort of coprocessor. I don't think we'll see "true" AGI until there is nothing but the network receiving inputs and feeding back into itself endlessly, i.e. introspecting - thinking about thinking about thinking, per Douglas Hofstadter. I have no basis for this assertion beyond intuition though.

RamblingCTO 3y ago
That's a very old discussion haha. Wasn't this what Chomsky et al were talking about back in the days? I think the answer still is "we don't know".
- danparsonson 3y ago
  Almost as old as humanity itself I suppose :-)