3 ms·

My hot take is that what some people are labeling as "emergent" is actually just "incidental encoding" or "implicit signal" -- latent properties that get embedd

My hot take is that what some people are labeling as "emergent" is actually just "incidental encoding" or "implicit signal" -- latent properties that get embedded just by nature of what's being looked at. For instance, if you have a massive tome of English text, a rather high percentage of it will be grammatically-correct (or close), syntactic and understandable, because humans who speak good English took the time to write it and wrote it how other humans would expect to read or hear it. This, by its very nature, embeds "English language" knowledge due to sequence, word choice, normally-hard-to-quantify expressions (colloquial or otherwise), etc. When you consider source data from many modes, there's all kinds of implicit stuff that gets incidentally written.. for instance, real photographs of outer space or deep sea would only show humans in protective gear, not swimming next to the Titanic. Conversely, you won't see polar bears eating at Chipotle, or giant humans standing on top of mountains. There's a statistical probability of "this showed up enough in the training data to loosely confirm its existence" / "can't say I ever saw that, so let's just synthesize it" aspect of the embeddings that one person could interpret as "emergent intelligence", while another could just-as-convincingly say it's probabilistic output that is mostly in line with what we expect to receive. Train the LLM on absolute nonsense instead and you'll receive exactly that back.

atoav 1y ago
Emergent as I have known and used it before is when more complex behavior emerges from simple rules. My goto example for this was Game of Life, where from very simple rules, a very organically behaving (turing complete) system emerges. Now Game of Life is a deterministic system, meaning that the same rules and the same start-configurarion will play out in exactly the same way each time — but given the simplicity of the logic and the rules the resulting complexity is what I'd call emergent. So maybe this is more about the definition of what we'd call emergent and what not. As someone who has programmed markov chains where the stochastic interpolation really shines through, transformer-based LLMS definitly show some emergent behavior one wouldn't have immediately suspected just from the rules. Emergent does not mean "conscious" or "self-reflective" or anything like that. But the things a LLM can infer from its training data is already quite impressive.

atoav 1y ago

Emergent as I have known and used it before is when more complex behavior emerges from simple rules. My goto example for this was Game of Life, where from very simple rules, a very organically behaving (turing complete) system emerges. Now Game of Life is a deterministic system, meaning that the same rules and the same start-configurarion will play out in exactly the same way each time — but given the simplicity of the logic and the rules the resulting complexity is what I'd call emergent. So maybe this is more about the definition of what we'd call emergent and what not. As someone who has programmed markov chains where the stochastic interpolation really shines through, transformer-based LLMS definitly show some emergent behavior one wouldn't have immediately suspected just from the rules. Emergent does not mean "conscious" or "self-reflective" or anything like that. But the things a LLM can infer from its training data is already quite impressive.