6 ms·

“Emergent anything” is probably the most obnoxious buzzword in all of machine learning.

by raydiatian 3y ago

joaquincabezas 3y ago
all you need is emergent anything
- raydiatian 3y ago
  Emergency emergent emergence
thomastjeffery 3y ago
It gives finality to the idea that we don't understand the thing or where it came from. Why? Did we just lose all interest in understanding things? Wasn't that the whole point in the first place? Somehow, people are throwing up their hands and giving up at understanding the thing; yet at the same time they are acting like the thing will magically evolve into their wildest dreams! The most fundamental feature of LLMs is that they cannot be literal. They can only infer, never define. Why is it that the people studying LLMs think they have to emulate that trait? It's like they are only allowing themselves to look at it as a mysterious black box: to infer its behavior from its results. Did they forget that they are the ones who wrote the damn thing?
- amw-zero 3y ago
  The whole idea of machine learning / AI is to build functionality indirectly though, i.e. to build a system which evolves into another system over time. They are inherently meta-systems, so it does make sense to think of them differently.
- Mike_12345 3y ago
  > Somehow, people are throwing up their hands and giving up at understanding the thing Go tell that to the researchers who are working hard on studying emergent properties.
  - thomastjeffery 3y ago
    Yes: the effects of the thing, and not the thing itself.
- opportune 3y ago
  Before blackbox AIs from deep learning there were basically a few different kinds of AI: one was basically “algorithm complicated enough that we thought it required intelligence”, another was “general problem solver” like you get by applying Constraint Satisfaction techniques and heuristics, and highly fine-tuned encodings of human knowledge and research (this is a decision tree clinicians use to perform a differential diagnosis of a fever, this is a function that finds edges in an image based on hand crafted CV algorithms). The first group is basically not AI, it was just assumed to be. The other two groups were fully explainable but required a ton of effort to get working outside of very tightly scoped situations. For a long time researchers thought that some combination of the two approaches would lead to more generalized models, but all attempts at morphing the two sucked ass because all knowledge had to be a hand crafted ontology of rules and atoms that could only explicitly encode relationships. Also, while computers can solve CSP/graph traversal algorithms impossibly fast compared to humans, those tasks are not good models for human cognition or tasks beyond stuff like crossword puzzles. You should consider that despite considerable effort, human brains are themselves black boxes. And you know less about your own knowledge than you think. I do not know where I learned that Timbuktu is both a placeholder name and real place, though I could go find evidence for both. I don’t have to expend any effort to distinguish the sounds of different words, I don’t know why two things can both taste “good”but in completely different ways. Nobody ever taught me that newly met acquaintances tended to not care to discuss current events in the business world, I just figured it out based on a collection of experiences whose individual instances I can’t even remember. Even the best neuroscientist could not tell you why neurons interacting in a certain way makes it so I can both drive a car and sing, or why one person’s brain seems to better at some or generalized tasks than another’s. And, well, deep learning overturned the paradigm of handcrafting AI systems by automating the process of “have the model produce this output from this input” without requiring humans to define the “how” beyond tuning the shape of the model, which was itself a hugely important innovation in reducing the human time required to build an AI system. But it’s not just faster to make these models, it’s so ridiculously better at making AI models for things like “is there a dog in this picture” that nobody would even consider doing those things without deep learning. You actually can fiddle with DNNs to get an idea of how they work similar to what we do with brains and CAT scans, you have it do some stuff with commonalities and you figure out which common parts get activated. This is easy to do with convolutional layers as they very commonly learn for themselves how to perform edge detection. Anyway, long story short, fully explainable AI utterly sucks ass at many tasks that are like a walk on the park for blackbox AI. And we cannot explain our own intelligence and knowledge except in terms of emergent phenomena, nor can we give the full provenance of some factoid or skill we have on demand (just like an LLM cannot tell you where it learned something) in many cases[0], so it seems reasonable that we’d be in the same situation with AI. [0] The main difference is that we have memory of the various discrete experiences of our lives (which we can associate with some knowledge or skill), and there is no binary separation between “learning mode” and “doing mode” or “active memory” and “long term memory” for us like with AI. We can definitely associate some knowledge with a particular event, but this seems like it could be a false ontological representation of our knowledge because if the knowledge and event were unimportant (like what you had for breakfast on a particular day) we’d forget both of them; it’s actually all the subsequent cases in which the knowledge and memory of the event came in handy that contribute to us being able to explain it.
opportune 3y ago
I’m sorry you find it obnoxious but emergent phenomena are everywhere in math and science and as annoying as it is to you, it also happens with AI. The quest for more generalized models boils down to studying emergent behavior because we could never prescriptively define all the parameters/behaviors/requirements necessary for such a complex outcome. We don’t even understand how the relatively easily observed interactions between neurons in our own brains result in emergent intelligence. What’s so impressive about LLMs is they understand the semantics of some concepts so well that they can consistently produce higher quality outputs for tasks like “explain this complicated concept with a nursery song from the perspective of a pirate” than humans could, with approximately no instances of that task in their training data. That is emergent behavior and it’s a pretty big deal.
- raydiatian 3y ago
  I agree that emergent behaviors are real, and important. I am skeptical, although not completely unconvinced, that LLMs like GPT are going to produce truly emergent phenomena, such as true first-principles logical reasoning. The limitations of the underlying transformer architecture itself are, in my opinion, the problem. The first problem is that the embedding space of the transformer needs to grow much, much larger, and it's already huge. This matters because you need to model the order of neurons in the brain. The second problem is that you're never going to train an LLM (as they're designed today) that is going to produce a truly good 'emergent-phenomena' answer without multiple network traversals. This is because the human mind constantly and autonomously refines its thoughts. Perhaps a good counter-argument is that emergent phenomena are fundamentally a space-time domain concept. I am aware that things like Conway's Game of Life are a fantastic counterargument to my "the transformer architecture doesn't support it" argument. But I agree that the definition of "emergent behavior" when it comes to machine learning is too easily corrupted to be novel rather than rigorous.

“Emergent anything” is probably the most obnoxious buzzword in all of machine learning.

joaquincabezas 3y ago

all you need is emergent anything

raydiatian 3y ago
Emergency emergent emergence

thomastjeffery 3y ago

It gives finality to the idea that we don't understand the thing or where it came from. Why? Did we just lose all interest in understanding things? Wasn't that the whole point in the first place? Somehow, people are throwing up their hands and giving up at understanding the thing; yet at the same time they are acting like the thing will magically evolve into their wildest dreams! The most fundamental feature of LLMs is that they cannot be literal. They can only infer, never define. Why is it that the people studying LLMs think they have to emulate that trait? It's like they are only allowing themselves to look at it as a mysterious black box: to infer its behavior from its results. Did they forget that they are the ones who wrote the damn thing?

amw-zero 3y ago
The whole idea of machine learning / AI is to build functionality indirectly though, i.e. to build a system which evolves into another system over time. They are inherently meta-systems, so it does make sense to think of them differently.
Mike_12345 3y ago
> Somehow, people are throwing up their hands and giving up at understanding the thing Go tell that to the researchers who are working hard on studying emergent properties.
- thomastjeffery 3y ago
  Yes: the effects of the thing, and not the thing itself.
opportune 3y ago
Before blackbox AIs from deep learning there were basically a few different kinds of AI: one was basically “algorithm complicated enough that we thought it required intelligence”, another was “general problem solver” like you get by applying Constraint Satisfaction techniques and heuristics, and highly fine-tuned encodings of human knowledge and research (this is a decision tree clinicians use to perform a differential diagnosis of a fever, this is a function that finds edges in an image based on hand crafted CV algorithms). The first group is basically not AI, it was just assumed to be. The other two groups were fully explainable but required a ton of effort to get working outside of very tightly scoped situations. For a long time researchers thought that some combination of the two approaches would lead to more generalized models, but all attempts at morphing the two sucked ass because all knowledge had to be a hand crafted ontology of rules and atoms that could only explicitly encode relationships. Also, while computers can solve CSP/graph traversal algorithms impossibly fast compared to humans, those tasks are not good models for human cognition or tasks beyond stuff like crossword puzzles. You should consider that despite considerable effort, human brains are themselves black boxes. And you know less about your own knowledge than you think. I do not know where I learned that Timbuktu is both a placeholder name and real place, though I could go find evidence for both. I don’t have to expend any effort to distinguish the sounds of different words, I don’t know why two things can both taste “good”but in completely different ways. Nobody ever taught me that newly met acquaintances tended to not care to discuss current events in the business world, I just figured it out based on a collection of experiences whose individual instances I can’t even remember. Even the best neuroscientist could not tell you why neurons interacting in a certain way makes it so I can both drive a car and sing, or why one person’s brain seems to better at some or generalized tasks than another’s. And, well, deep learning overturned the paradigm of handcrafting AI systems by automating the process of “have the model produce this output from this input” without requiring humans to define the “how” beyond tuning the shape of the model, which was itself a hugely important innovation in reducing the human time required to build an AI system. But it’s not just faster to make these models, it’s so ridiculously better at making AI models for things like “is there a dog in this picture” that nobody would even consider doing those things without deep learning. You actually can fiddle with DNNs to get an idea of how they work similar to what we do with brains and CAT scans, you have it do some stuff with commonalities and you figure out which common parts get activated. This is easy to do with convolutional layers as they very commonly learn for themselves how to perform edge detection. Anyway, long story short, fully explainable AI utterly sucks ass at many tasks that are like a walk on the park for blackbox AI. And we cannot explain our own intelligence and knowledge except in terms of emergent phenomena, nor can we give the full provenance of some factoid or skill we have on demand (just like an LLM cannot tell you where it learned something) in many cases[0], so it seems reasonable that we’d be in the same situation with AI. [0] The main difference is that we have memory of the various discrete experiences of our lives (which we can associate with some knowledge or skill), and there is no binary separation between “learning mode” and “doing mode” or “active memory” and “long term memory” for us like with AI. We can definitely associate some knowledge with a particular event, but this seems like it could be a false ontological representation of our knowledge because if the knowledge and event were unimportant (like what you had for breakfast on a particular day) we’d forget both of them; it’s actually all the subsequent cases in which the knowledge and memory of the event came in handy that contribute to us being able to explain it.

opportune 3y ago

I’m sorry you find it obnoxious but emergent phenomena are everywhere in math and science and as annoying as it is to you, it also happens with AI. The quest for more generalized models boils down to studying emergent behavior because we could never prescriptively define all the parameters/behaviors/requirements necessary for such a complex outcome. We don’t even understand how the relatively easily observed interactions between neurons in our own brains result in emergent intelligence. What’s so impressive about LLMs is they understand the semantics of some concepts so well that they can consistently produce higher quality outputs for tasks like “explain this complicated concept with a nursery song from the perspective of a pirate” than humans could, with approximately no instances of that task in their training data. That is emergent behavior and it’s a pretty big deal.

raydiatian 3y ago
I agree that emergent behaviors are real, and important. I am skeptical, although not completely unconvinced, that LLMs like GPT are going to produce truly emergent phenomena, such as true first-principles logical reasoning. The limitations of the underlying transformer architecture itself are, in my opinion, the problem. The first problem is that the embedding space of the transformer needs to grow much, much larger, and it's already huge. This matters because you need to model the order of neurons in the brain. The second problem is that you're never going to train an LLM (as they're designed today) that is going to produce a truly good 'emergent-phenomena' answer without multiple network traversals. This is because the human mind constantly and autonomously refines its thoughts. Perhaps a good counter-argument is that emergent phenomena are fundamentally a space-time domain concept. I am aware that things like Conway's Game of Life are a fantastic counterargument to my "the transformer architecture doesn't support it" argument. But I agree that the definition of "emergent behavior" when it comes to machine learning is too easily corrupted to be novel rather than rigorous.