3 ms·

You're absolutely right that AGI and ASI are not well-defined, but we also need to recognize that they definitely are actual things even though we lack a defini

You're absolutely right that AGI and ASI are not well-defined, but we also need to recognize that they definitely are actual things even though we lack a definition. For any AI concept, I think it might be instructive to consider the human equivalent. For example, what is the definition of a genius? We don't have one, but genius certainly is a thing that exists. We might argue about who is/isn't a genius because of the missing definition, but we probably agree that exceptional people are a thing.

jillesvangurp 1y ago
Spot on. We struggle to define what we don't understand. And with AGI, that would include understanding ourselves. And when people start dragging in philosophy, religion, etc. you kind of know it's one of those things that definitely don't have much consensus. That doesn't mean it's all nonsense. I think objectively we are getting quite a bit of emergent definitions that emerge from established fact and technology that kind of narrow this down. LLMs seem part of the solution on a path to some form of artificial intelligence that can keep up with us and that we might struggle to keep up with. But LLMs are not the whole solution. Though what they can't do keeps shifting. From glorified autocomplete to solving mathematical problems that were previously not solved. In the space of less than 3 years since the launch of chat GPT in November 2022. If the math wasn't solved before, there has to be a bit more to it than a glorified autocomplete. Of course there are also counter points to that. But it at least challenges the notion that LLMs can't come up with new stuff. Even the notion that they can is deeply upsetting to some people because it challenges their world views. This debate is as much about that as it is about coming up with workable definitions. LLMs are obviously lacking in a lot of ways. Perhaps the most obvious thing is that after their training is completed, they stop learning and adapting. They have no memory beyond their prompts and what they learned during training. A chat gpt conversation is just a large prompt with the entire history of the conversation and some clever engineering to load that into GPU memory relatively quickly. There are a lot of party tricks that involve elaborate system prompts and other workarounds. The ability to remember and learn seems pretty fundamental to AGI, whatever that is. It's also a thing that doesn't sound like it's unsolvable. Some kind of indexed DB external to the AI for RAG kind of works but it does not solve the learning problem. And it shifts the problem to how to query and filter the data, which might not be optimal. And it's more similar to us using Google to find something than it is to us remembering information. Also, it's not like we're particularly good at cramming large amounts of information down or learning. Learning is a slow process. And we kind of get set in our ways as we age. Meaning we're reluctant to learn more stuff and less capable of doing so. That suggests that even modest improvements might have dramatic results. LLMs + some short term memory and ability to learn might end up being a lot more useful. I like duck typing in programming and that's also my mental model for AGIs. The Turing test is kind of obsolete. But if it quacks like duck and walks like a duck, it probably is a duck. Turing was onto something. Once we have a hard time telling apart the average person in society from an AI over days/weeks/years of interaction, we'll have AGIs. I think that's doable. But I'm not an ML engineer. A lot of those seem to believe it's doable too though. The rest of society in the form of self appointed arm chair professors, religious leaders, eminent philosophers, and (too put this politely) lesser qualified individuals with strong opinions makes a lot of confused noises. But it does not generate a lot of definitions that have broad consensus.
- mitthrowaway2 1y ago
  I will also add that it wasn't until the 19th century that we understood and could define what fire is, but we invented it many millennia before that. We made productive use of it for thousands of years having no idea what it exactly was, but sometimes it escaped our control. Sometimes it still does.
  - jillesvangurp 1y ago
    Great analogy. Full understanding is not required to get utility.
rkuodys 1y ago
While it's definitely interesting comparison, I would say the that key thing is that genius is limited in scope. What I mean by that is given any single genius - we might agree that he is one on subject X but not on subject Y. With AGI it seems that expectation is to cover all the subjects. Which I think is more like god. You either believe it or you don't. Noone have definite proof of its existence or non-existence.

jillesvangurp 1y ago

Spot on. We struggle to define what we don't understand. And with AGI, that would include understanding ourselves. And when people start dragging in philosophy, religion, etc. you kind of know it's one of those things that definitely don't have much consensus. That doesn't mean it's all nonsense. I think objectively we are getting quite a bit of emergent definitions that emerge from established fact and technology that kind of narrow this down. LLMs seem part of the solution on a path to some form of artificial intelligence that can keep up with us and that we might struggle to keep up with. But LLMs are not the whole solution. Though what they can't do keeps shifting. From glorified autocomplete to solving mathematical problems that were previously not solved. In the space of less than 3 years since the launch of chat GPT in November 2022. If the math wasn't solved before, there has to be a bit more to it than a glorified autocomplete. Of course there are also counter points to that. But it at least challenges the notion that LLMs can't come up with new stuff. Even the notion that they can is deeply upsetting to some people because it challenges their world views. This debate is as much about that as it is about coming up with workable definitions. LLMs are obviously lacking in a lot of ways. Perhaps the most obvious thing is that after their training is completed, they stop learning and adapting. They have no memory beyond their prompts and what they learned during training. A chat gpt conversation is just a large prompt with the entire history of the conversation and some clever engineering to load that into GPU memory relatively quickly. There are a lot of party tricks that involve elaborate system prompts and other workarounds. The ability to remember and learn seems pretty fundamental to AGI, whatever that is. It's also a thing that doesn't sound like it's unsolvable. Some kind of indexed DB external to the AI for RAG kind of works but it does not solve the learning problem. And it shifts the problem to how to query and filter the data, which might not be optimal. And it's more similar to us using Google to find something than it is to us remembering information. Also, it's not like we're particularly good at cramming large amounts of information down or learning. Learning is a slow process. And we kind of get set in our ways as we age. Meaning we're reluctant to learn more stuff and less capable of doing so. That suggests that even modest improvements might have dramatic results. LLMs + some short term memory and ability to learn might end up being a lot more useful. I like duck typing in programming and that's also my mental model for AGIs. The Turing test is kind of obsolete. But if it quacks like duck and walks like a duck, it probably is a duck. Turing was onto something. Once we have a hard time telling apart the average person in society from an AI over days/weeks/years of interaction, we'll have AGIs. I think that's doable. But I'm not an ML engineer. A lot of those seem to believe it's doable too though. The rest of society in the form of self appointed arm chair professors, religious leaders, eminent philosophers, and (too put this politely) lesser qualified individuals with strong opinions makes a lot of confused noises. But it does not generate a lot of definitions that have broad consensus.

mitthrowaway2 1y ago
I will also add that it wasn't until the 19th century that we understood and could define what fire is, but we invented it many millennia before that. We made productive use of it for thousands of years having no idea what it exactly was, but sometimes it escaped our control. Sometimes it still does.
- jillesvangurp 1y ago
  Great analogy. Full understanding is not required to get utility.

rkuodys 1y ago

While it's definitely interesting comparison, I would say the that key thing is that genius is limited in scope. What I mean by that is given any single genius - we might agree that he is one on subject X but not on subject Y. With AGI it seems that expectation is to cover all the subjects. Which I think is more like god. You either believe it or you don't. Noone have definite proof of its existence or non-existence.