5 ms·

What is AGI?

Can someone explain what AGI is. I know it stands for Artificial general intelligence and its the idea that computer will overtake humans. But I'm not sure what that actually means. I mean surely a calculator can pass human levels of computing speed and accuracy on multiplying large numbers. An AI like ChatGPT could have an IQ of 300+, 500+. But that wouldn't make it 'conscious'. Can you have consciousness without a physical body? What does it even mean in the abstract? If ChatGPT can say "I'm just a language model", is that being conscious to some degree? ChatGPT already has more knowledge than any individual human. So, are we sure 'AGI' is even a thing. That AI can be 'self aware' rather than just high IQ and knowledgeable?

giorgioz 3y ago
That is a very good question. I think I read Sam Altman saying "AGI is everything we haven't done yet. Something new that will feel like magic.". Sadly, I can't find a quote and URL so I can't confirm Sam Altman said it. But the concept resonated with me. We have had Google Translate for a decade. It was and is simply a miracle of technology straight out of sci-fi books. Yet we grew quickly accustomed to it, taking it for granted. So, it's a bit the same with the concept of AGI. Now chatGPT 4.0 is the new normal and is taken for granted. So AGI is essentially anything better than ChatGPT 4.0 Frankly, I don't know if we are close to AGI. We could be years away or thousands of years away. I'll mention some past discoveries and how people reacted to put things in perspective. We went to the moon in 1969. We then dreamed we would be colonizing the galaxy. Yet space is very big and empty. If Starship lands on Mars 10 years from now, 65 years will have passed between the Moon and Mars landing. Other solar systems and Earth-like planets are much, much, much further away than Mars. It's the same in AI. We might just have landed on the moon, and we might have all these dreams about colonizing other planets (creating AGI). We might be technically very far, far away from it. We will make steady progress, and we will give names to each step of the process. AGI seems a generic name for the journey.
- kingkongjaffa 3y ago
  This is fine until "We will make steady progress", that's not guaranteed, in fact there's already a phrase for this https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter With space travel, there is a very well established physics to model and predict how we might do these things, you need to do some stuff like build rockets, have them escape earth's gravity, build things in space. Everything is a known known or a known unknown but we have the physics to model everything involved with space travel. This is an entirely different class of problem to "creating AGI", since there is basically a huge unknown unknown ocean to cross. So far everything we have done in AI is a statistical proxy for real thought processes, and there's one assumption that says lets make more and more of these kinds of AI techniques with "bigger" systems and we might approach something like AGI. But it isn't certain. so far with the GPT's we've really only succeeded in synthesizing human written text corpus into a statistical prediction engine, which can synthesise useful text, but does not have any semblance of 'thought'.
  - JoeAltmaier 3y ago
    That changes every day. The evidence is all in favor of 'steady progress'.
spidersenses 3y ago
What about (1) reasoning and (2) unassisted learning/self-improvement? Both currently don't apply to state-of-the-art LLMs.
proc0 3y ago
AGI is a human level machine intelligence. Think about all the sci-fi movies and games with robots that can talk and understand humans perfectly, and whether they are consciousness or not is a separate question. A good example is the Gith from Mass Effect that have an entire civilization (if that's what we could call it), but their intelligence is a different and behave more like a hive mind. Another example could be C3PO or even R2D2 (given that they both understand human language, intentions, etc). > I mean surely a calculator can pass human levels of computing speed and accuracy on multiplying large numbers. You can't talk to calculators, something that humans do. Transformers (GPT, Claude) are much closer to AI that we can talk to, however it becomes evident they do not think like humans despite their impressive amount of knowledge. I would say the biggest missing piece of the puzzle with LLMs and transformers is theory of mind, or the capacity of humans to model the thinking of other humans. Once AI can model your thinking base on the context of the conversation, it will be capable of holding long conversations and provide useful, realtime insights. I think at that point we will have AGI, and we will still not have a full understanding of consciousness or whether AGI is conscious.
- AnimalMuppet 3y ago
  One more step: An AGI must have a theory of its own mind, not just the mind of the person it's talking to. It must be able to "watch itself think", and to think about its own thinking.
  - proc0 3y ago
    Yes, that would be included in theory of mind, good point.
Staple_Diet 3y ago
>ChatGPT already has more knowledge than any individual human. It has access to the same amount of knowledge that any individual human (with internet) has. >So, are we sure 'AGI' is even a thing. I'm a cognitive neuroscientist and I'm going to say no... there's no plausible path towards creating an AGI that is a true facsimile of human intelligence BUT I'd recommend looking into P-Zombies, as there will most definitely be models that approach a believable bar, people are quick to anthropomorphizise anything. >That AI can be 'self aware' rather than just high IQ and knowledgeable? I question your references to AI having a high IQ. The IQ test is, among many things, a biased and imperfect tool. Further, it's not possible for an AI to complete the full test as one component is recall. Using IQ tests to measure AI is like comparing a car's acceleration to Olympic sprinting records. My view is that AI tools are exactly that, tools. ChatGPT is mish-mash of a thesaurus and Google. It bears remembering that ChatGPT was trained on data produced by humans, volumes of text, novels, scientific articles, blogs and reddit posts. To ascribe it intelligence for its ability to algorithmically reassemble it's training data is to do a disservice to all the authors of that original work.
- philomath_mn 3y ago
  > It has access to the same amount of knowledge that any individual human (with internet) has. Do you have a nuanced definition of "access"? GPT is generating most responses from its own model weights, it only reaches out to the internet in certain situations (and it tells you when it is doing so). > Further, it's not possible for an AI to complete the full test as one component is recall GPT has recall during one conversation and they recently added features to recall details from previous conversations (again, unless you mean something different by "recall") > It bears remembering that ChatGPT was trained on data produced by humans, volumes of text, novels, scientific articles, blogs and reddit posts. To ascribe it intelligence for its ability to algorithmically reassemble it's training data is to do a disservice to all the authors of that original work. I don't think GPT is conscious or anything, but I'm curious what you think the distinction is between a human child learning from its environment and a computer model learning from training data. GPT has created many truly novel outputs based on what it learned from the training data. Mechanistically, how is a brain learning from data different from a computer model learning from data?
dauertewigkeit 3y ago
The emphasis should be on "general". The easiest way to understand this is by thinking of the opposite: we can already build AI systems that are very good at one specific task: folding proteins, playing Chess, detecting cats in images, translating from one language to another, detecting market inefficiencies...etc But a human, the same human, can do all those things. A human has a sort of general intelligence. This is the goal of AGI. One can think of it as a some kind of unification theory for AI systems. One model that can be capable of excelling at vastly different tasks. AGI has nothing to do with surpassing human intelligence. However, the idea is, once you have a general intelligence running on a computer, you could easily scale it to create a super human level intelligence.
- AnimalMuppet 3y ago
  > However, the idea is, once you have a general intelligence running on a computer, you could easily scale it to create a super human level intelligence. I question that. I expect that "intelligence" (however defined) will not be a linear function of the scale of the system. It will be considerably less than linear. If it's log-log scaling, say, and you already had to go to a large scale just to get human-equivalent intelligence, how easy is it to scale to superhuman levels of intelligence? Don't assume that scaling will be easy. It very well might not be.
softwaredoug 3y ago
Perhaps to by cynical, we have attached "Machine Learning" and "AI" to existing things we see in the industry. Increasingly defined and quantifiable. IMO buzz typically builds around that which is not well understood or defined. At least consistently by people in the industry yet. I think there's some human psychology around hype cycles (and how to manipulate them) by creating terminology that has some kind of social status, but with not clearly defined value / understanding yet yet. Once the hype cycle completes, these two converge, and it becomes boring to human minds seeking novelty (and corporations looking for growth). So eventually we will have a well understood "thing" of value called AGI. But then there will be another milestone to achieve.
- JnBrymn 3y ago
  Feels like it's a dopaminergic response to hearing a word but not knowing what it is. It's a novelty seeking thing. But once the work seems to be well understood, the novelty wears off and the novelty seeking mechanisms in humans quit responding.
tikkun 3y ago
See Table 1 on Page 6 in this paper https://arxiv.org/pdf/2311.02462.pdf https://arxiv.org/pdf/2311.02462.pdf
grantcas 3y ago
[dead]
simne 3y ago
Looks like, AGI could be just some optimization algorithm, which could survive, grow and copy itself in typical for modern human environment. So it could be just feature of environment.

What is AGI?

giorgioz 3y ago

That is a very good question. I think I read Sam Altman saying "AGI is everything we haven't done yet. Something new that will feel like magic.". Sadly, I can't find a quote and URL so I can't confirm Sam Altman said it. But the concept resonated with me. We have had Google Translate for a decade. It was and is simply a miracle of technology straight out of sci-fi books. Yet we grew quickly accustomed to it, taking it for granted. So, it's a bit the same with the concept of AGI. Now chatGPT 4.0 is the new normal and is taken for granted. So AGI is essentially anything better than ChatGPT 4.0 Frankly, I don't know if we are close to AGI. We could be years away or thousands of years away. I'll mention some past discoveries and how people reacted to put things in perspective. We went to the moon in 1969. We then dreamed we would be colonizing the galaxy. Yet space is very big and empty. If Starship lands on Mars 10 years from now, 65 years will have passed between the Moon and Mars landing. Other solar systems and Earth-like planets are much, much, much further away than Mars. It's the same in AI. We might just have landed on the moon, and we might have all these dreams about colonizing other planets (creating AGI). We might be technically very far, far away from it. We will make steady progress, and we will give names to each step of the process. AGI seems a generic name for the journey.

kingkongjaffa 3y ago
This is fine until "We will make steady progress", that's not guaranteed, in fact there's already a phrase for this https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter With space travel, there is a very well established physics to model and predict how we might do these things, you need to do some stuff like build rockets, have them escape earth's gravity, build things in space. Everything is a known known or a known unknown but we have the physics to model everything involved with space travel. This is an entirely different class of problem to "creating AGI", since there is basically a huge unknown unknown ocean to cross. So far everything we have done in AI is a statistical proxy for real thought processes, and there's one assumption that says lets make more and more of these kinds of AI techniques with "bigger" systems and we might approach something like AGI. But it isn't certain. so far with the GPT's we've really only succeeded in synthesizing human written text corpus into a statistical prediction engine, which can synthesise useful text, but does not have any semblance of 'thought'.
- JoeAltmaier 3y ago
  That changes every day. The evidence is all in favor of 'steady progress'.

spidersenses 3y ago

What about (1) reasoning and (2) unassisted learning/self-improvement? Both currently don't apply to state-of-the-art LLMs.

proc0 3y ago

AGI is a human level machine intelligence. Think about all the sci-fi movies and games with robots that can talk and understand humans perfectly, and whether they are consciousness or not is a separate question. A good example is the Gith from Mass Effect that have an entire civilization (if that's what we could call it), but their intelligence is a different and behave more like a hive mind. Another example could be C3PO or even R2D2 (given that they both understand human language, intentions, etc). > I mean surely a calculator can pass human levels of computing speed and accuracy on multiplying large numbers. You can't talk to calculators, something that humans do. Transformers (GPT, Claude) are much closer to AI that we can talk to, however it becomes evident they do not think like humans despite their impressive amount of knowledge. I would say the biggest missing piece of the puzzle with LLMs and transformers is theory of mind, or the capacity of humans to model the thinking of other humans. Once AI can model your thinking base on the context of the conversation, it will be capable of holding long conversations and provide useful, realtime insights. I think at that point we will have AGI, and we will still not have a full understanding of consciousness or whether AGI is conscious.

AnimalMuppet 3y ago
One more step: An AGI must have a theory of its own mind, not just the mind of the person it's talking to. It must be able to "watch itself think", and to think about its own thinking.
- proc0 3y ago
  Yes, that would be included in theory of mind, good point.

Staple_Diet 3y ago

>ChatGPT already has more knowledge than any individual human. It has access to the same amount of knowledge that any individual human (with internet) has. >So, are we sure 'AGI' is even a thing. I'm a cognitive neuroscientist and I'm going to say no... there's no plausible path towards creating an AGI that is a true facsimile of human intelligence BUT I'd recommend looking into P-Zombies, as there will most definitely be models that approach a believable bar, people are quick to anthropomorphizise anything. >That AI can be 'self aware' rather than just high IQ and knowledgeable? I question your references to AI having a high IQ. The IQ test is, among many things, a biased and imperfect tool. Further, it's not possible for an AI to complete the full test as one component is recall. Using IQ tests to measure AI is like comparing a car's acceleration to Olympic sprinting records. My view is that AI tools are exactly that, tools. ChatGPT is mish-mash of a thesaurus and Google. It bears remembering that ChatGPT was trained on data produced by humans, volumes of text, novels, scientific articles, blogs and reddit posts. To ascribe it intelligence for its ability to algorithmically reassemble it's training data is to do a disservice to all the authors of that original work.

philomath_mn 3y ago
> It has access to the same amount of knowledge that any individual human (with internet) has. Do you have a nuanced definition of "access"? GPT is generating most responses from its own model weights, it only reaches out to the internet in certain situations (and it tells you when it is doing so). > Further, it's not possible for an AI to complete the full test as one component is recall GPT has recall during one conversation and they recently added features to recall details from previous conversations (again, unless you mean something different by "recall") > It bears remembering that ChatGPT was trained on data produced by humans, volumes of text, novels, scientific articles, blogs and reddit posts. To ascribe it intelligence for its ability to algorithmically reassemble it's training data is to do a disservice to all the authors of that original work. I don't think GPT is conscious or anything, but I'm curious what you think the distinction is between a human child learning from its environment and a computer model learning from training data. GPT has created many truly novel outputs based on what it learned from the training data. Mechanistically, how is a brain learning from data different from a computer model learning from data?

dauertewigkeit 3y ago

The emphasis should be on "general". The easiest way to understand this is by thinking of the opposite: we can already build AI systems that are very good at one specific task: folding proteins, playing Chess, detecting cats in images, translating from one language to another, detecting market inefficiencies...etc But a human, the same human, can do all those things. A human has a sort of general intelligence. This is the goal of AGI. One can think of it as a some kind of unification theory for AI systems. One model that can be capable of excelling at vastly different tasks. AGI has nothing to do with surpassing human intelligence. However, the idea is, once you have a general intelligence running on a computer, you could easily scale it to create a super human level intelligence.

AnimalMuppet 3y ago
> However, the idea is, once you have a general intelligence running on a computer, you could easily scale it to create a super human level intelligence. I question that. I expect that "intelligence" (however defined) will not be a linear function of the scale of the system. It will be considerably less than linear. If it's log-log scaling, say, and you already had to go to a large scale just to get human-equivalent intelligence, how easy is it to scale to superhuman levels of intelligence? Don't assume that scaling will be easy. It very well might not be.

softwaredoug 3y ago

Perhaps to by cynical, we have attached "Machine Learning" and "AI" to existing things we see in the industry. Increasingly defined and quantifiable. IMO buzz typically builds around that which is not well understood or defined. At least consistently by people in the industry yet. I think there's some human psychology around hype cycles (and how to manipulate them) by creating terminology that has some kind of social status, but with not clearly defined value / understanding yet yet. Once the hype cycle completes, these two converge, and it becomes boring to human minds seeking novelty (and corporations looking for growth). So eventually we will have a well understood "thing" of value called AGI. But then there will be another milestone to achieve.

JnBrymn 3y ago
Feels like it's a dopaminergic response to hearing a word but not knowing what it is. It's a novelty seeking thing. But once the work seems to be well understood, the novelty wears off and the novelty seeking mechanisms in humans quit responding.

tikkun 3y ago

See Table 1 on Page 6 in this paper https://arxiv.org/pdf/2311.02462.pdf https://arxiv.org/pdf/2311.02462.pdf

grantcas 3y ago

[dead]

simne 3y ago

Looks like, AGI could be just some optimization algorithm, which could survive, grow and copy itself in typical for modern human environment. So it could be just feature of environment.