5 ms·

Being superhuman is the defacto definition of AGI leading to singularity, mainly self-correcting is main requirement, and that's why they fear it. If you just

Being superhuman is the defacto definition of AGI leading to singularity, mainly self-correcting is main requirement, and that's why they fear it. If you just change the definition of AGI to what GPT does currently then indeed by your definition it's AGI. But that's not what OpenAI & musk mean with AGI.

ben_w 3y ago
"""Artificial general intelligence (AGI) is a type of artificial intelligence (AI) that can perform as well or better than humans on a wide range of cognitive tasks,[1] as opposed to narrow AI, which is designed for specific tasks.""" - the opening sentence of the Wikipedia page on this. "[A]s well or better" allows "better" without requiring it. I will also note that while OpenAI is indeed clear that their definition of AGI is superhuman[0], this is not a universal position[1]. [0] "(AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work" - https://openai.com/charter https://openai.com/charter [1] "The most popular definition of AGI is the ability to perform any intellectual task that a human can do. While highly desirable, this may not align with what’s essential for achieving the singularity." - natanael.wf, Dec 2023, https://community.openai.com/t/agi-is-what-we-want-but-not-what-we-need-for-singularity/571275 https://community.openai.com/t/agi-is-what-we-want-but-not-w...
- wouldbecouldbe 3y ago
  It's what we all mean without saying it, because singularity
  - ben_w 3y ago
    Given all the people you're critisising for saying ChatGPT a precursor to AGI, can we at least agree with my opening line?: > I think part of this is that all three letters of the initialism mean different things to different people. People only look "ridiculous" to you because those who consider AGI synonymous with ASI are doing exactly what you (apparently) don't like me doing: "just chang[ing] the definition of AGI".
    - wouldbecouldbe 3y ago
      No, because if you read the other comments in this thread you will see lots of people think that human's are just Token generators as well. The might say it with more sophistication at times, but essentially a lot of technoligist are arguing that from an statistical anaylisis alone, conciousness arises if it's complex enough. Okay that's a different debate, but it all gets mingled up.
      - ben_w 3y ago
        The mechanism of our minds vs. an AI, and the question of "consciousness", are utterly irrelevant. The mechanism doesn't matter because the only thing that does matter is results ("do submarines swim?"). Even the fact that current AI require more examples compared to a human matters less than the magnitude of the difference — given the speed difference between transistors and organic synapses is on-par with "runner vs. continental drift", the limit of current GenAI models is running out of training data, which we humans don't live long enough to do. "Consciousness" doesn't matter because that word has about 40 different definitions, many so bad that tape recorders pass some of the tests and humans fail to pass others. Economically speaking, it doesn't matter if an AI (be it narrow or general or anywhere between, or on the other axis be it sub/medi/superhuman) is a P-Zombie or not. … it doesn't matter if a transformer model feels something, only if it might take my job by writing code that works well enough that the market for software developers disappears. (This may be a higher or lower standard than "is it as good as a human?" due to it being unclear at what performance level the tool saturates demand: as a toy model, if it turns out 90% of apps are a CRUD database feeding a JSON API feeding a trivial UI, I can easily imagine an LLM (or even a future version of Figma) automating all that and forcing all the devs to compete for the remaining 10% of tasks and there's no guarantee whether or not the remaining 10% of tasks will expand to fit the supply of developers). … it doesn't matter if a diffusion model has a love of art, is inspired by the beauty of nature, genuinely empathises with newlyweds who ask for their alter photo to be transformed into a renaissance oil painting; I imagine that artists only care if demand for their services has gone down (see all the artist saying both that it threatens their livelihood and that it sucks). … it doesn't matter if a business agent has a monomaniacal focus on profits or if it has side-interests, hobbies, and a pet giraffe; the only thing that matters is if the business may harm my interests. (See any discussion along the lines "are corporations a type of AI? And are they aligned?" regardless of how you feel about terminology). … it doesn't matter if the Google search engine experiences joy when I click on a result it provides, just that it gets me results. … it doesn't matter if Deep Blue or AlphaZero enjoy winning, just that they did. There are two ways "consciousness" (in the sense of qualia) matter to me, but they are not performance issues: 1) Brain uploads, where anyone who wants to use it to circumvent death wants consciousness to be present 2) Slavery: Will this economic value be an accidental re-invention of slavery, where we use exactly the same "they're not really people" arguments to avoid the cognitive dissonance of how we treat conscious beings that we own, create, and destroy on a whim. These are ethical issues, which are not at all relevant to the question of "what do we need to get human-level capabilities?", any more than Eli Whitney needed to ask "what's the ethical aspect of this?" to be capable of actually inventing his cotton gin.

ben_w 3y ago

"""Artificial general intelligence (AGI) is a type of artificial intelligence (AI) that can perform as well or better than humans on a wide range of cognitive tasks,[1] as opposed to narrow AI, which is designed for specific tasks.""" - the opening sentence of the Wikipedia page on this. "[A]s well or better" allows "better" without requiring it. I will also note that while OpenAI is indeed clear that their definition of AGI is superhuman[0], this is not a universal position[1]. [0] "(AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work" - https://openai.com/charter https://openai.com/charter [1] "The most popular definition of AGI is the ability to perform any intellectual task that a human can do. While highly desirable, this may not align with what’s essential for achieving the singularity." - natanael.wf, Dec 2023, https://community.openai.com/t/agi-is-what-we-want-but-not-what-we-need-for-singularity/571275 https://community.openai.com/t/agi-is-what-we-want-but-not-w...

wouldbecouldbe 3y ago
It's what we all mean without saying it, because singularity
- ben_w 3y ago
  Given all the people you're critisising for saying ChatGPT a precursor to AGI, can we at least agree with my opening line?: > I think part of this is that all three letters of the initialism mean different things to different people. People only look "ridiculous" to you because those who consider AGI synonymous with ASI are doing exactly what you (apparently) don't like me doing: "just chang[ing] the definition of AGI".
  - wouldbecouldbe 3y ago
    No, because if you read the other comments in this thread you will see lots of people think that human's are just Token generators as well. The might say it with more sophistication at times, but essentially a lot of technoligist are arguing that from an statistical anaylisis alone, conciousness arises if it's complex enough. Okay that's a different debate, but it all gets mingled up.
    - ben_w 3y ago
      The mechanism of our minds vs. an AI, and the question of "consciousness", are utterly irrelevant. The mechanism doesn't matter because the only thing that does matter is results ("do submarines swim?"). Even the fact that current AI require more examples compared to a human matters less than the magnitude of the difference — given the speed difference between transistors and organic synapses is on-par with "runner vs. continental drift", the limit of current GenAI models is running out of training data, which we humans don't live long enough to do. "Consciousness" doesn't matter because that word has about 40 different definitions, many so bad that tape recorders pass some of the tests and humans fail to pass others. Economically speaking, it doesn't matter if an AI (be it narrow or general or anywhere between, or on the other axis be it sub/medi/superhuman) is a P-Zombie or not. … it doesn't matter if a transformer model feels something, only if it might take my job by writing code that works well enough that the market for software developers disappears. (This may be a higher or lower standard than "is it as good as a human?" due to it being unclear at what performance level the tool saturates demand: as a toy model, if it turns out 90% of apps are a CRUD database feeding a JSON API feeding a trivial UI, I can easily imagine an LLM (or even a future version of Figma) automating all that and forcing all the devs to compete for the remaining 10% of tasks and there's no guarantee whether or not the remaining 10% of tasks will expand to fit the supply of developers). … it doesn't matter if a diffusion model has a love of art, is inspired by the beauty of nature, genuinely empathises with newlyweds who ask for their alter photo to be transformed into a renaissance oil painting; I imagine that artists only care if demand for their services has gone down (see all the artist saying both that it threatens their livelihood and that it sucks). … it doesn't matter if a business agent has a monomaniacal focus on profits or if it has side-interests, hobbies, and a pet giraffe; the only thing that matters is if the business may harm my interests. (See any discussion along the lines "are corporations a type of AI? And are they aligned?" regardless of how you feel about terminology). … it doesn't matter if the Google search engine experiences joy when I click on a result it provides, just that it gets me results. … it doesn't matter if Deep Blue or AlphaZero enjoy winning, just that they did. There are two ways "consciousness" (in the sense of qualia) matter to me, but they are not performance issues: 1) Brain uploads, where anyone who wants to use it to circumvent death wants consciousness to be present 2) Slavery: Will this economic value be an accidental re-invention of slavery, where we use exactly the same "they're not really people" arguments to avoid the cognitive dissonance of how we treat conscious beings that we own, create, and destroy on a whim. These are ethical issues, which are not at all relevant to the question of "what do we need to get human-level capabilities?", any more than Eli Whitney needed to ask "what's the ethical aspect of this?" to be capable of actually inventing his cotton gin.