4 ms·

Nowhere do they define "AGI". I guarantee that is a big reason why the predictions have so much variance. For many people, what GPT-4 does qualified as AGI --

by ilaksh 3y ago

Nowhere do they define "AGI". I guarantee that is a big reason why the predictions have so much variance. For many people, what GPT-4 does qualified as AGI -- up until GPT-4 came out and then everyone seemed to decide that AGI meant ASI. I am guessing for many people answering this poll it means "a full emulation of a person". Or maybe it had to be "alive". The thing that irritates me so much is that there is this lack of definition or moving of goalposts and also stupidly people seem to assume that AI can't be general purpose or useful unless it has all of those animal/human characteristics. GPT-4 is very general. Make it say 10-20 times faster, open up the image modality to the public, and you will be able to do most human tasks with it. You don't need to invent a lot of other stuff to be general purpose. You DO need to invent a lot of other stuff to become some kind of humanlike digital god. But that is not the least bit necessary to accomplish most human work output.

davidmanheim 3y ago
> Nowhere do they define "AGI" Ummm, maybe you should have looked? At the top of the very first prediction, here: https://www.metaculus.com/questions/5121/date-of-artificial-general-intelligence/ https://www.metaculus.com/questions/5121/date-of-artificial-... We will thus define "an AI system" as a single unified software system that can satisfy the following criteria, all completable by at least some humans. Able to reliably pass a 2-hour, adversarial Turing test during which the participants can send text, images, and audio files (as is done in ordinary text messaging applications) during the course of their conversation. An 'adversarial' Turing test is one in which the human judges are instructed to ask interesting and difficult questions, designed to advantage human participants, and to successfully unmask the computer as an impostor. A single demonstration of an AI passing such a Turing test, or one that is sufficiently similar, will be sufficient for this condition, so long as the test is well-designed to the estimation of Metaculus Admins. Has general robotic capabilities, of the type able to autonomously, when equipped with appropriate actuators and when given human-readable instructions, satisfactorily assemble a (or the equivalent of a) circa-2021 Ferrari 312 T4 1:8 scale automobile model. A single demonstration of this ability, or a sufficiently similar demonstration, will be considered sufficient. High competency at a diverse fields of expertise, as measured by achieving at least 75% accuracy in every task and 90% mean accuracy across all tasks in the Q&A dataset developed by Dan Hendrycks et al.. Able to get top-1 strict accuracy of at least 90.0% on interview-level problems found in the APPS benchmark introduced by Dan Hendrycks, Steven Basart et al. Top-1 accuracy is distinguished, as in the paper, from top-k accuracy in which k outputs from the model are generated, and the best output is selected. By "unified" we mean that the system is integrated enough that it can, for example, explain its reasoning on a Q&A task, or verbally report its progress and identify objects during model assembly. (This is not really meant to be an additional capability of "introspection" so much as a provision that the system not simply be cobbled together as a set of sub-systems specialized to tasks like the above, but rather a single system applicable to many problems.) Resolution will come from any of three forms, whichever comes first: (1) direct demonstration of such a system achieving ALL of the above criteria, (2) confident credible statement by its developers that an existing system is able to satisfy these criteria, or (3) judgement by a majority vote in a special committee composed of the question author and two AI experts chosen in good faith by him, for the sole purpose of resolving this question. Resolution date will be the first date at which the system (subsequently judged to satisfy the criteria) and its capabilities are publicly described in a talk, press release, paper, or other report available to the general public.
- gpderetta 3y ago
  You do realize that many, if not most, humans would fail this test, right?
  - TheOtherHobbes 3y ago
    AGI is competing with human culture as a whole. Individual humans are not exactly the best of all possible tests for AGI.
  - kayodelycaon 3y ago
    Yes, but humans as a group can do it. An AGI needs to show a similar number of AGIs can do the same given the same starting template. The AGI will need to look at all of the tasks written, determine what the success criteria is, and then combine that that into a single set of answers. With the instructions in human-readable form, not machine readable. It can use as many or as few AGIs as it needs to accomplish this. It's the same as if we gave these instructions to a human with sufficient skill and resources to delegate.
  - famouswaffles 3y ago
    Yes you'll find that any testable definition of AGI that has not been passed yet would be unpassable for a big chunk of the human population. In other words, General, Artificial and Intelligent have been passed. That's why a few papers/researchers opt to call these models "General Artificial Intelligence" instead https://jamanetwork.com/journals/jama/article-abstract/2806457 https://jamanetwork.com/journals/jama/article-abstract/28064... https://arxiv.org/abs/2303.12003 https://arxiv.org/abs/2303.12003 Or some such variant like "General Purpose Technologies" as Open AI did. https://arxiv.org/abs/2303.10130 https://arxiv.org/abs/2303.10130 since "AGI" has so much baggage with posts shifting at the speed of light.
- sdwr 3y ago
  I think we're reaching a point where the Turing test is no longer useful. If you get into the nitty-gritty of it (instead of just handwaving "computer should act like person"), it's about roleplaying a fake identity. Which is a specific skill, not a general test of competence.
jiggywiggy 3y ago
Chatgpt 4 is amazing. Still cant comprehend they build this. However if you work with it a lot you realize it doesn't understand reality, it predicts language. It reminds me of YouTube videos of chess when human start to figure out chess bots and the tricks they use to draw out time. Understanding language is super impressive, but a far cry from general intelligence. Maybe from here open ai can build further. But I have a hard time believing that this will be a foundation for general intelligence. A lot of animals have a form of general intelligence but cant do math or language. Yet in some ways are more capable then the latest self driving cars.
- _ea1k 3y ago
  Is it surprising that a system that was exclusively trained on emitted language will only do well on language? I don't see how I can extrapolate to anything beyond that. I certainly can't extrapolate that it would be unable to learn other tasks.
  - bg24 3y ago
    Everything is defined in language. As a system is able to learn from the corpus of texts, it also builds reasoning to a large extent. For example, GPT4 can guide you through solving a problem like building a flying and crawling drone for inspecting an attic. Or, watering plants. So this goes beyond language, unless I missed your point.
    - akomtu 3y ago
      Not everything. For example, how did Wagner create his music? The ability to appreciate beauty stands above reasoning. But your point largely stands, as langugage processing alone allows to build a "linear thinking machine" with super human reasoning ability. It won't be able to make music or art, but it will create a spaceship and take science to the next level. As a side effect, such AI will erase all art, as art will look like noise in its "mind".
    - _ea1k 3y ago
      Karpathy explains it far better than I did in his "State of GPT" talk about a month ago: https://www.youtube.com/watch?v=bZQun8Y4L2A https://www.youtube.com/watch?v=bZQun8Y4L2A Yes, language can do that, but books and other texts, on average, leave out a lot of steps that humans have learned to do naturally. They can be prompted to do them, and their answers improve, indicates that there is definitely some level of "reasoning" there. But it has weird limitations in its current forms. I expect that to improve pretty quickly, tbh.
sublinear 3y ago
> you will be able to do most human tasks with it. You don't need to invent a lot of other stuff to be general purpose. I think this is where most people strongly disagree with you. A probabilistic language model is not good enough to do anything requiring context particularly well.
- cwkoss 3y ago
  Agentic uses of GPT can solve the context problem by breaking problems into steps and building prompts to solve those steps. Can't do everything, but GPT+APIs can do a lot.
  - namaria 3y ago
    What in Chomsk's name is "agentic"?
akomtu 3y ago
One argument against ChatGPT's AI-ness was that it's wildly unstable: a small change in the input may become a big change in the output. When that's fixed, GPT will still need memory to learn things, as AI that can't learn isn't really AI. However the memory can be added easily: it's just a hidden step after every prompt when GPT is asked to update it's memory state (a blob of text, its scratchpad). If GPT is stable, it will update the memory more or less properly. With these two changes, and they are fairly simple, GPT5 will earn its AI badge. P.S. I find it interesting that eye and AI sound similarly, so perhaps the symbol of the future AI deity will be an eye.
Aerroon 3y ago
In my opinion a critical part of an AGI is learning on the fly from few examples. Humans do this well. When humans play (video)games against an AI they will usually find some pattern of behavior that the AI falls into. Once they find it, they can continuously take advantage of this pattern of behavior and abuse it for great success. I think current LLMs still fall into this category. But I think they might be able to handle this relatively well quite soon.

by ilaksh 3y ago

davidmanheim 3y ago

> Nowhere do they define "AGI" Ummm, maybe you should have looked? At the top of the very first prediction, here: https://www.metaculus.com/questions/5121/date-of-artificial-general-intelligence/ https://www.metaculus.com/questions/5121/date-of-artificial-... We will thus define "an AI system" as a single unified software system that can satisfy the following criteria, all completable by at least some humans. Able to reliably pass a 2-hour, adversarial Turing test during which the participants can send text, images, and audio files (as is done in ordinary text messaging applications) during the course of their conversation. An 'adversarial' Turing test is one in which the human judges are instructed to ask interesting and difficult questions, designed to advantage human participants, and to successfully unmask the computer as an impostor. A single demonstration of an AI passing such a Turing test, or one that is sufficiently similar, will be sufficient for this condition, so long as the test is well-designed to the estimation of Metaculus Admins. Has general robotic capabilities, of the type able to autonomously, when equipped with appropriate actuators and when given human-readable instructions, satisfactorily assemble a (or the equivalent of a) circa-2021 Ferrari 312 T4 1:8 scale automobile model. A single demonstration of this ability, or a sufficiently similar demonstration, will be considered sufficient. High competency at a diverse fields of expertise, as measured by achieving at least 75% accuracy in every task and 90% mean accuracy across all tasks in the Q&A dataset developed by Dan Hendrycks et al.. Able to get top-1 strict accuracy of at least 90.0% on interview-level problems found in the APPS benchmark introduced by Dan Hendrycks, Steven Basart et al. Top-1 accuracy is distinguished, as in the paper, from top-k accuracy in which k outputs from the model are generated, and the best output is selected. By "unified" we mean that the system is integrated enough that it can, for example, explain its reasoning on a Q&A task, or verbally report its progress and identify objects during model assembly. (This is not really meant to be an additional capability of "introspection" so much as a provision that the system not simply be cobbled together as a set of sub-systems specialized to tasks like the above, but rather a single system applicable to many problems.) Resolution will come from any of three forms, whichever comes first: (1) direct demonstration of such a system achieving ALL of the above criteria, (2) confident credible statement by its developers that an existing system is able to satisfy these criteria, or (3) judgement by a majority vote in a special committee composed of the question author and two AI experts chosen in good faith by him, for the sole purpose of resolving this question. Resolution date will be the first date at which the system (subsequently judged to satisfy the criteria) and its capabilities are publicly described in a talk, press release, paper, or other report available to the general public.

gpderetta 3y ago
You do realize that many, if not most, humans would fail this test, right?
- TheOtherHobbes 3y ago
  AGI is competing with human culture as a whole. Individual humans are not exactly the best of all possible tests for AGI.
- kayodelycaon 3y ago
  Yes, but humans as a group can do it. An AGI needs to show a similar number of AGIs can do the same given the same starting template. The AGI will need to look at all of the tasks written, determine what the success criteria is, and then combine that that into a single set of answers. With the instructions in human-readable form, not machine readable. It can use as many or as few AGIs as it needs to accomplish this. It's the same as if we gave these instructions to a human with sufficient skill and resources to delegate.
- famouswaffles 3y ago
  Yes you'll find that any testable definition of AGI that has not been passed yet would be unpassable for a big chunk of the human population. In other words, General, Artificial and Intelligent have been passed. That's why a few papers/researchers opt to call these models "General Artificial Intelligence" instead https://jamanetwork.com/journals/jama/article-abstract/2806457 https://jamanetwork.com/journals/jama/article-abstract/28064... https://arxiv.org/abs/2303.12003 https://arxiv.org/abs/2303.12003 Or some such variant like "General Purpose Technologies" as Open AI did. https://arxiv.org/abs/2303.10130 https://arxiv.org/abs/2303.10130 since "AGI" has so much baggage with posts shifting at the speed of light.
sdwr 3y ago
I think we're reaching a point where the Turing test is no longer useful. If you get into the nitty-gritty of it (instead of just handwaving "computer should act like person"), it's about roleplaying a fake identity. Which is a specific skill, not a general test of competence.

jiggywiggy 3y ago

Chatgpt 4 is amazing. Still cant comprehend they build this. However if you work with it a lot you realize it doesn't understand reality, it predicts language. It reminds me of YouTube videos of chess when human start to figure out chess bots and the tricks they use to draw out time. Understanding language is super impressive, but a far cry from general intelligence. Maybe from here open ai can build further. But I have a hard time believing that this will be a foundation for general intelligence. A lot of animals have a form of general intelligence but cant do math or language. Yet in some ways are more capable then the latest self driving cars.

_ea1k 3y ago
Is it surprising that a system that was exclusively trained on emitted language will only do well on language? I don't see how I can extrapolate to anything beyond that. I certainly can't extrapolate that it would be unable to learn other tasks.
- bg24 3y ago
  Everything is defined in language. As a system is able to learn from the corpus of texts, it also builds reasoning to a large extent. For example, GPT4 can guide you through solving a problem like building a flying and crawling drone for inspecting an attic. Or, watering plants. So this goes beyond language, unless I missed your point.
  - akomtu 3y ago
    Not everything. For example, how did Wagner create his music? The ability to appreciate beauty stands above reasoning. But your point largely stands, as langugage processing alone allows to build a "linear thinking machine" with super human reasoning ability. It won't be able to make music or art, but it will create a spaceship and take science to the next level. As a side effect, such AI will erase all art, as art will look like noise in its "mind".
  - _ea1k 3y ago
    Karpathy explains it far better than I did in his "State of GPT" talk about a month ago: https://www.youtube.com/watch?v=bZQun8Y4L2A https://www.youtube.com/watch?v=bZQun8Y4L2A Yes, language can do that, but books and other texts, on average, leave out a lot of steps that humans have learned to do naturally. They can be prompted to do them, and their answers improve, indicates that there is definitely some level of "reasoning" there. But it has weird limitations in its current forms. I expect that to improve pretty quickly, tbh.

sublinear 3y ago

> you will be able to do most human tasks with it. You don't need to invent a lot of other stuff to be general purpose. I think this is where most people strongly disagree with you. A probabilistic language model is not good enough to do anything requiring context particularly well.

cwkoss 3y ago
Agentic uses of GPT can solve the context problem by breaking problems into steps and building prompts to solve those steps. Can't do everything, but GPT+APIs can do a lot.
- namaria 3y ago
  What in Chomsk's name is "agentic"?

akomtu 3y ago

One argument against ChatGPT's AI-ness was that it's wildly unstable: a small change in the input may become a big change in the output. When that's fixed, GPT will still need memory to learn things, as AI that can't learn isn't really AI. However the memory can be added easily: it's just a hidden step after every prompt when GPT is asked to update it's memory state (a blob of text, its scratchpad). If GPT is stable, it will update the memory more or less properly. With these two changes, and they are fairly simple, GPT5 will earn its AI badge. P.S. I find it interesting that eye and AI sound similarly, so perhaps the symbol of the future AI deity will be an eye.

Aerroon 3y ago

In my opinion a critical part of an AGI is learning on the fly from few examples. Humans do this well. When humans play (video)games against an AI they will usually find some pattern of behavior that the AI falls into. Once they find it, they can continuously take advantage of this pattern of behavior and abuse it for great success. I think current LLMs still fall into this category. But I think they might be able to handle this relatively well quite soon.