4 ms·

The Last Week in AI

59 points by nofilk 4y ago

whoisthis12 4y ago
People keep mentioning we will have agi in our lifetimes but do we have any proper definition of agi? Are there any benchmarks to prove it?
- jasfi 4y ago
  AGI can solve problems in any general context using its capabilities, including problems never seen before. There's weak and strong AGI. Right now I'd argue we have weak AGI with ChatGPT and its plug-ins. The GPT-tech allows for general purpose problem solving, but it often makes mistakes and gets things wrong. We are on the path towards strong AGI, and I have no idea how long it will take to get there. OpenAI and others are developing benchmarks to test this tech against various problems.
  - keyle 4y ago
    I think personally we should also classify human level AGI and smart AGI. After all, the goal is not to make fake humans but to solve problems like cancer.
    - jasfi 4y ago
      After watching how people are applying GPT3.5 and GPT4, I'd say both goals those types of goals will be attempted, and anything else that could be of use.
  - gremlinsinc 4y ago
    I wouldn't consider chatGPT by itself or with plugins AGI, however the langchain+long-term memory + automated agents, yeah that's f'n AGI at least as it's aptly called: Baby AGI.
    - roflyear 4y ago
      It might be baby agi but as it stands I would argue it is not.
      - gremlinsinc 4y ago
        you realize, babies aren't as impressive as a PhD. right? I mean they're cute and impressive in how they learn and pick things up, baby AGI is kinda like that. the hype is basically going gaga around a baby. but it will grow up and before you know it.
        roflyear 4y ago
        How could I not realize those things? Do you think I'm a moron?
- archon1410 4y ago
  The resolution criteria for a Metaculus question[1] attempts to explain what a "weakly" general AI might entail. The question will be obsolete and the definition of AGI might need shifting if some things turn out more or less difficult than previously assumed though. "For these purposes we will thus define "AI system" as a single unified software system that can satisfy the following criteria, all easily completable by a typical college-educated human. Able to reliably pass a Turing test of the type that would win the Loebner Silver Prize. Able to score 90% or more on a robust version of the Winograd Schema Challenge, e.g. the "Winogrande" challenge or comparable data set for which human performance is at 90+% Be able to score 75th percentile (as compared to the corresponding year's human students; this was a score of 600 in 2016) on all the full mathematics section of a circa-2015-2020 standard SAT exam, using just images of the exam pages and having less than ten SAT exams as part of the training data. (Training on other corpuses of math problems is fair game as long as they are arguably distinct from SAT exams.) Be able to learn the classic Atari game "Montezuma's revenge" (based on just visual inputs and standard controls) and explore all 24 rooms based on the equivalent of less than 100 hours of real-time play (see closely-related question.)" [1]https://www.metaculus.com/questions/3479/date-weakly-general-ai-is-publicly-known/ https://www.metaculus.com/questions/3479/date-weakly-general...
- iKevinShah 4y ago
  This is just my opinion. The thing is - our ability is limited to our understanding. AI maybe already doing things we do not understand (which could be classified as AGI). Thinking of it like how dogs do not have the cognitive ability to understand the concept of the "future" or tomorrow - There is a good chance AI would already be doing things which are beyond our cognitive ability (including the smartest people working on the tech.) But I am sure if we let 2 fairly good LLMs talk to each other - They'd shortly start talking things we feel are hallucinations but the 2 LLMs would understand and take it further. Again, this is just my opinion and I have never worked on any LLMs. So an outsider.
  - lm28469 4y ago
    > but the 2 LLMs would understand and take it further. Some kind of future AI maybe but LLMs are not designed for that and structurally incapable of that
- QuadmasterXLII 4y ago
  The simple definition is “its faster for a manager with medium computer skills to spin up an instance than to train a human for almost all desk jobs.” We look for other definitions because if an AI satisfying this definition showed up tomorrow, we’d all starve.
- more_corn 4y ago
  I’m not sure we have a proper definition of consciousness, so probably no. But I suspect we’re five years from being able to create a self-directed simulation of a person that could behave in ways we consider autonomous. That is to say it will be able to fool most people observing from the outside. I have some ideas on how I’d go about it so I imagine better minds than mine are hard at work on the problem. I’m not sure we should be building that, but I’m sure we are.
- kolinko 4y ago
  An AGI will be able to improve itself - i.e. all the AI dev team at OpenAI will get fired, and only people policy/alignment people will remain (hopefully).
amilios 4y ago
https://www.reddit.com/r/OpenAI/comments/12696oq/im_dating_a_chatbot_trained_on_old_conversations/ https://www.reddit.com/r/OpenAI/comments/12696oq/im_dating_a... This one is wild. I expected that Black Mirror episode to become real eventually, but definitely not so soon...
- brumar 4y ago
  On the same subject, a great sci-fi audio drama named LifeAfter gets increasingly relevant.
- nofilk 4y ago
  This isn't even as bad as all the other things out there already. We're going to have a term for someone who's in a relationship with an AI soon enough
  - roflyear 4y ago
    Someone was already devastated when their chat got erased with their gpt therapist....
  - Gigachad 4y ago
    Take a look at what happened with Replika. The company restricted the bots ability to have sexual conversations and we had hundreds of people posting on reddit like the company had murdered their girlfriend.
lionkor 4y ago
The only thing I can think of when I read posts like that is the early days when crypto started to take off. Its near the peak of the hype, but not quite there yet. People may make drastic life changes due to AI soon. People will keep claiming its the future, and everyone else is just blind. It'll be compared to the industrial revolution (or similar). It may take a while, and AI will always be around in products, but the hype will die down, and it'll be just another piece of tech. And all these silly hype people, with there "INSANE CRAZY THINGS IN AI THIS WEEK!!! TRULY CRAZY!" will move on to the next well-marketed mediocre tech idea. Dont compare this to smartphones, or to the internet. LLMs are more like the invention of the "makeup application robot", or the invention of those weird animatronics that look super realistic. I wonder when the discussion about the climate impact of LLMs will start - just to draw more parallels to the overhyped crypto bubble. Im glad you guys are having fun, I guess. Just dont make drastic life changes based on LLMs existing.
- gremlinsinc 4y ago
  Except, crypto honestly seemed like it had utility but really didn't... I've had this thing just create pretty decent youtube shorts with zero knowledge or very little advice on how to do it, and also tweets, instagram posts, etc... it's now working on creating direct to etsy store items from midjourney using 'tiled layouts', digital downloads for paper/etc... The people in charge of both of these projects are using it to also actually continually improve itself. The one dude isn't a developer and doesn't know how to code, he's a VC. That's insane by itself. I've been through hype trains, but this is more foundational/bigger than anything we've experienced. Even without AGI we could create a single software that can deliver any layout a user wants, any outputs, from any inputs... it's an accounting software, ERP, CRM, Paintshop, etc... everything one app, because you don't even need to write functions, simply writing the function name and it's attributes is enough for the AI to figure it out.
  - lionkor 4y ago
    I just see zero actual value in something anyone can make, like midjourney art.
roflyear 4y ago
Saying the stuff autogpt is going to create are going to be world changing... Ok...

59 points by nofilk 4y ago

People keep mentioning we will have agi in our lifetimes but do we have any proper definition of agi? Are there any benchmarks to prove it?

jasfi 4y ago
AGI can solve problems in any general context using its capabilities, including problems never seen before. There's weak and strong AGI. Right now I'd argue we have weak AGI with ChatGPT and its plug-ins. The GPT-tech allows for general purpose problem solving, but it often makes mistakes and gets things wrong. We are on the path towards strong AGI, and I have no idea how long it will take to get there. OpenAI and others are developing benchmarks to test this tech against various problems.
- keyle 4y ago
  I think personally we should also classify human level AGI and smart AGI. After all, the goal is not to make fake humans but to solve problems like cancer.
  - jasfi 4y ago
    After watching how people are applying GPT3.5 and GPT4, I'd say both goals those types of goals will be attempted, and anything else that could be of use.
- gremlinsinc 4y ago
  I wouldn't consider chatGPT by itself or with plugins AGI, however the langchain+long-term memory + automated agents, yeah that's f'n AGI at least as it's aptly called: Baby AGI.
  - roflyear 4y ago
    It might be baby agi but as it stands I would argue it is not.
    - gremlinsinc 4y ago
      you realize, babies aren't as impressive as a PhD. right? I mean they're cute and impressive in how they learn and pick things up, baby AGI is kinda like that. the hype is basically going gaga around a baby. but it will grow up and before you know it.
      - roflyear 4y ago
        How could I not realize those things? Do you think I'm a moron?
archon1410 4y ago
The resolution criteria for a Metaculus question[1] attempts to explain what a "weakly" general AI might entail. The question will be obsolete and the definition of AGI might need shifting if some things turn out more or less difficult than previously assumed though. "For these purposes we will thus define "AI system" as a single unified software system that can satisfy the following criteria, all easily completable by a typical college-educated human. Able to reliably pass a Turing test of the type that would win the Loebner Silver Prize. Able to score 90% or more on a robust version of the Winograd Schema Challenge, e.g. the "Winogrande" challenge or comparable data set for which human performance is at 90+% Be able to score 75th percentile (as compared to the corresponding year's human students; this was a score of 600 in 2016) on all the full mathematics section of a circa-2015-2020 standard SAT exam, using just images of the exam pages and having less than ten SAT exams as part of the training data. (Training on other corpuses of math problems is fair game as long as they are arguably distinct from SAT exams.) Be able to learn the classic Atari game "Montezuma's revenge" (based on just visual inputs and standard controls) and explore all 24 rooms based on the equivalent of less than 100 hours of real-time play (see closely-related question.)" [1]https://www.metaculus.com/questions/3479/date-weakly-general-ai-is-publicly-known/ https://www.metaculus.com/questions/3479/date-weakly-general...
iKevinShah 4y ago
This is just my opinion. The thing is - our ability is limited to our understanding. AI maybe already doing things we do not understand (which could be classified as AGI). Thinking of it like how dogs do not have the cognitive ability to understand the concept of the "future" or tomorrow - There is a good chance AI would already be doing things which are beyond our cognitive ability (including the smartest people working on the tech.) But I am sure if we let 2 fairly good LLMs talk to each other - They'd shortly start talking things we feel are hallucinations but the 2 LLMs would understand and take it further. Again, this is just my opinion and I have never worked on any LLMs. So an outsider.
- lm28469 4y ago
  > but the 2 LLMs would understand and take it further. Some kind of future AI maybe but LLMs are not designed for that and structurally incapable of that
QuadmasterXLII 4y ago
The simple definition is “its faster for a manager with medium computer skills to spin up an instance than to train a human for almost all desk jobs.” We look for other definitions because if an AI satisfying this definition showed up tomorrow, we’d all starve.
more_corn 4y ago
I’m not sure we have a proper definition of consciousness, so probably no. But I suspect we’re five years from being able to create a self-directed simulation of a person that could behave in ways we consider autonomous. That is to say it will be able to fool most people observing from the outside. I have some ideas on how I’d go about it so I imagine better minds than mine are hard at work on the problem. I’m not sure we should be building that, but I’m sure we are.
kolinko 4y ago
An AGI will be able to improve itself - i.e. all the AI dev team at OpenAI will get fired, and only people policy/alignment people will remain (hopefully).

amilios 4y ago

https://www.reddit.com/r/OpenAI/comments/12696oq/im_dating_a_chatbot_trained_on_old_conversations/ https://www.reddit.com/r/OpenAI/comments/12696oq/im_dating_a... This one is wild. I expected that Black Mirror episode to become real eventually, but definitely not so soon...

brumar 4y ago
On the same subject, a great sci-fi audio drama named LifeAfter gets increasingly relevant.
nofilk 4y ago
This isn't even as bad as all the other things out there already. We're going to have a term for someone who's in a relationship with an AI soon enough
- roflyear 4y ago
  Someone was already devastated when their chat got erased with their gpt therapist....
- Gigachad 4y ago
  Take a look at what happened with Replika. The company restricted the bots ability to have sexual conversations and we had hundreds of people posting on reddit like the company had murdered their girlfriend.

lionkor 4y ago

The only thing I can think of when I read posts like that is the early days when crypto started to take off. Its near the peak of the hype, but not quite there yet. People may make drastic life changes due to AI soon. People will keep claiming its the future, and everyone else is just blind. It'll be compared to the industrial revolution (or similar). It may take a while, and AI will always be around in products, but the hype will die down, and it'll be just another piece of tech. And all these silly hype people, with there "INSANE CRAZY THINGS IN AI THIS WEEK!!! TRULY CRAZY!" will move on to the next well-marketed mediocre tech idea. Dont compare this to smartphones, or to the internet. LLMs are more like the invention of the "makeup application robot", or the invention of those weird animatronics that look super realistic. I wonder when the discussion about the climate impact of LLMs will start - just to draw more parallels to the overhyped crypto bubble. Im glad you guys are having fun, I guess. Just dont make drastic life changes based on LLMs existing.

gremlinsinc 4y ago
Except, crypto honestly seemed like it had utility but really didn't... I've had this thing just create pretty decent youtube shorts with zero knowledge or very little advice on how to do it, and also tweets, instagram posts, etc... it's now working on creating direct to etsy store items from midjourney using 'tiled layouts', digital downloads for paper/etc... The people in charge of both of these projects are using it to also actually continually improve itself. The one dude isn't a developer and doesn't know how to code, he's a VC. That's insane by itself. I've been through hype trains, but this is more foundational/bigger than anything we've experienced. Even without AGI we could create a single software that can deliver any layout a user wants, any outputs, from any inputs... it's an accounting software, ERP, CRM, Paintshop, etc... everything one app, because you don't even need to write functions, simply writing the function name and it's attributes is enough for the AI to figure it out.
- lionkor 4y ago
  I just see zero actual value in something anyone can make, like midjourney art.

roflyear 4y ago

Saying the stuff autogpt is going to create are going to be world changing... Ok...