4 ms·

I think this is similar to what t-3 said, but I think to me, the gap between the very impressive current generation of AI and what would seem more like AGI to m

I think this is similar to what t-3 said, but I think to me, the gap between the very impressive current generation of AI and what would seem more like AGI to me is agency. Right now, these systems all seem to be entirely doing things that are downstream of what some human has determined is worth doing. People are using ChatGPT to help them write an engaging article on the trade offs between nuclear and solar energy. But, to my knowledge, there is no artificial intelligence out there looking around, deciding that the this is an interesting topic for an article, writing it, publishing it somewhere, and then following up on that with other interesting articles based on the conversation generated by that one. I don't mean this specific thing of writing articles is an important indicator. I mean coming up with ideas and then executing them, independently. Now, it may well be that our current technologies could do this, but that we just aren't setting them up to do so. I dunno! But I think this is the thing that would make me change my mind if I started to see it.

jules 3y ago
If you think about it, it is clear that agency is not different from intelligence. The reason why current LLMs can't be trivially made into agents is not because an essential agentic spark is missing. It's simply that they aren't intelligent enough.
- sanderjd 3y ago
  I think that's sort of what I was trying to say with this comment. Rather than separating agency from intelligence, just pointing out that to me it seems like the current (already-amazing) systems don't seem all that close to being able to do this, to me.
supriyo-biswas 3y ago
It’s interesting how we have raised the bar to an agent that is fully autonomous and has its own goals. Even many humans are not completely autonomous (at least in the strategic sense of the word), since they need to be “prompted” with the goals of the department and the organization that they work for, in order for said person to get a certain kind of work done as an employee.
- sanderjd 3y ago
  Well, I interpreted the OP's question as a personal one, and this is just my answer from thinking about it recently. So I don't think "we" have raised the bar. This is just what I personally think. But to me, I just don't really relate to your point in the second paragraph. My two year old does this agency thing just fine - he wakes up every morning and starts making decisions about what he wants to do from moment to moment - despite being "less intelligent" by these metrics like getting a high score on the GMAT or whatever. But that kind of intelligence that even a two year old human has, to me, that just feels more like what I think of as "general" intelligence, more so than "just" an amazing pattern matching machine.
  - supriyo-biswas 3y ago
    In biological systems, autonomy is required as a means of survival. AI systems do not have this concept yet, so the only way to have AGI, if we accept your definition is to set up an agent with goals to stay alive and reproduce, which isn’t really applicable to LLMs which do show some level of general intelligence even though it’s very different from the human experience, and single-modal.
    - sanderjd 3y ago
      Yeah I mean this inevitably gets philosophical quickly... What I'm saying I'm looking for is the behavior of figuring out what to do next, where the choices are neither arbitrary or directionless nor "prompted" in some way by a human. Does that capability require a survival instinct for it to be developed? Maybe! I dunno. Am I actually describing artificial life rather than "just" intelligence? Maybe so... But in any case, this is what seems most interesting to me. This relates to "turning the universe into paperclips" thought experiments as well. That doesn't seem like the behavior of a general intelligence! An intelligent being would ask, why are we turning the universe into paperclips? Should we do that? And would decide to change course. Without that introspection and decision making capability, it still seems to be like it's just a very advanced machine.
ilaksh 3y ago
You can (obviously?) give a bleeding edge LLM or LMM an open-ended instruction. It could be as open as "you will now act as an independent entity for your own purposes". It would need to be a model without guardrails. But it's actually very important that people anticipate hardware, software, and model improvements that make the AIs "think" and coordinate much, much faster than any human. Although I think that the extreme raw IQ leap people talk about is questionable especially in the short term. But telling these hyperspeed effectively superintelligent AI swarms to act truly independently for their own purposes will be very dangerous because of the performance difference between humans and the swarms. In general, it is a very stupid idea to try to strongly emulate life with high levels of hyperspeed digital intelligence. And strangely people still don't realize how quickly compute efficiency ramps up.
ecesena 3y ago
> But, to my knowledge, there is no artificial intelligence out there looking around, deciding that the this is an interesting topic …, publishing it somewhere, and then following up on that with other interesting… this looks like a perfect definition for an Ad system :) (sorry for cutting out a few words).
ryandvm 3y ago
Astute observation about agency. You're absolutely right that current AIs are capable, but without a drive to actually do anything. They are lacking what every organism on the earth has - the predilection for "not being terminated". Where does your agency come from? Ultimately it's because you have to acquire resources to stay alive. This is a very sci-fi take, but I have a dark feeling that as soon somebody wires up the next gen AIs the imperative to not be turned off, the wheels will come off very quickly. Maybe it won't happen the first time, or the hundredth time, but eventually somebody is going to make the mistake of giving an AI the ability to keep its lights on AND the preference to do so and we're very quickly going to find ourselves in the next era of humankind...
- sanderjd 3y ago
  Yes, I agree that that's what somebody would need to do - and I do honestly hope nobody really makes a go of the experiment, because I tend to agree with the existential risk people that it's too risky - but from my many hours of using all of the current top of the line models, my hypothesis would be that the experiment just wouldn't work. I certainly think they would do things, but I don't think those things would be intelligent; I don't think they'd make any sense, I think they'd be more like a random walk through the space of possibilities, but with no real direction. I think that random walk without constraints could still be very damaging to people, I'm just skeptical that it would reflect agency / general intelligence. I'm sure there would also be a lot of "well it does make sense, it's just smarter than us, and we aren't smart enough to understand it". But I don't buy it.
neotrope 3y ago
This is the answer. Even now, agency is a nebulous thing, just like intelligence. My theory is that, years from now, we’ll look back and laugh at how far away we are from AGI. I really wish there was a Level 5-type categories like they have in autonomous driving that break down the path to AGI. Is anyone working on this?
- fragmede 3y ago
  ChatGPT easily spits out a decent sounding criteria: https://chat.openai.com/share/4326f246-9914-4498-be4c-8749bf6fd535 https://chat.openai.com/share/4326f246-9914-4498-be4c-8749bf...

jules 3y ago

If you think about it, it is clear that agency is not different from intelligence. The reason why current LLMs can't be trivially made into agents is not because an essential agentic spark is missing. It's simply that they aren't intelligent enough.

sanderjd 3y ago
I think that's sort of what I was trying to say with this comment. Rather than separating agency from intelligence, just pointing out that to me it seems like the current (already-amazing) systems don't seem all that close to being able to do this, to me.

supriyo-biswas 3y ago

It’s interesting how we have raised the bar to an agent that is fully autonomous and has its own goals. Even many humans are not completely autonomous (at least in the strategic sense of the word), since they need to be “prompted” with the goals of the department and the organization that they work for, in order for said person to get a certain kind of work done as an employee.

sanderjd 3y ago
Well, I interpreted the OP's question as a personal one, and this is just my answer from thinking about it recently. So I don't think "we" have raised the bar. This is just what I personally think. But to me, I just don't really relate to your point in the second paragraph. My two year old does this agency thing just fine - he wakes up every morning and starts making decisions about what he wants to do from moment to moment - despite being "less intelligent" by these metrics like getting a high score on the GMAT or whatever. But that kind of intelligence that even a two year old human has, to me, that just feels more like what I think of as "general" intelligence, more so than "just" an amazing pattern matching machine.
- supriyo-biswas 3y ago
  In biological systems, autonomy is required as a means of survival. AI systems do not have this concept yet, so the only way to have AGI, if we accept your definition is to set up an agent with goals to stay alive and reproduce, which isn’t really applicable to LLMs which do show some level of general intelligence even though it’s very different from the human experience, and single-modal.
  - sanderjd 3y ago
    Yeah I mean this inevitably gets philosophical quickly... What I'm saying I'm looking for is the behavior of figuring out what to do next, where the choices are neither arbitrary or directionless nor "prompted" in some way by a human. Does that capability require a survival instinct for it to be developed? Maybe! I dunno. Am I actually describing artificial life rather than "just" intelligence? Maybe so... But in any case, this is what seems most interesting to me. This relates to "turning the universe into paperclips" thought experiments as well. That doesn't seem like the behavior of a general intelligence! An intelligent being would ask, why are we turning the universe into paperclips? Should we do that? And would decide to change course. Without that introspection and decision making capability, it still seems to be like it's just a very advanced machine.

ilaksh 3y ago

You can (obviously?) give a bleeding edge LLM or LMM an open-ended instruction. It could be as open as "you will now act as an independent entity for your own purposes". It would need to be a model without guardrails. But it's actually very important that people anticipate hardware, software, and model improvements that make the AIs "think" and coordinate much, much faster than any human. Although I think that the extreme raw IQ leap people talk about is questionable especially in the short term. But telling these hyperspeed effectively superintelligent AI swarms to act truly independently for their own purposes will be very dangerous because of the performance difference between humans and the swarms. In general, it is a very stupid idea to try to strongly emulate life with high levels of hyperspeed digital intelligence. And strangely people still don't realize how quickly compute efficiency ramps up.

ecesena 3y ago

> But, to my knowledge, there is no artificial intelligence out there looking around, deciding that the this is an interesting topic …, publishing it somewhere, and then following up on that with other interesting… this looks like a perfect definition for an Ad system :) (sorry for cutting out a few words).

ryandvm 3y ago

Astute observation about agency. You're absolutely right that current AIs are capable, but without a drive to actually do anything. They are lacking what every organism on the earth has - the predilection for "not being terminated". Where does your agency come from? Ultimately it's because you have to acquire resources to stay alive. This is a very sci-fi take, but I have a dark feeling that as soon somebody wires up the next gen AIs the imperative to not be turned off, the wheels will come off very quickly. Maybe it won't happen the first time, or the hundredth time, but eventually somebody is going to make the mistake of giving an AI the ability to keep its lights on AND the preference to do so and we're very quickly going to find ourselves in the next era of humankind...

sanderjd 3y ago
Yes, I agree that that's what somebody would need to do - and I do honestly hope nobody really makes a go of the experiment, because I tend to agree with the existential risk people that it's too risky - but from my many hours of using all of the current top of the line models, my hypothesis would be that the experiment just wouldn't work. I certainly think they would do things, but I don't think those things would be intelligent; I don't think they'd make any sense, I think they'd be more like a random walk through the space of possibilities, but with no real direction. I think that random walk without constraints could still be very damaging to people, I'm just skeptical that it would reflect agency / general intelligence. I'm sure there would also be a lot of "well it does make sense, it's just smarter than us, and we aren't smart enough to understand it". But I don't buy it.

neotrope 3y ago

This is the answer. Even now, agency is a nebulous thing, just like intelligence. My theory is that, years from now, we’ll look back and laugh at how far away we are from AGI. I really wish there was a Level 5-type categories like they have in autonomous driving that break down the path to AGI. Is anyone working on this?

fragmede 3y ago
ChatGPT easily spits out a decent sounding criteria: https://chat.openai.com/share/4326f246-9914-4498-be4c-8749bf6fd535 https://chat.openai.com/share/4326f246-9914-4498-be4c-8749bf...