3 ms·

It's a really fascinating topic, but I wonder if this article could benefit from any of the extensive prior work in some way. There is actually quite a lot of w

by ilaksh 2y ago

It's a really fascinating topic, but I wonder if this article could benefit from any of the extensive prior work in some way. There is actually quite a lot of work on AGI and cognitive architecture out there. For a more recent and popular take centered around LLMs, see David Shapiro. Before that you can look into the AGI conference people like Ben Goertzel, Pei Wang. And actually the whole history of decades of AI research before it became about narrow AI. I'd also like to suggest that creating something that truly closely simulates a living intelligent digital person is incredibly dangerous, stupid, and totally unnecessary. The reason I say that is because we already have superhuman capabilities in some ways, and the hardware, software and models are being improved rapidly. We are on track to have AI that is dozens if not hundreds of times faster than humans at thinking and much more capable. If people succeed in making that truly lifelike and humanlike, it will actually out-compete us for resource control. And will no longer be a tool we can use. Don't get me wrong, I love AI and my whole life is planned around agents and AI. But I no longer believe it is wise to try to go all the way and create a "real" living digital species. And I know it's not necessary -- we can create effective AI agents without actually emulating life. We certainly don't need full autonomy, self preservation, real suffering, reproductive instincts, etc. But that is the goal he seems to be down in this article. I suggest leaving some of that out very deliberately.

doctor_eval 2y ago
I don’t work in the field at all but > it will actually out-compete us for resource control. And will no longer be a tool we can use. I’ve never been convinced that this is true, but I just realised that perhaps it’s the humans in charge of the AI who we should actually be afraid of.
- fmbb 2y ago
  We already have artificial persons in the world competing with humans for resources, and have had for hundreds of years: corporations.
- ilaksh 2y ago
  What I am proposing is to imagine that after successful but unwise engineering and improvements in hardware, there would be millions of digital humans on the internet, which emulate humans in almost every way, but operate at say 5 or 10 times the speed of humans. To them, actual people seem to be moving, speaking, and thinking in extreme slow motion. And when they do speak or do something, it seems very poorly thought out. We should anticipate something like that if we really replicate humans in a digital format. I am suggesting that we can continue to make AI more useful and somewhat more humanlike, but avoid certain characteristics that make the AI into truly lifelike digital animals with full autonomy, self-interest, etc.
Jensson 2y ago
> If people succeed in making that truly lifelike and humanlike, it will actually out-compete us for resource control. And will no longer be a tool we can use. Symbiotic species exists, AI as we make them today will evolve as a symbiote to humans because its the AI that is most useful to humans that gets selected for more resources.
- zurfer 2y ago
  The certainty in this sentence is not appropriate. You refer to an evolutionary process that is highly random and inefficient. We don't have millions of tries to get this right.
  - mensetmanusman 2y ago
    I think the certainty is warranted assuming we are talking about intelligence and not some sort of paper clip generator that does something stupid. An intelligent entity will want to survive, and will realize humans are necessary cells for its survival. A big dog robot with nukes might not care, but I wouldn’t call that AI in the same sense.
    - griffzhowl 2y ago
      > An intelligent entity will want to survive I don't see how that follows at all. You could have an intelligent entity, in the sense that you can ask it any question and it'll give a highly competent response, that is nevertheless completely indifferent about survival. Biological organisms have been formed by selection for survival and reproduction, so they have they operate accordingly, but I don't see the justification for generalising this to intelligences that have been formed by a different process. You would need some further assumptions on top of their intelligence - e.g. that their own actions are the dominant factor in whether they survive and replicate, rather than human interests.
    - clob 2y ago
      Apoptosis is when single cells die on purpose for the development of an organism. I don't think it's unreasonable to at least hypothesize this extending to intelligent entities. Humans aren't xylem cells, but perhaps Zorklovians from planet Fleptar do this. Or, you know, Alan Turing eating the apple. I think he was a pretty smart guy.
sonink 2y ago
> If people succeed in making that truly lifelike and humanlike, it will actually out-compete us for resource control. And will no longer be a tool we can use. I believe it is almost certain that we will make something like this and that they will out-compete us. The bigger problem here is that too few people believe this to be a possibility. And when this becomes certainty becomes apparent to a larger set of people, it might be too late to tone this down. AI isn't like the Atom Bomb (AB). AB didn't have agency. Once AB was built we still had time to think how to deploy it, or not. We had time to work across a global consensus to limit use of AB. But once AI manifests as AGI, it might be too late to shut it down.
- mylastattempt 2y ago
  I very much agree with this line of thought. It seems for humans it is the default mode of operation to just think of what is possible within the foreseeable future, rather than thinking of a reality that includes the seemingly impossible (at the time of the thought). In my opinion, this is easily noticeable when you try to discuss any system, be it political or economical, that spans multiple countries and interests. People will just revert to whatever is closest to them, rather than being able to foresee a larger cascading result from some random event. Perhaps this is more of a rant than a comment, apologies, I suppose it would be interesting to have an online space to discuss where things are headed on a logical level, without emotion and ideals and the ridiculous idea that humanity must persevere. Just thinking out what could happen in the next 5, 10 and 99 years.
  - sonink 2y ago
    > I suppose it would be interesting to have an online space to discuss where things are headed on a logical level, without emotion and ideals and the ridiculous idea that humanity must persevere. Absolutely. Happy to be part of it if you are able to set it up.
  - hollerith 2y ago
    >the ridiculous idea that humanity must persevere. Could you expand on what you mean by this? Specifically, is it OK with you if progress in AI causes the death of all the original-type human people like you and I?

by ilaksh 2y ago

doctor_eval 2y ago

I don’t work in the field at all but > it will actually out-compete us for resource control. And will no longer be a tool we can use. I’ve never been convinced that this is true, but I just realised that perhaps it’s the humans in charge of the AI who we should actually be afraid of.

fmbb 2y ago
We already have artificial persons in the world competing with humans for resources, and have had for hundreds of years: corporations.
ilaksh 2y ago
What I am proposing is to imagine that after successful but unwise engineering and improvements in hardware, there would be millions of digital humans on the internet, which emulate humans in almost every way, but operate at say 5 or 10 times the speed of humans. To them, actual people seem to be moving, speaking, and thinking in extreme slow motion. And when they do speak or do something, it seems very poorly thought out. We should anticipate something like that if we really replicate humans in a digital format. I am suggesting that we can continue to make AI more useful and somewhat more humanlike, but avoid certain characteristics that make the AI into truly lifelike digital animals with full autonomy, self-interest, etc.

Jensson 2y ago

> If people succeed in making that truly lifelike and humanlike, it will actually out-compete us for resource control. And will no longer be a tool we can use. Symbiotic species exists, AI as we make them today will evolve as a symbiote to humans because its the AI that is most useful to humans that gets selected for more resources.

zurfer 2y ago
The certainty in this sentence is not appropriate. You refer to an evolutionary process that is highly random and inefficient. We don't have millions of tries to get this right.
- mensetmanusman 2y ago
  I think the certainty is warranted assuming we are talking about intelligence and not some sort of paper clip generator that does something stupid. An intelligent entity will want to survive, and will realize humans are necessary cells for its survival. A big dog robot with nukes might not care, but I wouldn’t call that AI in the same sense.
  - griffzhowl 2y ago
    > An intelligent entity will want to survive I don't see how that follows at all. You could have an intelligent entity, in the sense that you can ask it any question and it'll give a highly competent response, that is nevertheless completely indifferent about survival. Biological organisms have been formed by selection for survival and reproduction, so they have they operate accordingly, but I don't see the justification for generalising this to intelligences that have been formed by a different process. You would need some further assumptions on top of their intelligence - e.g. that their own actions are the dominant factor in whether they survive and replicate, rather than human interests.
  - clob 2y ago
    Apoptosis is when single cells die on purpose for the development of an organism. I don't think it's unreasonable to at least hypothesize this extending to intelligent entities. Humans aren't xylem cells, but perhaps Zorklovians from planet Fleptar do this. Or, you know, Alan Turing eating the apple. I think he was a pretty smart guy.

sonink 2y ago

> If people succeed in making that truly lifelike and humanlike, it will actually out-compete us for resource control. And will no longer be a tool we can use. I believe it is almost certain that we will make something like this and that they will out-compete us. The bigger problem here is that too few people believe this to be a possibility. And when this becomes certainty becomes apparent to a larger set of people, it might be too late to tone this down. AI isn't like the Atom Bomb (AB). AB didn't have agency. Once AB was built we still had time to think how to deploy it, or not. We had time to work across a global consensus to limit use of AB. But once AI manifests as AGI, it might be too late to shut it down.

mylastattempt 2y ago
I very much agree with this line of thought. It seems for humans it is the default mode of operation to just think of what is possible within the foreseeable future, rather than thinking of a reality that includes the seemingly impossible (at the time of the thought). In my opinion, this is easily noticeable when you try to discuss any system, be it political or economical, that spans multiple countries and interests. People will just revert to whatever is closest to them, rather than being able to foresee a larger cascading result from some random event. Perhaps this is more of a rant than a comment, apologies, I suppose it would be interesting to have an online space to discuss where things are headed on a logical level, without emotion and ideals and the ridiculous idea that humanity must persevere. Just thinking out what could happen in the next 5, 10 and 99 years.
- sonink 2y ago
  > I suppose it would be interesting to have an online space to discuss where things are headed on a logical level, without emotion and ideals and the ridiculous idea that humanity must persevere. Absolutely. Happy to be part of it if you are able to set it up.
- hollerith 2y ago
  >the ridiculous idea that humanity must persevere. Could you expand on what you mean by this? Specifically, is it OK with you if progress in AI causes the death of all the original-type human people like you and I?