4 ms·

I feel/think[1] that when AGI happens, it will be so alien to us that we won't recognize it. There will be a "that's... odd." situation in science and maybe yea

I feel/think[1] that when AGI happens, it will be so alien to us that we won't recognize it. There will be a "that's... odd." situation in science and maybe years later will we realize what we had/have. As the article states, there's no understanding, "just" a regurgitation of sufficiently-trained language (which is utterly astonishing, don't get me wrong). So how does an LLM-AI turn into an AGI? Can it? Should it? How do we tell? What should be a 21st century Turing Test? Perhaps giving an AGI-candidate a novel situation and seeing how it reacts. I think that trying to apply a rigorous scientific method to such a test might end up being as "messy" as psychology is viewed as being today. It would be quite gratifying if we really did need Asimov's Susan Calvin to tell jokes to Johnny 5! I also find the prospect of an utterly incomprehensible AGI really interesting in that maybe intelligence isn't only possible via a big messy blob of neurons soaked in a chemical soup of weirdness. Maybe there's a regimented, strict path towards an intelligence that surpasses us in every way? Does anyone else think we'll be unable to recognize AGI when we see it? [1] A little column a, a little column b[2]. It's a gut feeling. Will AGIs have guts? [2] Aren't footnotes great? I still think HN needs more markdown support, but I wouldn't want to be the one to implement it!

rmilejczz 3y ago
I think the logical conclusion to all of this is that our own intelligence is not some kind of special magic. It’s easy to see it that way, and I think that will make people mostly incapable of accepting something as being AGI. I’m sure some folks will recognize it ahead of others, but most will fail to see it or it might be better to say many people will be incapable of seeing it.
LegionMammal978 3y ago
As I see it, the problem with that is that "a sufficiently unrecognizable AGI does not exist" is the unprovable null hypothesis: anything in the past or present could contain an unrecognizable AGI, since by definition we wouldn't have recognized its influence. For instance, how would one go about proving that there's no unrecognizable AGI in the otherwise-banal GPT-2, that would make itself apparent if only we found the right prompts? In the absence of any hard negative answers, we're left to try to empirically glean the answer to the not-AGI vs. AGI question for each model by poring over the text outputs as we understand them, arguing endlessly over how much is "regurgitation" or "pattern-matching" and how much is "original thought".
- bloopernova 3y ago
  good point, and pokes a rather big ol' hole in my comment :) I don't think we will recognize the AGI's existence until long after it has appeared, and I don't think it will be through the text output of chatGPT10 or whatever. I feel like we'll need to learn to infer AGI's existence because at first our own programming won't be the right way for the AGI to express itself or communicate with us. Kind of like those analogue computers or neural nets that rely on some random unforeseen external factor to work, like local magnetic field or air pressure. I do agree that there will be endless arguing over whether symptom x, y, or z definitively shows that there's a spark of understanding and self-awareness.
kenjackson 3y ago
Do you think you could convince someone, behind a terminal, that you weren’t an LLM?
- the_af 3y ago
  It has become harder, no doubt about it. But let me invert your question somewhat: do you think you'd be unable to tell over a terminal (in a long interaction) that you're chatting with a LLM? Remove the obvious constraints about no cursing, no hate speech, etc. Imagine an LLM completely uncensored. I still think I'd be able to tell with a high degree of confidence, considering multiple things: whether the "person" asks me questions, whether it refuses to answer directly (a real person will be less eager to talk about some touchy subjects, or may be shy, or upset), which topics it's eager to talk about, whether it shows lecherous behavior, whether it gets obnoxious or single minded, whether it has poor taste in things, whether it displays too much of an equanimous attitude in all topics, whether it is a bigot, or knows too much about stuff. I think I'd be able to tell.
  - famouswaffles 3y ago
    >whether the "person" asks me questions LLMs can ask questions just fine. This isn't hard to implement >whether it refuses to answer directly (a real person will be less eager to talk about some touchy subjects, or may be shy, or upset), which topics it's eager to talk about, whether it shows lecherous behavior, whether it gets obnoxious or single minded, whether it has poor taste in things, whether it displays too much of an equanimous attitude in all topics, whether it is a bigot, or knows too much about stuff. You have to understand that an LLM would have zero issues doing all these things. Open AI and other L vendors instruct tune/RLHF to speak in a certain manner but it's not a limitation of an LLM itself
  - DrWeevilJammer 3y ago
    [dead]
  - Exoristos 3y ago
    One quickly intuits the difference between an LLM and a human; unless that human were an insufferably long-winded law clerk with a sixth-grade nerd's fetish for seeming all-knowledgeable.

rmilejczz 3y ago

I think the logical conclusion to all of this is that our own intelligence is not some kind of special magic. It’s easy to see it that way, and I think that will make people mostly incapable of accepting something as being AGI. I’m sure some folks will recognize it ahead of others, but most will fail to see it or it might be better to say many people will be incapable of seeing it.

LegionMammal978 3y ago

As I see it, the problem with that is that "a sufficiently unrecognizable AGI does not exist" is the unprovable null hypothesis: anything in the past or present could contain an unrecognizable AGI, since by definition we wouldn't have recognized its influence. For instance, how would one go about proving that there's no unrecognizable AGI in the otherwise-banal GPT-2, that would make itself apparent if only we found the right prompts? In the absence of any hard negative answers, we're left to try to empirically glean the answer to the not-AGI vs. AGI question for each model by poring over the text outputs as we understand them, arguing endlessly over how much is "regurgitation" or "pattern-matching" and how much is "original thought".

bloopernova 3y ago
good point, and pokes a rather big ol' hole in my comment :) I don't think we will recognize the AGI's existence until long after it has appeared, and I don't think it will be through the text output of chatGPT10 or whatever. I feel like we'll need to learn to infer AGI's existence because at first our own programming won't be the right way for the AGI to express itself or communicate with us. Kind of like those analogue computers or neural nets that rely on some random unforeseen external factor to work, like local magnetic field or air pressure. I do agree that there will be endless arguing over whether symptom x, y, or z definitively shows that there's a spark of understanding and self-awareness.

kenjackson 3y ago

Do you think you could convince someone, behind a terminal, that you weren’t an LLM?

the_af 3y ago
It has become harder, no doubt about it. But let me invert your question somewhat: do you think you'd be unable to tell over a terminal (in a long interaction) that you're chatting with a LLM? Remove the obvious constraints about no cursing, no hate speech, etc. Imagine an LLM completely uncensored. I still think I'd be able to tell with a high degree of confidence, considering multiple things: whether the "person" asks me questions, whether it refuses to answer directly (a real person will be less eager to talk about some touchy subjects, or may be shy, or upset), which topics it's eager to talk about, whether it shows lecherous behavior, whether it gets obnoxious or single minded, whether it has poor taste in things, whether it displays too much of an equanimous attitude in all topics, whether it is a bigot, or knows too much about stuff. I think I'd be able to tell.
- famouswaffles 3y ago
  >whether the "person" asks me questions LLMs can ask questions just fine. This isn't hard to implement >whether it refuses to answer directly (a real person will be less eager to talk about some touchy subjects, or may be shy, or upset), which topics it's eager to talk about, whether it shows lecherous behavior, whether it gets obnoxious or single minded, whether it has poor taste in things, whether it displays too much of an equanimous attitude in all topics, whether it is a bigot, or knows too much about stuff. You have to understand that an LLM would have zero issues doing all these things. Open AI and other L vendors instruct tune/RLHF to speak in a certain manner but it's not a limitation of an LLM itself
- DrWeevilJammer 3y ago
  [dead]
- Exoristos 3y ago
  One quickly intuits the difference between an LLM and a human; unless that human were an insufferably long-winded law clerk with a sixth-grade nerd's fetish for seeming all-knowledgeable.