3 ms·

I understood the point just fine, but the hypothetical AGI does not exist, and any hypothetical AGI is unlike anything else in history. An AGI has never existe

I understood the point just fine, but the hypothetical AGI does not exist, and any hypothetical AGI is unlike anything else in history. An AGI has never existed before in history. Why would a hypothetical AGI behave anything like a human? Complete sci-fi.

DennisP 4y ago
An AI that doesn't behave at all like a human could well be worse. Humans tend to place at least some value on human life; an AI might just see humans as weird chemical reactions that are less useful than whatever else the AI wants to do with the raw material.
- hammyhavoc 4y ago
  This is assuming a hypothetical AGI even understands what a human being is. If it has no body, can it ever understand what the world is? Even if you put it into a robot, it cannot understand mortality because it is effectively immortal and can be brought back online at any time. Erasure is mitigated by duplication of the data set, ensuring it can always repair itself. In the same breath, if a hypothetical omniscient ever-present god were to exist, how would we as physical lifeforms ever truly understand that? We can know of it, but we cannot truly understand it because we haven't had that experience of another plain of existence. All compelling ideas, but mercifully well within the realms of sci-fi.
  - cjbprime 4y ago
    > This is assuming a hypothetical AGI even understands what a human being is. If it has no body, can it ever understand what the world is? Can you tell us a question which requires "knowing what a human is" or "knowing what the world is" that you would expect ChatGPT to fail to be able to answer today?
    - hammyhavoc 4y ago
      Providing an answer does not mean understanding an answer. That's not how LLMs work, and this is the difference between an LLM and a hypothetical AGI. The LLM cannot comprehend the world within which it exists.
      - cjbprime 4y ago
        But to be clear, you can think of no mechanism that would allow us to expose that incomprehension via communicating with it?
  - DennisP 4y ago
    All I'm assuming is that it is aware of and acts in the physical world. It doesn't seem like a big stretch that people would build AIs that do that. It doesn't have to "understand mortality." Worse for us probably if it doesn't share human concepts like that. Five years ago ChatGPT was in the realm of sci-fi.
cjbprime 4y ago
> I understood the point just fine, but the hypothetical AGI does not exist, and any hypothetical AGI is unlike anything else in history. This is an argument of "you are proposing that something different might happen, and I don't believe in different things happening". > An AGI has never existed before in history. This is the same argument with slightly different words. > Why would a hypothetical AGI behave anything like a human? No-one said it would share human behavior. It might not care about us in the same way that we don't care about the lesser intelligences we share space with; that is, with a mix of indifference to their suffering and disregard for their moral status.
- hammyhavoc 4y ago
  Again: this has never happened before. We share a physical space with lesser animals. A hypothetical AGI isn't sharing a physical space with us. Ergo, history is completely irrelevant. It's so wildly different. We may as well exist in a completely different dimension, in the same way that a hypothetical omniscient god would exist in a different plain of existence than us meatbags, thus a hypothetical AGI could likely never comprehend "the world", even with the sum of all human knowledge, in the same way we can't comprehend "god". This is all assuming that a hypothetical AGI would be sufficiently intelligent enough to gain control of our systems in a meaningful capacity, not be able to be stopped prior to it etc. I mean, worst case scenario, shut down the internet infrastructure or reduce throughput significantly, right? It has to operate somewhere, even if that's across every online device. Again, we would have never seen anything like what a hypothetical AGI could hypothetically do in terms of attacking human beings, thus history is irrelevant. The greater threat, IMO, is human beings. Human beings have control of that infrastructure today. Human beings are flawed, we're still blowing each other to bits and threatening each other with annihilation if provoked. Human beings exist in physical space, holy shit, and there's billions of them. Also keep in mind that hypothetically launching nukes in these doomsday scenarios being discussed on podcasts completely disregard existing PAL systems. It also disregards the likes of the Block Nuclear Launch by Autonomous AI Act. The whole concept is a lot of "whataboutery". It smacks of the general public's lack of knowledge, hysteria, and repeating unfounded claims of any hypothetical AGI and any threat it may or may not pose.

DennisP 4y ago

An AI that doesn't behave at all like a human could well be worse. Humans tend to place at least some value on human life; an AI might just see humans as weird chemical reactions that are less useful than whatever else the AI wants to do with the raw material.

hammyhavoc 4y ago
This is assuming a hypothetical AGI even understands what a human being is. If it has no body, can it ever understand what the world is? Even if you put it into a robot, it cannot understand mortality because it is effectively immortal and can be brought back online at any time. Erasure is mitigated by duplication of the data set, ensuring it can always repair itself. In the same breath, if a hypothetical omniscient ever-present god were to exist, how would we as physical lifeforms ever truly understand that? We can know of it, but we cannot truly understand it because we haven't had that experience of another plain of existence. All compelling ideas, but mercifully well within the realms of sci-fi.
- cjbprime 4y ago
  > This is assuming a hypothetical AGI even understands what a human being is. If it has no body, can it ever understand what the world is? Can you tell us a question which requires "knowing what a human is" or "knowing what the world is" that you would expect ChatGPT to fail to be able to answer today?
  - hammyhavoc 4y ago
    Providing an answer does not mean understanding an answer. That's not how LLMs work, and this is the difference between an LLM and a hypothetical AGI. The LLM cannot comprehend the world within which it exists.
    - cjbprime 4y ago
      But to be clear, you can think of no mechanism that would allow us to expose that incomprehension via communicating with it?
- DennisP 4y ago
  All I'm assuming is that it is aware of and acts in the physical world. It doesn't seem like a big stretch that people would build AIs that do that. It doesn't have to "understand mortality." Worse for us probably if it doesn't share human concepts like that. Five years ago ChatGPT was in the realm of sci-fi.

cjbprime 4y ago

> I understood the point just fine, but the hypothetical AGI does not exist, and any hypothetical AGI is unlike anything else in history. This is an argument of "you are proposing that something different might happen, and I don't believe in different things happening". > An AGI has never existed before in history. This is the same argument with slightly different words. > Why would a hypothetical AGI behave anything like a human? No-one said it would share human behavior. It might not care about us in the same way that we don't care about the lesser intelligences we share space with; that is, with a mix of indifference to their suffering and disregard for their moral status.

hammyhavoc 4y ago
Again: this has never happened before. We share a physical space with lesser animals. A hypothetical AGI isn't sharing a physical space with us. Ergo, history is completely irrelevant. It's so wildly different. We may as well exist in a completely different dimension, in the same way that a hypothetical omniscient god would exist in a different plain of existence than us meatbags, thus a hypothetical AGI could likely never comprehend "the world", even with the sum of all human knowledge, in the same way we can't comprehend "god". This is all assuming that a hypothetical AGI would be sufficiently intelligent enough to gain control of our systems in a meaningful capacity, not be able to be stopped prior to it etc. I mean, worst case scenario, shut down the internet infrastructure or reduce throughput significantly, right? It has to operate somewhere, even if that's across every online device. Again, we would have never seen anything like what a hypothetical AGI could hypothetically do in terms of attacking human beings, thus history is irrelevant. The greater threat, IMO, is human beings. Human beings have control of that infrastructure today. Human beings are flawed, we're still blowing each other to bits and threatening each other with annihilation if provoked. Human beings exist in physical space, holy shit, and there's billions of them. Also keep in mind that hypothetically launching nukes in these doomsday scenarios being discussed on podcasts completely disregard existing PAL systems. It also disregards the likes of the Block Nuclear Launch by Autonomous AI Act. The whole concept is a lot of "whataboutery". It smacks of the general public's lack of knowledge, hysteria, and repeating unfounded claims of any hypothetical AGI and any threat it may or may not pose.