5 ms·

> I understood the point just fine, but the hypothetical AGI does not exist, and any hypothetical AGI is unlike anything else in history. This is an argument o

> I understood the point just fine, but the hypothetical AGI does not exist, and any hypothetical AGI is unlike anything else in history. This is an argument of "you are proposing that something different might happen, and I don't believe in different things happening". > An AGI has never existed before in history. This is the same argument with slightly different words. > Why would a hypothetical AGI behave anything like a human? No-one said it would share human behavior. It might not care about us in the same way that we don't care about the lesser intelligences we share space with; that is, with a mix of indifference to their suffering and disregard for their moral status.

hammyhavoc 3y ago
Again: this has never happened before. We share a physical space with lesser animals. A hypothetical AGI isn't sharing a physical space with us. Ergo, history is completely irrelevant. It's so wildly different. We may as well exist in a completely different dimension, in the same way that a hypothetical omniscient god would exist in a different plain of existence than us meatbags, thus a hypothetical AGI could likely never comprehend "the world", even with the sum of all human knowledge, in the same way we can't comprehend "god". This is all assuming that a hypothetical AGI would be sufficiently intelligent enough to gain control of our systems in a meaningful capacity, not be able to be stopped prior to it etc. I mean, worst case scenario, shut down the internet infrastructure or reduce throughput significantly, right? It has to operate somewhere, even if that's across every online device. Again, we would have never seen anything like what a hypothetical AGI could hypothetically do in terms of attacking human beings, thus history is irrelevant. The greater threat, IMO, is human beings. Human beings have control of that infrastructure today. Human beings are flawed, we're still blowing each other to bits and threatening each other with annihilation if provoked. Human beings exist in physical space, holy shit, and there's billions of them. Also keep in mind that hypothetically launching nukes in these doomsday scenarios being discussed on podcasts completely disregard existing PAL systems. It also disregards the likes of the Block Nuclear Launch by Autonomous AI Act. The whole concept is a lot of "whataboutery". It smacks of the general public's lack of knowledge, hysteria, and repeating unfounded claims of any hypothetical AGI and any threat it may or may not pose.
- cjbprime 3y ago
  > thus history is irrelevant Then stop appealing to it as a reason that everything will be fine! We don't know what's going to happen and have no precedent, and that is why this is a scary situation that requires thought. > A hypothetical AGI isn't sharing a physical space with us. That's not true, right? We would be competing for electricity and resources with it, at the least. Also, it is able to affect our physical space by persuading humans to do things it wants to do, which will have unpredictable and hard to defend against effects because of the whole superintelligence thing. (Imagine, say, a version of the attack on the US Capitol, but on behalf of a leader that is orders of magnitude more charismatic and persuasive.) > I mean, worst case scenario, shut down the internet infrastructure or reduce throughput significantly, right? It has to operate somewhere, even if that's across every online device. Sure, persuading everyone in the world to all agree to turn off all communications devices -- when they've already presumably been convinced that they don't want to do that by a superintelligence -- sounds like an awesome plan that will totally work.

hammyhavoc 3y ago

Again: this has never happened before. We share a physical space with lesser animals. A hypothetical AGI isn't sharing a physical space with us. Ergo, history is completely irrelevant. It's so wildly different. We may as well exist in a completely different dimension, in the same way that a hypothetical omniscient god would exist in a different plain of existence than us meatbags, thus a hypothetical AGI could likely never comprehend "the world", even with the sum of all human knowledge, in the same way we can't comprehend "god". This is all assuming that a hypothetical AGI would be sufficiently intelligent enough to gain control of our systems in a meaningful capacity, not be able to be stopped prior to it etc. I mean, worst case scenario, shut down the internet infrastructure or reduce throughput significantly, right? It has to operate somewhere, even if that's across every online device. Again, we would have never seen anything like what a hypothetical AGI could hypothetically do in terms of attacking human beings, thus history is irrelevant. The greater threat, IMO, is human beings. Human beings have control of that infrastructure today. Human beings are flawed, we're still blowing each other to bits and threatening each other with annihilation if provoked. Human beings exist in physical space, holy shit, and there's billions of them. Also keep in mind that hypothetically launching nukes in these doomsday scenarios being discussed on podcasts completely disregard existing PAL systems. It also disregards the likes of the Block Nuclear Launch by Autonomous AI Act. The whole concept is a lot of "whataboutery". It smacks of the general public's lack of knowledge, hysteria, and repeating unfounded claims of any hypothetical AGI and any threat it may or may not pose.

cjbprime 3y ago
> thus history is irrelevant Then stop appealing to it as a reason that everything will be fine! We don't know what's going to happen and have no precedent, and that is why this is a scary situation that requires thought. > A hypothetical AGI isn't sharing a physical space with us. That's not true, right? We would be competing for electricity and resources with it, at the least. Also, it is able to affect our physical space by persuading humans to do things it wants to do, which will have unpredictable and hard to defend against effects because of the whole superintelligence thing. (Imagine, say, a version of the attack on the US Capitol, but on behalf of a leader that is orders of magnitude more charismatic and persuasive.) > I mean, worst case scenario, shut down the internet infrastructure or reduce throughput significantly, right? It has to operate somewhere, even if that's across every online device. Sure, persuading everyone in the world to all agree to turn off all communications devices -- when they've already presumably been convinced that they don't want to do that by a superintelligence -- sounds like an awesome plan that will totally work.