5 ms·

Do any of these LLMs have their own agenda where they operate under their own agency and could plausibly take over all power generation on earth? Or are we tal

Do any of these LLMs have their own agenda where they operate under their own agency and could plausibly take over all power generation on earth? Or are we talking about a variation of the system we have right now, where someone could use the AI as a part of a control system and then the "algorithm" doesn't operate the way we want and causes an outage? Because that happens all the time without LLMs. I am struggling to understand you people who jump from "LLMs are an amazing technology" to "A new lifeform is here making moves to seize control!"

digging 3y ago
> Do any of these LLMs have their own agenda where they operate under their own agency and could plausibly take over all power generation on earth? No, but that's actually irrelevant. The reason it's irrelevant is because of the answer to: > I am struggling to understand you people who jump from "LLMs are an amazing technology" to "A new lifeform is here making moves to seize control!" Nobody is saying the second thing (that I'm aware of). This is the chain of reasoning: 1. "LLMs are an amazing technology [which advanced at an unexpected rate]" 2. Because LLMs advanced at an unexpected rate, this indicates an acceleration of research 3. Such acceleration could conceivably happen or be happening now in other forms of AI 4. The path to AGI is COMPLETELY unknown. 5. The origins and structures of consciousness and agency in intelligent systems are completely unknown 6. It is impossible for humans today to know if they've crossed a threshold into creating AGI, or a new form of intelligent alien life. 7. AGI is inherently alien and there's no reason to expect it to think the way we do. 8. Even human goals and systems are often (or even mostly) misaligned - look at any for-profit corporation or totalitarian state for an extreme example of how their existence harms people in many cases (pollution, murder, exploitation, discrimination, etc) So, I think I've covered the bases. I'll sum it up like this: We don't know what AGI looks like, we can't expect it to have our best interests in mind because it's an alien that's smarter than us, and we don't know when or how we'll achieve AGI. So unless you don't believe superhuman AGI is possible at all, we're in a very scary time. We have know way of knowing if it will take 5 years or 100, but we also know that people all over the world are racing to make it happen as soon as possible.
- droopyEyelids 3y ago
  The path to many things are completely unknown but that doesn't make them likely to happen or worth worrying about. I still don't get it. We can't know if we've started the process to summon alien entities from another dimension through a rift in spacetime, either.
  - digging 3y ago
    Yes, but we also don't have reason to believe summoning alien entities through any kind of rift is at all possible. According to known physics, it can't happen. On the other hand, we don't have any scientific reason to believe that superhuman AGI isn't possible. We know AI is possible, because we have it right now. What's preventing it from getting smarter than us? The human soul? To clarify, most believe we likely have started on the path to superhuman AGI. We just don't know how or when we'll reach an arbitrary threshold where it becomes dangerous. There are many finish lines and we can't see any of them, but we're running as fast as we can down the path of least resistance and hoping we don't cross a finish line we don't like.
    - tome 3y ago
      > we don't have any scientific reason to believe that superhuman AGI isn't possible Sorry, what? According to which scientific theories is AGI possible, and what principles are those theories based on? What testable hypotheses say that it is, and which experimental evidence has been produced in favour of AGI being possible?
      - digging 3y ago
        We have no evidence one way or the other, but it's kind of insane to me to think that superhuman AGI is not possible. Human-level intelligence is possible. See: humans Do you think it's physically impossible to recreate human-level general intelligence artificially? Do you think it's physically impossible for any intelligence to be superhuman, even a natural intelligence? Unless you answered "yes" to either of the above, why shouldn't it be physically possible for humans to create superhuman AGI? Is there any physical law whatsoever that even hints that intelligence should be able to create AGI at its own level, but in no way more intelligent? I never said there was a physical theory that predicts superhuman AGI. But in contrast, there are other concepts such as time travel which are prohibited by certain theories we have. It seems very unlikely for time travel to be possible. Superhuman AGI is not prohibited from existence.

digging 3y ago

> Do any of these LLMs have their own agenda where they operate under their own agency and could plausibly take over all power generation on earth? No, but that's actually irrelevant. The reason it's irrelevant is because of the answer to: > I am struggling to understand you people who jump from "LLMs are an amazing technology" to "A new lifeform is here making moves to seize control!" Nobody is saying the second thing (that I'm aware of). This is the chain of reasoning: 1. "LLMs are an amazing technology [which advanced at an unexpected rate]" 2. Because LLMs advanced at an unexpected rate, this indicates an acceleration of research 3. Such acceleration could conceivably happen or be happening now in other forms of AI 4. The path to AGI is COMPLETELY unknown. 5. The origins and structures of consciousness and agency in intelligent systems are completely unknown 6. It is impossible for humans today to know if they've crossed a threshold into creating AGI, or a new form of intelligent alien life. 7. AGI is inherently alien and there's no reason to expect it to think the way we do. 8. Even human goals and systems are often (or even mostly) misaligned - look at any for-profit corporation or totalitarian state for an extreme example of how their existence harms people in many cases (pollution, murder, exploitation, discrimination, etc) So, I think I've covered the bases. I'll sum it up like this: We don't know what AGI looks like, we can't expect it to have our best interests in mind because it's an alien that's smarter than us, and we don't know when or how we'll achieve AGI. So unless you don't believe superhuman AGI is possible at all, we're in a very scary time. We have know way of knowing if it will take 5 years or 100, but we also know that people all over the world are racing to make it happen as soon as possible.

droopyEyelids 3y ago
The path to many things are completely unknown but that doesn't make them likely to happen or worth worrying about. I still don't get it. We can't know if we've started the process to summon alien entities from another dimension through a rift in spacetime, either.
- digging 3y ago
  Yes, but we also don't have reason to believe summoning alien entities through any kind of rift is at all possible. According to known physics, it can't happen. On the other hand, we don't have any scientific reason to believe that superhuman AGI isn't possible. We know AI is possible, because we have it right now. What's preventing it from getting smarter than us? The human soul? To clarify, most believe we likely have started on the path to superhuman AGI. We just don't know how or when we'll reach an arbitrary threshold where it becomes dangerous. There are many finish lines and we can't see any of them, but we're running as fast as we can down the path of least resistance and hoping we don't cross a finish line we don't like.
  - tome 3y ago
    > we don't have any scientific reason to believe that superhuman AGI isn't possible Sorry, what? According to which scientific theories is AGI possible, and what principles are those theories based on? What testable hypotheses say that it is, and which experimental evidence has been produced in favour of AGI being possible?
    - digging 3y ago
      We have no evidence one way or the other, but it's kind of insane to me to think that superhuman AGI is not possible. Human-level intelligence is possible. See: humans Do you think it's physically impossible to recreate human-level general intelligence artificially? Do you think it's physically impossible for any intelligence to be superhuman, even a natural intelligence? Unless you answered "yes" to either of the above, why shouldn't it be physically possible for humans to create superhuman AGI? Is there any physical law whatsoever that even hints that intelligence should be able to create AGI at its own level, but in no way more intelligent? I never said there was a physical theory that predicts superhuman AGI. But in contrast, there are other concepts such as time travel which are prohibited by certain theories we have. It seems very unlikely for time travel to be possible. Superhuman AGI is not prohibited from existence.