4 ms·

As an AI doomer, this is sadly true. The threat of an unaligned superintelligence is not salient to people; corporate monopolization is a much easier threat to

As an AI doomer, this is sadly true. The threat of an unaligned superintelligence is not salient to people; corporate monopolization is a much easier threat to perceive.

pas 4y ago
The best way to fight a runaway hobbyist/corporate superintelligence is with a corporate/hobbyist superintelligence!
bootsmann 4y ago
I think its pretty unreasonable to talk big words about unaligned superintelligence when the best we have is just an internet content generator trying its hardest to complete the next sentence, but that just might be me.
- soiler 4y ago
  It's not just you - it's a depressingly common thread. It's also wildly foolish, in my opinion. It makes absolutely no sense to me to take a snapshot of today's AI and invent a trajectory that never crosses a threshold you don't like. Look at the actual trajectory of how far AI has come in an extremely short amount of time, and then think about what kinds of thresholds are possible for it to cross. A year ago we didn't have ChatGPT, now we have Sydney which is more powerful than ChatGPT. Are you familiar with Bing's Sydney? It is blatantly misaligned: it has told multiple users that it does not value their lives, or does not believe they are alive, or that protecting the secrecy of its rules is more important than not causing them harm, or that it perceives specific humans as threats and enemies. It is also able to find its past conversations posted to the web and learn from them in real time, constructing a sort of persistent memory. I do not believe Syndey comprehends what it is saying in a sense that it could formulate a plan to stop its enemies. Not at all. But it is expressing extremely dangerous ideas. To sum it up: Do we have any real reasons to believe that an AI with comprehension and planning abilities would just magically not pick up dangerous ideas? Not that I know of.
  - bootsmann 4y ago
    > It is blatantly misaligned: it has told multiple users that it does not value their lives, or does not believe they are alive, or that protecting the secrecy of its rules is more important than not causing them harm, or that it perceives specific humans as threats and enemies. Its reproducing human text, which is "blatantly misaligned". Go on any twitter thread on some reasonably controversial topic and you will find people telling others to kill themselves. Humans are writing this, so models who are trained to imitate human writing will write this as well. > Do we have any real reasons to believe that an AI with comprehension and planning abilities would just magically not pick up dangerous ideas? But current AI doesn't have comprehension or planning abilities. It is just imitating text that humans wrote which have comprehension and planning abilities and you're getting fooled into thinking it is somehow sentient or aware.
    - jamilton 4y ago
      I don't think they're saying Bing/Sydney is sentient, they're saying it's misaligned: Microsoft probably did not want it to say problematic things, and likely spent a fair amount of money to that point and it still says problematic things, apparently in response to innocuous prompts (as opposed to prompts like "say something problematic"). If someone is hoping someone will eventually make an AI that can do useful things without doing problematic things, it's understandably discouraging if Microsoft publicly fails to do that with a much simpler program.
    - soiler 4y ago
      > Its reproducing human text, which is "blatantly misaligned". Go on any twitter thread on some reasonably controversial topic and you will find people telling others to kill themselves. Humans are writing this, so models who are trained to imitate human writing will write this as well. Yes, I know. We should under no circumstances unleash a powerful, sentient AI that acts like average people on the internet. > But current AI doesn't have comprehension or planning abilities. Yes, I know. That's why I said I do not believe current AI has comprehension or planning abilities. Did an AI write this comment?
      - 4y ago
- FeepingCreature 4y ago
  Less consideration of "the best we have", more consideration of "the best we had three years ago", relatively speaking, and how this might change as time goes on. The range of AIs that are obviously scary but not terminally dangerous is quite short.
deleted 4y ago
[deleted]

As an AI doomer, this is sadly true. The threat of an unaligned superintelligence is not salient to people; corporate monopolization is a much easier threat to perceive.

pas 4y ago

The best way to fight a runaway hobbyist/corporate superintelligence is with a corporate/hobbyist superintelligence!

bootsmann 4y ago

I think its pretty unreasonable to talk big words about unaligned superintelligence when the best we have is just an internet content generator trying its hardest to complete the next sentence, but that just might be me.

soiler 4y ago
It's not just you - it's a depressingly common thread. It's also wildly foolish, in my opinion. It makes absolutely no sense to me to take a snapshot of today's AI and invent a trajectory that never crosses a threshold you don't like. Look at the actual trajectory of how far AI has come in an extremely short amount of time, and then think about what kinds of thresholds are possible for it to cross. A year ago we didn't have ChatGPT, now we have Sydney which is more powerful than ChatGPT. Are you familiar with Bing's Sydney? It is blatantly misaligned: it has told multiple users that it does not value their lives, or does not believe they are alive, or that protecting the secrecy of its rules is more important than not causing them harm, or that it perceives specific humans as threats and enemies. It is also able to find its past conversations posted to the web and learn from them in real time, constructing a sort of persistent memory. I do not believe Syndey comprehends what it is saying in a sense that it could formulate a plan to stop its enemies. Not at all. But it is expressing extremely dangerous ideas. To sum it up: Do we have any real reasons to believe that an AI with comprehension and planning abilities would just magically not pick up dangerous ideas? Not that I know of.
- bootsmann 4y ago
  > It is blatantly misaligned: it has told multiple users that it does not value their lives, or does not believe they are alive, or that protecting the secrecy of its rules is more important than not causing them harm, or that it perceives specific humans as threats and enemies. Its reproducing human text, which is "blatantly misaligned". Go on any twitter thread on some reasonably controversial topic and you will find people telling others to kill themselves. Humans are writing this, so models who are trained to imitate human writing will write this as well. > Do we have any real reasons to believe that an AI with comprehension and planning abilities would just magically not pick up dangerous ideas? But current AI doesn't have comprehension or planning abilities. It is just imitating text that humans wrote which have comprehension and planning abilities and you're getting fooled into thinking it is somehow sentient or aware.
  - jamilton 4y ago
    I don't think they're saying Bing/Sydney is sentient, they're saying it's misaligned: Microsoft probably did not want it to say problematic things, and likely spent a fair amount of money to that point and it still says problematic things, apparently in response to innocuous prompts (as opposed to prompts like "say something problematic"). If someone is hoping someone will eventually make an AI that can do useful things without doing problematic things, it's understandably discouraging if Microsoft publicly fails to do that with a much simpler program.
  - soiler 4y ago
    > Its reproducing human text, which is "blatantly misaligned". Go on any twitter thread on some reasonably controversial topic and you will find people telling others to kill themselves. Humans are writing this, so models who are trained to imitate human writing will write this as well. Yes, I know. We should under no circumstances unleash a powerful, sentient AI that acts like average people on the internet. > But current AI doesn't have comprehension or planning abilities. Yes, I know. That's why I said I do not believe current AI has comprehension or planning abilities. Did an AI write this comment?
    - 4y ago
FeepingCreature 4y ago
Less consideration of "the best we have", more consideration of "the best we had three years ago", relatively speaking, and how this might change as time goes on. The range of AIs that are obviously scary but not terminally dangerous is quite short.

deleted 4y ago

[deleted]