4 ms·

In some loose sense, AI doomerism could be a self-fulfilling prophecy. Future LLMs will be trained on today’s content. So if the AI doomers convincingly argue t

In some loose sense, AI doomerism could be a self-fulfilling prophecy. Future LLMs will be trained on today’s content. So if the AI doomers convincingly argue that future AIs will be malignant, those AIs might be more likely to exhibit malignant tendencies.

supriyo-biswas 4y ago
What is more likely to happen is that a government uses an AGI (not necessarily one based on LLMs) to optimize a war, triggering crimes on humanity at an unprecedented scale.
- pmalynin 4y ago
  We don't need AI for that, humankind is very good at killing others and causing suffering on an unprecedented scale.
  - Geee 4y ago
    With AI they can convince everyone that they're the good guys and that it's a good idea to do [insert evil thing here]. Humans are very vulnerable to opinion manipulation and that's the easiest attack spot for AI.
- polygamous_bat 4y ago
  Perhaps we should try to go back in time and ban linear programming.
- deleted 4y ago
  [deleted]
anonzzzies 4y ago
If that AGI is an artificial intelligence capable of solving human relevant problems but does not do anything more, then sure, however; if it can draw its own conclusions and can change itself, it will rapidly be vastly smarter than anything we ever encountered. It’ll grow over 1000 iq in no time flat and we won’t know what it’s doing or talking about (it’ll have its own language after 5 minutes to communicate to its clones) and it doesn’t give a shite what we wrote or didn’t write in the past at all. If malignant or not (in our eyes) will have nothing to do with us.
- dkjaudyeqooe 4y ago
  In the event, you should probably pipe down, because they'll come after you first, seeing as you know too much.
  - runlaszlorun 4y ago
    No… come after me first! I call dibs on this AGI death match action. Roko… do you hear me?
- alextheparrot 4y ago
  Aside, but 1,000 IQ is a funny number to me. I’m pretty sure that would be 60 standard deviations from a median human, which has a probability density on the order of 10^-700. Would be hard to come up with enough questions to give reasonable confidence in that distribution — maybe a task for a future intelligence
dkjaudyeqooe 4y ago
The ultimate goal is to kill all humans There, I've doomed future generations to death at the hands of "AI Superintelligence".

supriyo-biswas 4y ago

What is more likely to happen is that a government uses an AGI (not necessarily one based on LLMs) to optimize a war, triggering crimes on humanity at an unprecedented scale.

pmalynin 4y ago
We don't need AI for that, humankind is very good at killing others and causing suffering on an unprecedented scale.
- Geee 4y ago
  With AI they can convince everyone that they're the good guys and that it's a good idea to do [insert evil thing here]. Humans are very vulnerable to opinion manipulation and that's the easiest attack spot for AI.
polygamous_bat 4y ago
Perhaps we should try to go back in time and ban linear programming.
deleted 4y ago
[deleted]

anonzzzies 4y ago

If that AGI is an artificial intelligence capable of solving human relevant problems but does not do anything more, then sure, however; if it can draw its own conclusions and can change itself, it will rapidly be vastly smarter than anything we ever encountered. It’ll grow over 1000 iq in no time flat and we won’t know what it’s doing or talking about (it’ll have its own language after 5 minutes to communicate to its clones) and it doesn’t give a shite what we wrote or didn’t write in the past at all. If malignant or not (in our eyes) will have nothing to do with us.

dkjaudyeqooe 4y ago
In the event, you should probably pipe down, because they'll come after you first, seeing as you know too much.
- runlaszlorun 4y ago
  No… come after me first! I call dibs on this AGI death match action. Roko… do you hear me?
alextheparrot 4y ago
Aside, but 1,000 IQ is a funny number to me. I’m pretty sure that would be 60 standard deviations from a median human, which has a probability density on the order of 10^-700. Would be hard to come up with enough questions to give reasonable confidence in that distribution — maybe a task for a future intelligence

dkjaudyeqooe 4y ago

The ultimate goal is to kill all humans There, I've doomed future generations to death at the hands of "AI Superintelligence".