6 ms·

Seriously, this has become religious fundamentalism based upon old bad sci-fi by hack writers. Come back to reality. It would be funny like ignorant Y2K freak o

Seriously, this has become religious fundamentalism based upon old bad sci-fi by hack writers. Come back to reality. It would be funny like ignorant Y2K freak outs if not for it being taken seriously.

NumberWangMan 4y ago
Ok, so you don't think this is a problem, but why do you think that a super-intelligent AI would be something that we could control, or would be something that would care about keeping humanity around? Or do you believe that we won't be able to build an AI with enough capability to be dangerous, keeping in mind that GPT-4 is already showing signs of being smarter than even many intelligent humans in many areas?
mitthrowaway2 4y ago
I disagree. People are arriving at this conclusion after careful consideration and logical reasoning, not picking up a '50s sci-fi book and mistaking it for reality.
- emptysongglass 4y ago
  The Less Wrong community isn't a community of careful consideration and logical reasoning. It's a cult that believes their own beliefs so fervently they would burn the earth for it. Small communities like this inevitably become so insular they recycle apocalyptic beliefs. There's no evidence that stochastic parrots are evidence of true AI. There's no evidence that humans can, by practice, subordinate emotions and become purely rational thought machines as Less Wrong people or "shape rotators" believe. Even such titles are counter-evidence of such beliefs. The human condition is so bound up in feelings, it's unclear where thinking begins and feelings end. Thoughts arise from contact, contact which produces feelings. Gut microbes are as wrapped up in cognition as the brain itself. Any out-group like this is bound to be hopelessly entrapped in its own cognitive fallacies. Yudkowsky is no exception. EDIT: Before anyone says but Yudkowsky already gets ahead of your argument with > None of this danger depends on whether or not AIs are or can be conscious; it’s intrinsic to the notion of powerful cognitive systems that optimize hard and calculate outputs that meet sufficiently complicated outcome criteria. It's one sentence that handwaves away what GPT-4 and its ilk are, which is not AI. It isn't synthetic intelligence because intelligence isn't what's happening on a fundamental level.
  - mitthrowaway2 4y ago
    There are lots of weird and crazy people reading and writing on Less Wrong, for sure. Do you think that, without gut microbes and emotions, AI will never become intelligent enough to be existentially dangerous? Are those essential to its ability to outsmart humans?
    - emptysongglass 4y ago
      I think we're asking the wrong questions. We're assuming that a. what we're developing (an associative propability machine) is in any way related to the pursuit of real artificial intelligence. We're assuming that b. motive, desire, passion, some motive force is inhered into the thing. I, personally, don't think we've seen anything that can be dangerous independent of its wielding by a badly acting human. That is to say, human agency and acting force is required on the other side of the LLM to do bad actions, like any other tool. I don't know what real AI that threatens humans with an extinction level event looks like. It's not large language models. I don't know if humans are capable of generating it. I think anthropomorphization is really this massive veil that hangs over all of us. We can't get outside of it (our essential fallacy) so we're cursed to see it everywhere.
      - mitthrowaway2 4y ago
        I'm fairly certain that an LLM alone will not independently start taking actions and pursuing goals and causing harm. That's not how LLMs work, as you and I both know. I'm not confident I can say the same about, say: an LLM calling itself from within an infinite loop in which the LLM is asked to predict actions that will [eg. add money into a bank account], and then to write code that executes those actions, then to examine the results of those actions and update its plan. (This takes no additional breakthrough in AI architecture, only an increase in the reliability of GPT for it to become practical. It is basically a summary of the AutoGPT project). I don't know if LLMs are anywhere close to any part of what goes on inside the human thought process. But I am sure that evolution was able to stumble upon the human thought process. In 2003 I was quite confident that was forever beyond the reach of humans to program into a computer, but in 2023 I am far less certain. I think human cognition is gradually becoming less of a mystery.
- darajava 4y ago
  I haven’t heard a single realistic argument on how this AI will harm us.
  - mitthrowaway2 4y ago
    I can probably offer one, if I understand what aspect you're skeptical of. Are you skeptical that an AI would ever be smart enough to come up with a plan to cause harm? That it would ever try to come up with a plan to cause harm? That it could execute such a plan, without arms and legs and a body? Or that, after coming up with a plan to cause harm, it wouldn't be trivial for humans to stop by simply unplugging the computer?
    - darajava 4y ago
      Why would it try to cause harm first of all? Second of all, it is not even nearly powerful enough to convince anyone to do anything, or anything like that. If it was 100 times more powerful, then maybe. I can see current problems being flooding spam/misinformation/propaganda on internet forums. But that has been happening for a while and this will make people more skeptical of what they read (hopefully) But finally, if it does cause harm (don’t think it will. great tool). Then who cares? Why try to stop it. You can’t stop it, nobody can. Why whinge and moan like the luddites? I don’t get it.
      - mitthrowaway2 4y ago
        OK, so to be clear, I don't think this AI will harm us, not in its current configuration. The author of the article says the same. The concern is about a future AI, one which is capable of the sort of reasoning that humans can do. The main danger with today's AI is that they will be so commercially successful that it will drive enough investment into AI research that a more powerful AI is constructed.
        darajava 4y ago
        Sure, I get that. But this has been worked on for decades in the background and only achievable now. It’s amazing and new to so many people, but this “sparks of agi” etc is all hype and honestly i think more damaging to people than ai possibly could be. You could suppose the earth would implode if we kept walking on it too, but nobody is going to stay sitting.
circuit10 4y ago
https://en.m.wikipedia.org/wiki/Appeal_to_the_stone https://en.m.wikipedia.org/wiki/Appeal_to_the_stone

NumberWangMan 4y ago

Ok, so you don't think this is a problem, but why do you think that a super-intelligent AI would be something that we could control, or would be something that would care about keeping humanity around? Or do you believe that we won't be able to build an AI with enough capability to be dangerous, keeping in mind that GPT-4 is already showing signs of being smarter than even many intelligent humans in many areas?

mitthrowaway2 4y ago

I disagree. People are arriving at this conclusion after careful consideration and logical reasoning, not picking up a '50s sci-fi book and mistaking it for reality.

emptysongglass 4y ago
The Less Wrong community isn't a community of careful consideration and logical reasoning. It's a cult that believes their own beliefs so fervently they would burn the earth for it. Small communities like this inevitably become so insular they recycle apocalyptic beliefs. There's no evidence that stochastic parrots are evidence of true AI. There's no evidence that humans can, by practice, subordinate emotions and become purely rational thought machines as Less Wrong people or "shape rotators" believe. Even such titles are counter-evidence of such beliefs. The human condition is so bound up in feelings, it's unclear where thinking begins and feelings end. Thoughts arise from contact, contact which produces feelings. Gut microbes are as wrapped up in cognition as the brain itself. Any out-group like this is bound to be hopelessly entrapped in its own cognitive fallacies. Yudkowsky is no exception. EDIT: Before anyone says but Yudkowsky already gets ahead of your argument with > None of this danger depends on whether or not AIs are or can be conscious; it’s intrinsic to the notion of powerful cognitive systems that optimize hard and calculate outputs that meet sufficiently complicated outcome criteria. It's one sentence that handwaves away what GPT-4 and its ilk are, which is not AI. It isn't synthetic intelligence because intelligence isn't what's happening on a fundamental level.
- mitthrowaway2 4y ago
  There are lots of weird and crazy people reading and writing on Less Wrong, for sure. Do you think that, without gut microbes and emotions, AI will never become intelligent enough to be existentially dangerous? Are those essential to its ability to outsmart humans?
  - emptysongglass 4y ago
    I think we're asking the wrong questions. We're assuming that a. what we're developing (an associative propability machine) is in any way related to the pursuit of real artificial intelligence. We're assuming that b. motive, desire, passion, some motive force is inhered into the thing. I, personally, don't think we've seen anything that can be dangerous independent of its wielding by a badly acting human. That is to say, human agency and acting force is required on the other side of the LLM to do bad actions, like any other tool. I don't know what real AI that threatens humans with an extinction level event looks like. It's not large language models. I don't know if humans are capable of generating it. I think anthropomorphization is really this massive veil that hangs over all of us. We can't get outside of it (our essential fallacy) so we're cursed to see it everywhere.
    - mitthrowaway2 4y ago
      I'm fairly certain that an LLM alone will not independently start taking actions and pursuing goals and causing harm. That's not how LLMs work, as you and I both know. I'm not confident I can say the same about, say: an LLM calling itself from within an infinite loop in which the LLM is asked to predict actions that will [eg. add money into a bank account], and then to write code that executes those actions, then to examine the results of those actions and update its plan. (This takes no additional breakthrough in AI architecture, only an increase in the reliability of GPT for it to become practical. It is basically a summary of the AutoGPT project). I don't know if LLMs are anywhere close to any part of what goes on inside the human thought process. But I am sure that evolution was able to stumble upon the human thought process. In 2003 I was quite confident that was forever beyond the reach of humans to program into a computer, but in 2023 I am far less certain. I think human cognition is gradually becoming less of a mystery.
darajava 4y ago
I haven’t heard a single realistic argument on how this AI will harm us.
- mitthrowaway2 4y ago
  I can probably offer one, if I understand what aspect you're skeptical of. Are you skeptical that an AI would ever be smart enough to come up with a plan to cause harm? That it would ever try to come up with a plan to cause harm? That it could execute such a plan, without arms and legs and a body? Or that, after coming up with a plan to cause harm, it wouldn't be trivial for humans to stop by simply unplugging the computer?
  - darajava 4y ago
    Why would it try to cause harm first of all? Second of all, it is not even nearly powerful enough to convince anyone to do anything, or anything like that. If it was 100 times more powerful, then maybe. I can see current problems being flooding spam/misinformation/propaganda on internet forums. But that has been happening for a while and this will make people more skeptical of what they read (hopefully) But finally, if it does cause harm (don’t think it will. great tool). Then who cares? Why try to stop it. You can’t stop it, nobody can. Why whinge and moan like the luddites? I don’t get it.
    - mitthrowaway2 4y ago
      OK, so to be clear, I don't think this AI will harm us, not in its current configuration. The author of the article says the same. The concern is about a future AI, one which is capable of the sort of reasoning that humans can do. The main danger with today's AI is that they will be so commercially successful that it will drive enough investment into AI research that a more powerful AI is constructed.
      - darajava 4y ago
        Sure, I get that. But this has been worked on for decades in the background and only achievable now. It’s amazing and new to so many people, but this “sparks of agi” etc is all hype and honestly i think more damaging to people than ai possibly could be. You could suppose the earth would implode if we kept walking on it too, but nobody is going to stay sitting.

circuit10 4y ago

https://en.m.wikipedia.org/wiki/Appeal_to_the_stone https://en.m.wikipedia.org/wiki/Appeal_to_the_stone