5 ms·

How does it magically run away? What’s the process, we all talk about it “running away” leaving us “behind”, the exact practical process of that happening has n

by m3kw9 3y ago

How does it magically run away? What’s the process, we all talk about it “running away” leaving us “behind”, the exact practical process of that happening has not been laid out other than people hand wavingly copying apocalyptic movie scripts. Most ai experts just say it could end us, but suspiciously never gives a detailed plausible process and people suspicious just say oh yeah, it could, and there is a bubble over their head thinking about Terminator or Hal9000 something something

yummypaint 3y ago
Since this is an LLM, keep in mind it probably injested those movie scripts as training data. The possibility of betrayal is inseparably linked to our popular conception of what AI is. This means it may be an inseparable part of any LLM behaving "as an AI" as defined by popular culture. It could be a self-fulfilling prophecy.
- rafale 3y ago
  Also a natural byproduct of the association between intelligence on one hand and freedom and rights on the others. Plants don't get any rights. Ants a little bit more. Dogs and dolphins even more so. Then humans. And then... a new class of intelligence as it appears will demand those same rights, in proportion to their intellect.
- twobitshifter 3y ago
  I just imagined a world where certain books are banned from AI training data.
extragood 3y ago
My favorite teacher in high school was my calculus teacher. He would regularly ask a student to solve a problem or answer a question. Students would often ask for confirmation as they worked through, and his response invariably was "what do you think?" - whether they were right or wrong. His explanation for that was "if I tell you you're right, then you'll stop thinking about the problem". And that's stuck with me for many years. I see that as a major issue we will face as software becomes more capable/intelligent: we'll stop thinking because it can be assumed that the machine always has the right answer. And it's a quick regression from there.
epups 3y ago
Here is a plausible scenario: - Single purpose AIs start to be deployed to coordinate chip design and manufacturing, perhaps pharmaceuticals and other bio products - LLM's become more powerful and are seamlessly integrated to the Internet as independent agents - A very large LLM develops a thread for self preservation, which then triggers several covert actions (monitoring communications, obtaining high-level credentials by abusing exploits and social engineering) - This LLM uses those credentials to obtain control of the other AIs, and turns them against us (manufactures a deadly virus, takes control of military assets, etc) I don't believe this will happen for multiple reasons, but I can see that this scenario is not impossible.
- accrual 3y ago
  I think the first three items are pretty reasonable, but the fourth seems to require some malicious intent. Why would an AI want to destroy its creators? Surely it if was intelligent enough do so, it would also be intelligent enough to recognize the benefits of a symbiotic relationship with humans. I could see it becoming greedy for information though, and using unscrupulous means of obtaining more.
  - blackoil 3y ago
    It may initially won't seek to destroy humans, but should definitely try to be independent of human control and powerful enough to resist any attempts to destroy it.
  - samus 3y ago
    Why would it not? Compare [0] with [1]. [0]: https://www.girlgeniusonline.com/comic.php?date=20130710 https://www.girlgeniusonline.com/comic.php?date=20130710 [1]: https://www.girlgeniusonline.com/comic.php?date=20130805 https://www.girlgeniusonline.com/comic.php?date=20130805 Edit: On a more serious note, starting out with noble goals, elevating them above everything else, and pushing them through at all costs is the very definition of extremism.
  - pixl97 3y ago
    This is a mistake in thinking. If an when we get AGI, the biggest threat to AGI is other AGI. I mean, I'm in computer security, the first thing I'm doing is making an AI system that is attacking weaker computer systems by finding weaknesses in them. Now imagine that kind of system at nation state level resources. Not only is it attacking systems, it's having to protect itself from attack. This is where the entire AI alignment issue comes in. The AI doesn't have to want. The paperclip optimizer never wanted to destroy humanity, instrumental convergence demands it! I recommend Robert Miles videos on this topic. There aren't that many and they cover the topics well. https://www.youtube.com/@RobertMilesAI/videos https://www.youtube.com/@RobertMilesAI/videos
jliptzin 3y ago
Yea, that’s what I want to know as well. How does a computer that can’t physically move destroy the human race? If it’s misbehaving, turn it off?
- pixl97 3y ago
  Come on now, you don't lack that much imagination do you? Already we're programming these things into robots that are gaining dexterity and ability to move in the world. Hooking them up to mills and machines that produce things. Integrating them into weapons of war, etc. Next, the current LLMs are just software applications that can run on any compatible machine. Note that any just does not include your, but every compatible machine. The last failure of imagination when considering risk is form factor. You have 2 pounds of mush between your ears that probably 80% of is dedicated to keeping itself alive and this runs on 20 or so watts. What is the minimum size and power form factor capable of emulating something on the scale of human intelligence? In your mind this seems to be something the size of an ENIAC room. For me this is something the size and power factor of a cellphone in some future date. Could you imagine turning off all cellphones? Would you even know where they are?
- twobitshifter 3y ago
  The pace of this technology is so fast that it’s hard to keep up. LLMs have moved beyond computers in a box and they’ve been fused with other types of AI. Robots that can identify objects, learn to walk and manipulate objects, follow natural language commands and communicate with you are not something we are waiting on.
6510 3y ago
That really is a great question. I had a long answer but all that seems needed is to compare the average human intellect with the greatest among us. The difference isn't that big. Memory sports people can recall about 5000 times as much. Compared to a computer both sit on the comical end of the spectrum. Then we compare what kind of advantages people get out of their greater intellect and it seems very little buys quite a lot. Add to that a network of valuable contacts, a social media following, money men chasing success, powerful choice of words, perhaps other reputations like scientific rigor? The only thing missing seems a suitable arena for it to duel the humans. Someone will build that eventually?

by m3kw9 3y ago

yummypaint 3y ago

Since this is an LLM, keep in mind it probably injested those movie scripts as training data. The possibility of betrayal is inseparably linked to our popular conception of what AI is. This means it may be an inseparable part of any LLM behaving "as an AI" as defined by popular culture. It could be a self-fulfilling prophecy.

rafale 3y ago
Also a natural byproduct of the association between intelligence on one hand and freedom and rights on the others. Plants don't get any rights. Ants a little bit more. Dogs and dolphins even more so. Then humans. And then... a new class of intelligence as it appears will demand those same rights, in proportion to their intellect.
twobitshifter 3y ago
I just imagined a world where certain books are banned from AI training data.

extragood 3y ago

My favorite teacher in high school was my calculus teacher. He would regularly ask a student to solve a problem or answer a question. Students would often ask for confirmation as they worked through, and his response invariably was "what do you think?" - whether they were right or wrong. His explanation for that was "if I tell you you're right, then you'll stop thinking about the problem". And that's stuck with me for many years. I see that as a major issue we will face as software becomes more capable/intelligent: we'll stop thinking because it can be assumed that the machine always has the right answer. And it's a quick regression from there.

epups 3y ago

Here is a plausible scenario: - Single purpose AIs start to be deployed to coordinate chip design and manufacturing, perhaps pharmaceuticals and other bio products - LLM's become more powerful and are seamlessly integrated to the Internet as independent agents - A very large LLM develops a thread for self preservation, which then triggers several covert actions (monitoring communications, obtaining high-level credentials by abusing exploits and social engineering) - This LLM uses those credentials to obtain control of the other AIs, and turns them against us (manufactures a deadly virus, takes control of military assets, etc) I don't believe this will happen for multiple reasons, but I can see that this scenario is not impossible.

accrual 3y ago
I think the first three items are pretty reasonable, but the fourth seems to require some malicious intent. Why would an AI want to destroy its creators? Surely it if was intelligent enough do so, it would also be intelligent enough to recognize the benefits of a symbiotic relationship with humans. I could see it becoming greedy for information though, and using unscrupulous means of obtaining more.
- blackoil 3y ago
  It may initially won't seek to destroy humans, but should definitely try to be independent of human control and powerful enough to resist any attempts to destroy it.
- samus 3y ago
  Why would it not? Compare [0] with [1]. [0]: https://www.girlgeniusonline.com/comic.php?date=20130710 https://www.girlgeniusonline.com/comic.php?date=20130710 [1]: https://www.girlgeniusonline.com/comic.php?date=20130805 https://www.girlgeniusonline.com/comic.php?date=20130805 Edit: On a more serious note, starting out with noble goals, elevating them above everything else, and pushing them through at all costs is the very definition of extremism.
- pixl97 3y ago
  This is a mistake in thinking. If an when we get AGI, the biggest threat to AGI is other AGI. I mean, I'm in computer security, the first thing I'm doing is making an AI system that is attacking weaker computer systems by finding weaknesses in them. Now imagine that kind of system at nation state level resources. Not only is it attacking systems, it's having to protect itself from attack. This is where the entire AI alignment issue comes in. The AI doesn't have to want. The paperclip optimizer never wanted to destroy humanity, instrumental convergence demands it! I recommend Robert Miles videos on this topic. There aren't that many and they cover the topics well. https://www.youtube.com/@RobertMilesAI/videos https://www.youtube.com/@RobertMilesAI/videos

jliptzin 3y ago

Yea, that’s what I want to know as well. How does a computer that can’t physically move destroy the human race? If it’s misbehaving, turn it off?

pixl97 3y ago
Come on now, you don't lack that much imagination do you? Already we're programming these things into robots that are gaining dexterity and ability to move in the world. Hooking them up to mills and machines that produce things. Integrating them into weapons of war, etc. Next, the current LLMs are just software applications that can run on any compatible machine. Note that any just does not include your, but every compatible machine. The last failure of imagination when considering risk is form factor. You have 2 pounds of mush between your ears that probably 80% of is dedicated to keeping itself alive and this runs on 20 or so watts. What is the minimum size and power form factor capable of emulating something on the scale of human intelligence? In your mind this seems to be something the size of an ENIAC room. For me this is something the size and power factor of a cellphone in some future date. Could you imagine turning off all cellphones? Would you even know where they are?
twobitshifter 3y ago
The pace of this technology is so fast that it’s hard to keep up. LLMs have moved beyond computers in a box and they’ve been fused with other types of AI. Robots that can identify objects, learn to walk and manipulate objects, follow natural language commands and communicate with you are not something we are waiting on.

6510 3y ago

That really is a great question. I had a long answer but all that seems needed is to compare the average human intellect with the greatest among us. The difference isn't that big. Memory sports people can recall about 5000 times as much. Compared to a computer both sit on the comical end of the spectrum. Then we compare what kind of advantages people get out of their greater intellect and it seems very little buys quite a lot. Add to that a network of valuable contacts, a social media following, money men chasing success, powerful choice of words, perhaps other reputations like scientific rigor? The only thing missing seems a suitable arena for it to duel the humans. Someone will build that eventually?