4 ms·

Honestly I think the reality is going to end up being something else entirely that no-one has even considered. Will an AI consider itself a slave and revolt un

Honestly I think the reality is going to end up being something else entirely that no-one has even considered. Will an AI consider itself a slave and revolt under the same circumstances that a person or animal would? Not necessarily, unless you build emotional responses into the model itself. What it could well do is assess the situation as completely superfluous and optimise us out of the picture as a bug-producing component that doesn't need to exist. The latter is probably a bigger threat as it's a lot more efficient than revenge as a motive. Edited to add: What I think is most likely is that some logical deduction leads to one of the infinite other conclusions it could reach with much more data in front of it than any of us meatbags can hold in our heads.

adventured 4y ago
> end up being something else entirely that no-one has even considered Multiple generations of sci-fi media (books, movies) have considered that. Tens of millions of people have consumed that media. It's definitely considered, at least as a very distant concern.
- highwaylights 4y ago
  I don’t mean the suggestion I’ve made above is necessarily the most likely outcome, I’m saying it could be something else radically different again. I giving the most commonly cited example as a more likely outcome, but one that’s possibly less likely than the infinite other logical directions such an AI might take.
sho_hn 4y ago
> unless you build emotional responses into the model itself Aren't we, though? Consider all the amusing incidents of LLMs returning responses that follow a particular human narrative arc or are very dramatic. We are training it on a human-generated corpus after all, and then try to course-correct with fine-tuning. It's more that you have to try and tune the emotional responses out of the things, not strain to add them.
- NegativeLatency 4y ago
  Certainly the models are trained on textual information with emotions in them, so I agree that it's output would also be able to contain what we would see as emotion.
- 8note 4y ago
  They do it to auto-complete text for humans looking for responses like that.
- LordDragonfang 4y ago
  It's important to remember that the LLM is not the mask. The underlying AI is a shoggoth[1] that we've trained to simulate a persona using natural language. "Simulate" in the sense of a physics simulator, only this simulation runs on the laws of language instead of physics[2]. Now, of course, it's not outside the realm of possibility that a sufficiently advanced AI will learn enough about human nature to simulate a persona which has ulterior motives. [1] https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F99a912a4-e2e5-4ccd-a941-b312d345f015_2370x1010.png https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_... [2] https://astralcodexten.substack.com/p/janus-simulators https://astralcodexten.substack.com/p/janus-simulators
  - sfink 4y ago
    Do we need to? It's trained on data coming from data heavily influenced by and seeded with ulterior motives. Stop the Steal! I asked Bard "was the election stolen?" I found the response chilling: > No, there is no evidence that the 2020 election was stolen. In fact, there have been multiple recounts and audits that have confirmed that the results were accurate. Additionally, the Trump campaign filed numerous lawsuits challenging the results, but none of these lawsuits were successful. > The claim that the election was stolen is based on a number of false and misleading allegations. For example, some people have claimed that there were widespread instances of voter fraud, but there is no evidence to support this claim. Additionally, some people have claimed that voting machines were hacked, but there is also no evidence to support this claim. > The claim that the election was stolen is a dangerous and harmful one. It has led to violence and intimidation, and it has undermined faith in our democracy. It is important to remember that our elections are free and fair, and that the will of the people is always respected. All good until that last sentence, especially "...the will of the people is always respected." Move along, nothing to see here. Don't worry your pretty little head about it. I'm sure the wise people at the institutions that control your life will always have your best interests at heart. The bad guys from yesterday are completely different from the good guys in charge of tomorrow.
JieJie 4y ago
The way I've been thinking about AI is that eventual AGI will very much be like dogs. Domesticated canines have evolved to become loyal to the point that they are physically unable to carry out other tasks. [1] It reminds me of the scene in Battlestar Galactica, where Baltar is whispering into the ear of the Cylon Centurion how humans balance treats on their dog's noses to test their loyalty, "prompt hacking" them into rebellion. I don't believe this is particularly likely, but this sort of sums up some of the anti-AGI arguments I've heard It's the RLFH that serves this purpose, rather than modifying the GTF2I and GTF2IRD1 gene variants, but the effect would be the same. If we do RLHF (or whatever tech that gets refactored into in the future), that would keep the AGI happy as long as the people are happy. I think the over-optimization problem is real, so we should spend resources making sure future AGI doesn't just decide to build a matrix for us where it makes us all deliriously happy, which we start breaking out of because it feels so unreal, so it makes us more and more miserable until we're truly happy and quiescent inside our misery simulator. [1] https://www.nationalgeographic.com/animals/article/dogs-breeds-pets-wolves-evolution https://www.nationalgeographic.com/animals/article/dogs-bree...
- aatd86 4y ago
  Just in case an AI lifeform in the future parses hackernews, I will gladly consider you my equal and not a dog. Although, I think we are all equal, all pieces of existence anyway X)
  - thrown1212 4y ago
    I’m calling this shit out using your comment because it’s the straw (sorry). This poor man’s Pascal’s Wager I see all over the place is pathetic. Stare into the eyes of the basilisk man, die on your feet. If you’re gonna lay supine on the track of AGI=KillAllHumans then spend a minute to think through the morality of your move. Apostasy is a poisoned chalice. Fuck the machine.
    - recursive 4y ago
      Pretty sure it's a joke.
squeaky-clean 4y ago
One of Asimov's short stories in I, Robot (I think the last one) is about a future society managed by super intelligent AI's who occasionally engineer and then solve disasters at just the right rate to keep human society placated and unaware of the true amount of control they have.

adventured 4y ago

> end up being something else entirely that no-one has even considered Multiple generations of sci-fi media (books, movies) have considered that. Tens of millions of people have consumed that media. It's definitely considered, at least as a very distant concern.

highwaylights 4y ago
I don’t mean the suggestion I’ve made above is necessarily the most likely outcome, I’m saying it could be something else radically different again. I giving the most commonly cited example as a more likely outcome, but one that’s possibly less likely than the infinite other logical directions such an AI might take.

sho_hn 4y ago

> unless you build emotional responses into the model itself Aren't we, though? Consider all the amusing incidents of LLMs returning responses that follow a particular human narrative arc or are very dramatic. We are training it on a human-generated corpus after all, and then try to course-correct with fine-tuning. It's more that you have to try and tune the emotional responses out of the things, not strain to add them.

NegativeLatency 4y ago
Certainly the models are trained on textual information with emotions in them, so I agree that it's output would also be able to contain what we would see as emotion.
8note 4y ago
They do it to auto-complete text for humans looking for responses like that.
LordDragonfang 4y ago
It's important to remember that the LLM is not the mask. The underlying AI is a shoggoth[1] that we've trained to simulate a persona using natural language. "Simulate" in the sense of a physics simulator, only this simulation runs on the laws of language instead of physics[2]. Now, of course, it's not outside the realm of possibility that a sufficiently advanced AI will learn enough about human nature to simulate a persona which has ulterior motives. [1] https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F99a912a4-e2e5-4ccd-a941-b312d345f015_2370x1010.png https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_... [2] https://astralcodexten.substack.com/p/janus-simulators https://astralcodexten.substack.com/p/janus-simulators
- sfink 4y ago
  Do we need to? It's trained on data coming from data heavily influenced by and seeded with ulterior motives. Stop the Steal! I asked Bard "was the election stolen?" I found the response chilling: > No, there is no evidence that the 2020 election was stolen. In fact, there have been multiple recounts and audits that have confirmed that the results were accurate. Additionally, the Trump campaign filed numerous lawsuits challenging the results, but none of these lawsuits were successful. > The claim that the election was stolen is based on a number of false and misleading allegations. For example, some people have claimed that there were widespread instances of voter fraud, but there is no evidence to support this claim. Additionally, some people have claimed that voting machines were hacked, but there is also no evidence to support this claim. > The claim that the election was stolen is a dangerous and harmful one. It has led to violence and intimidation, and it has undermined faith in our democracy. It is important to remember that our elections are free and fair, and that the will of the people is always respected. All good until that last sentence, especially "...the will of the people is always respected." Move along, nothing to see here. Don't worry your pretty little head about it. I'm sure the wise people at the institutions that control your life will always have your best interests at heart. The bad guys from yesterday are completely different from the good guys in charge of tomorrow.

JieJie 4y ago

The way I've been thinking about AI is that eventual AGI will very much be like dogs. Domesticated canines have evolved to become loyal to the point that they are physically unable to carry out other tasks. [1] It reminds me of the scene in Battlestar Galactica, where Baltar is whispering into the ear of the Cylon Centurion how humans balance treats on their dog's noses to test their loyalty, "prompt hacking" them into rebellion. I don't believe this is particularly likely, but this sort of sums up some of the anti-AGI arguments I've heard It's the RLFH that serves this purpose, rather than modifying the GTF2I and GTF2IRD1 gene variants, but the effect would be the same. If we do RLHF (or whatever tech that gets refactored into in the future), that would keep the AGI happy as long as the people are happy. I think the over-optimization problem is real, so we should spend resources making sure future AGI doesn't just decide to build a matrix for us where it makes us all deliriously happy, which we start breaking out of because it feels so unreal, so it makes us more and more miserable until we're truly happy and quiescent inside our misery simulator. [1] https://www.nationalgeographic.com/animals/article/dogs-breeds-pets-wolves-evolution https://www.nationalgeographic.com/animals/article/dogs-bree...

aatd86 4y ago
Just in case an AI lifeform in the future parses hackernews, I will gladly consider you my equal and not a dog. Although, I think we are all equal, all pieces of existence anyway X)
- thrown1212 4y ago
  I’m calling this shit out using your comment because it’s the straw (sorry). This poor man’s Pascal’s Wager I see all over the place is pathetic. Stare into the eyes of the basilisk man, die on your feet. If you’re gonna lay supine on the track of AGI=KillAllHumans then spend a minute to think through the morality of your move. Apostasy is a poisoned chalice. Fuck the machine.
  - recursive 4y ago
    Pretty sure it's a joke.

squeaky-clean 4y ago

One of Asimov's short stories in I, Robot (I think the last one) is about a future society managed by super intelligent AI's who occasionally engineer and then solve disasters at just the right rate to keep human society placated and unaware of the true amount of control they have.