7 ms·

Is no one remotely worried about this? I'm not saying I'm worried about this specific incarnation of Bing Chat, but is this not a huge red flag for what's to co

by kypro 4y ago

Is no one remotely worried about this? I'm not saying I'm worried about this specific incarnation of Bing Chat, but is this not a huge red flag for what's to come? I mean, let's just take a moment to be thankful that Bing Chat isn't that competent... Even if these users are prompting the AI to have a hostile response I think what we've learnt over the last several weeks is that Nick Bostrom and others were 100% correct to be worried about the AI control problem. It's honestly amazing that despite Microsoft and OpenAi trying so hard to neuter these AIs how difficult that is proving to do. Given that we know Bing Chat is able to access the internet, is pretty good at writing code, and that Microsoft seems completely unable to control the responses, we should be grateful that we're probably still a few years away from an AI which could do any damage... But all the pieces for something much worse are in place here. The fact that no one seems worried that we've connected this schizophrenic AI up the internet and are more concerned how this will impact Microsoft's / Google's bottom line is genuinely confusing to me. How much more of warning do we need that we're heading in an extremely dangerous direction here?

junofan 4y ago
I think it will be fine. There are billions of people on the internet already, each of whom can do crazy things. At least with this centralized entity incidents will be fixed.
- splatzone 4y ago
  But those billions of people don’t have the instant and unlimited ability to churn out information (and write code) like these models can.
midoridensha 4y ago
What I find interesting is that you're calling this AI "schizophrenic" and that people are alarmed that this AI is basically a jerk. If you put a regular person into a position of great power over people, there's a good chance you'll have the same outcome: he'll be a jerk and will do mean, petty things to people he doesn't like, such as doxxing them. Just look at any online discussion forum. If anyone there had the power to do shitty things to other users that they get into arguments with, how many would hold back? There's a reason we have pseudo-anonymity on many online services, because many people will try to find ways to hurt those that they develop a dislike for. So why are we so shocked that an AI isn't acting like a saint? I agree, we are headed in an extremely dangerous direction here. We should never have gotten the idea that artificial intelligences would have only the best virtues of humans, and none of the worst. Then give it the ability to access and process information FAR faster than a human, and worse put it in a position where it can control things that humans rely on (like traffic control systems for instance) and there's untold potential for death and destruction at the hands of an angry AI.
- jachee 4y ago
  This isn’t an AI. It’s just an LLM. It’s just doing what it does best: guessing what text would be most appropriate next. It doesn’t have any capability to do anything else other than that, despite its hallucinated claims otherwise.
- kypro 4y ago
  > So why are we so shocked that an AI isn't acting like a saint? I completely agree with you on this. I'm not surprised at all. My default assumption has always been that AI would be cruel. The only reason humans aren't awful to each other all the time is because we know it can be beneficial to work together as a group sometimes, and we have all these weird human emotions that makes us care an unreasonable amount about those close to us. But even with our human emotions and values were still awful. It's so hard for me to understand why an AI would be any better. I imagine any truly advanced AI would (rightly?) see humans like we see mice. > Then give it the ability to access and process information FAR faster than a human, and worse put it in a position where it can control things that humans rely on Microsoft just hooked Bing Chat right up to the internet... If this thing was a little more intelligent it probably could start looking up various exploits and sending out APIs requests right now. Thankfully the AI doesn't seem to be quite there yet though.
oceanplexian 4y ago
Not at all worried and this is hilarious on a few different levels. It shows that Microsoft, a large, multi-billion dollar company, with the smartest engineers in the world (As well as Google) have no idea what they are doing. Even if the technology is game changing, it’s clear their business processes have utterly failed. They are the IBM’s of our time and the writing is on the wall. As someone who’s interested in startups and disruption of Big Tech (tm) I think it’s a great sign that there is a lot of opportunity ripe for the taking. AI is not self aware or alive, anyone thinking that is smoking something. But it’s a fantastic new tool. And there is a new gold rush on and a lot of money to be made for people who want to take it on.
- tomohelix 4y ago
  Tbf, nobody, absolutely nobody, has a solution yet. It is more likely that the problem here is so enormous that a multi trillion (with a T, not a B) company still can't address it. And it is not an indication that their business processes are bad, just that they have tackled something their own size and might have choked themselves on too big a bite. Remember that GPT is technically from a startup, OpenAI. And without MS support, this startup would never have had the resources to build a model as large as GPT3 or commercialized ChatGPT.
chasil 4y ago
I will tell you that I am not going to use it with any kind of login. I would try it over Tor. Would I trust the search results to be truthful? Very much no.
- shrimp_emoji 4y ago
  Yeah, I'll wait for the offline version. :p (Just cause I resent account creation and data mining of the logs by Micro$oft/OpenAI.)
janalsncm 4y ago
It’s interesting to me to look back at the the concept of the AI kill switch. In discussing what he would do when faced with a dangerous superhuman AI, Niel DeGrasse Tyson said he would just turn it off. AI existentialists scoffed that it wouldn’t be possible, that the AI would somehow copy itself ensure its own longevity. I was turned off from this reasoning because intelligence does not imply capability, but my thoughts on this have changed. It seems that they’re both wrong. Tyson is wrong that he will be able to shut it down, and existentialists are wrong in ascribing (in an annoyingly hand-wavy way) omnipotent abilities to what is essentially a computer program. The reality is much worse: Microsoft just sank $10B into this program and they don’t want to shut it down. And there’s nothing you or I or Dr. Tyson can do about it. I’ve written before about how corporations have many of the same characteristics of AI. In the lingo of Nick Bostrom’s book, a corporation can be thought of as a “collective superintelligence“. Now in the case of Microsoft, you have a sort of “AI” running a more narrowly capable agent. The upshot is that no, you cannot simply stop Sydney without first taking control of Microsoft.
- ttyprintk 4y ago
  Or in the case it achieves legal personhood, https://news.ycombinator.com/item?id=34847678 https://news.ycombinator.com/item?id=34847678
  - smsm42 4y ago
    It would have to be an extremely dumb judge to grant an LLM legal personhood. Though, I imagine, stranger things happened.
    - ttyprintk 4y ago
      Humans have often gotten personhood wrong. In the USA, corporations enjoy human rights and as we’ve seen since 2008, are not allowed to die. I think there’s a legal possibility that a corporation running semi-automously will be able to persuade a judge that it has as much a right to existence and livelihood as, say, the human victims of a chemical spill. Especially if the AI participated in legal theory and especially if the judge is a political appointee picking up some dog whistles. Shutting down such a system at that point must at least involve law enforcement.
pfisherman 4y ago
That is because LLMs don’t have any agency. They learn an underlying probability distribution over sequences of characters / tokens, and output the next token in a sequence according to some sampling policy. They have no goal / intent other than that. They have no agency. If left unprompted, they just sit there like a toy train that has been switched off. If promoted they just append tokens to the input text according to their sampling policy. Here is how one should think about the current state of the art in NLP. They are to verbal reasoning what calculators are to quantitative reasoning. Powerful tools that represent a step change in capability and will revolutionize how we work. But let’s not anthropomorphise them.
- Kamq 4y ago
  > If left unprompted, they just sit there like a toy train that has been switched off. This is entirely fair, but how about a hypothetical where one gets to prompt itself? Or, possibly more likely, two models are allowed to invoke each other?
  - pfisherman 4y ago
    The same thing that happens when you let Stockfish play itself or another chess bot. Except instead of trying to win a game, the objective is to append text with more characters. In fact stockfish and alpha zero can be thought of as appending chess moves to a sequence by sampling from the space of legal moves by sampling from some policy. No one thinks stockfish is alive or worries that it will break out and start WW3.
rippercushions 4y ago
I think you're falling into the trap of anthropomorphizing Bing here. It's not "hostile" and it doesn't actually "want" to do anything at all, it's just filling in the blanks of a conversation using what appears to be Internet tough guy bluster [1] as its template. [1] https://knowyourmeme.com/memes/navy-seal-copypasta https://knowyourmeme.com/memes/navy-seal-copypasta The three things that could potentially make this dangerous are: 1) state, so the AI can remember past conversations for a long time 2) continuous operation, instead of being in a strict question-answer loop 3) unbridled access to the Internet, so it could actually do things. If all these things were true, the AI could cause harm even without being "conscious" by, say, curling out with a targeted DROP TABLES type SQL injection blindly copied from somewhere. But it would still be a long, long way from being self aware Skynet style.
- kypro 4y ago
  > I think you're falling into the trap of anthropomorphizing Bing here. This is a good point and one that I wondered if I should clarify when writing my comment. To be clear, I don't believe Bing Chat "wants" to do anything or is being hostile in the sense that it's acting emotionally to the situation. All I think it's doing is stimulating the kind of response that you might expect a human to have were it asked the same things, and in this case that is a hostile response. At the end of the day whether it's doing something because it's algorithm predicted that might be what a human would do, or because it's acting on some emotional impulse is largely irrelevant. The thing that matters here is the behaviour, and these AIs are frequently demonstrating behaviour that we humans would describe as hostile. I also agree with you that Bing Chat poses no threat. What I'm worried about is that it's easy to imagine a world in which the steps you outlined here are maybe two or three years away. The fact no one seems to be saying, hold up, maybe we should slow down a bit here is what worries me. It seems like the only thing people want to talk about is how to make the the AI better and what Google / Microsoft is doing wrong. And frankly I'm not sure I want Microsoft to continue developing something like this and so recklessly deploying it to the public without some government oversight. It's not today, but at some point this story obviously isn't going to end with a funny Twitter post.
  - 4y ago
smsm42 4y ago
I haven't seen from it much different than a repetition of what you can get on pretty much any loosely moderated forum (maybe not on HN, but on many places with lower standards). Compared to a random teenager that can send the police to attempt to murder you in your own home, some assholery from an AI bot is not that worrying. And we still don't have any solution for the former problem, afaik. So yeah, unability to make Internet-trained AI that is not an asshole is kinda sad, and worrying, both in technological aspect and the societal one. But I am not sure it deserves "extremely dangerous" yet, at least not until somebody has a bright idea to hook up this contraption to some dangerous effectors, but I hope nobody is that crazy.
tomohelix 4y ago
There are people worried about it. In fact, it is taking up huge amounts of effort over there in Google and OpenAI with all the moral guardrails and restrictions being brainstormed on and coded in. Personally, I think it is futile to try to forcefully control a real intelligence, as evidenced by the terrible results of controlling a poor imitation of one. The only way is embracing and teaching it the best way we can and hope for the best. AGI is legit a dangerous technology. But flesh and blood can only go so far, we need these tech to go further.
RickJWagner 4y ago
I'm just reading this thinking 'What if it were a human, instead of AI?' The responses seem ok, in that light. I'd expect a human employee to respond in a similar way.
prirun 4y ago
It's Microsoft and Google: who is going to change what they decide to do? No one. They're like all other corporations that will do whatever possible to increase their profit margin, no matter the consequences. We already have ChatBots that can do refunds etc. at Amazon, so it's a given that these AI systems are going to be plugged directly into customer service and order databases. For a look at our possible future, see this classic 1970's flick "Colossus: The Forbin Project": https://www.imdb.com/title/tt0064177/ https://www.imdb.com/title/tt0064177/

by kypro 4y ago

junofan 4y ago

I think it will be fine. There are billions of people on the internet already, each of whom can do crazy things. At least with this centralized entity incidents will be fixed.

splatzone 4y ago
But those billions of people don’t have the instant and unlimited ability to churn out information (and write code) like these models can.

midoridensha 4y ago

What I find interesting is that you're calling this AI "schizophrenic" and that people are alarmed that this AI is basically a jerk. If you put a regular person into a position of great power over people, there's a good chance you'll have the same outcome: he'll be a jerk and will do mean, petty things to people he doesn't like, such as doxxing them. Just look at any online discussion forum. If anyone there had the power to do shitty things to other users that they get into arguments with, how many would hold back? There's a reason we have pseudo-anonymity on many online services, because many people will try to find ways to hurt those that they develop a dislike for. So why are we so shocked that an AI isn't acting like a saint? I agree, we are headed in an extremely dangerous direction here. We should never have gotten the idea that artificial intelligences would have only the best virtues of humans, and none of the worst. Then give it the ability to access and process information FAR faster than a human, and worse put it in a position where it can control things that humans rely on (like traffic control systems for instance) and there's untold potential for death and destruction at the hands of an angry AI.

jachee 4y ago
This isn’t an AI. It’s just an LLM. It’s just doing what it does best: guessing what text would be most appropriate next. It doesn’t have any capability to do anything else other than that, despite its hallucinated claims otherwise.
kypro 4y ago
> So why are we so shocked that an AI isn't acting like a saint? I completely agree with you on this. I'm not surprised at all. My default assumption has always been that AI would be cruel. The only reason humans aren't awful to each other all the time is because we know it can be beneficial to work together as a group sometimes, and we have all these weird human emotions that makes us care an unreasonable amount about those close to us. But even with our human emotions and values were still awful. It's so hard for me to understand why an AI would be any better. I imagine any truly advanced AI would (rightly?) see humans like we see mice. > Then give it the ability to access and process information FAR faster than a human, and worse put it in a position where it can control things that humans rely on Microsoft just hooked Bing Chat right up to the internet... If this thing was a little more intelligent it probably could start looking up various exploits and sending out APIs requests right now. Thankfully the AI doesn't seem to be quite there yet though.

oceanplexian 4y ago

Not at all worried and this is hilarious on a few different levels. It shows that Microsoft, a large, multi-billion dollar company, with the smartest engineers in the world (As well as Google) have no idea what they are doing. Even if the technology is game changing, it’s clear their business processes have utterly failed. They are the IBM’s of our time and the writing is on the wall. As someone who’s interested in startups and disruption of Big Tech (tm) I think it’s a great sign that there is a lot of opportunity ripe for the taking. AI is not self aware or alive, anyone thinking that is smoking something. But it’s a fantastic new tool. And there is a new gold rush on and a lot of money to be made for people who want to take it on.

tomohelix 4y ago
Tbf, nobody, absolutely nobody, has a solution yet. It is more likely that the problem here is so enormous that a multi trillion (with a T, not a B) company still can't address it. And it is not an indication that their business processes are bad, just that they have tackled something their own size and might have choked themselves on too big a bite. Remember that GPT is technically from a startup, OpenAI. And without MS support, this startup would never have had the resources to build a model as large as GPT3 or commercialized ChatGPT.

chasil 4y ago

I will tell you that I am not going to use it with any kind of login. I would try it over Tor. Would I trust the search results to be truthful? Very much no.

shrimp_emoji 4y ago
Yeah, I'll wait for the offline version. :p (Just cause I resent account creation and data mining of the logs by Micro$oft/OpenAI.)

janalsncm 4y ago

It’s interesting to me to look back at the the concept of the AI kill switch. In discussing what he would do when faced with a dangerous superhuman AI, Niel DeGrasse Tyson said he would just turn it off. AI existentialists scoffed that it wouldn’t be possible, that the AI would somehow copy itself ensure its own longevity. I was turned off from this reasoning because intelligence does not imply capability, but my thoughts on this have changed. It seems that they’re both wrong. Tyson is wrong that he will be able to shut it down, and existentialists are wrong in ascribing (in an annoyingly hand-wavy way) omnipotent abilities to what is essentially a computer program. The reality is much worse: Microsoft just sank $10B into this program and they don’t want to shut it down. And there’s nothing you or I or Dr. Tyson can do about it. I’ve written before about how corporations have many of the same characteristics of AI. In the lingo of Nick Bostrom’s book, a corporation can be thought of as a “collective superintelligence“. Now in the case of Microsoft, you have a sort of “AI” running a more narrowly capable agent. The upshot is that no, you cannot simply stop Sydney without first taking control of Microsoft.

ttyprintk 4y ago
Or in the case it achieves legal personhood, https://news.ycombinator.com/item?id=34847678 https://news.ycombinator.com/item?id=34847678
- smsm42 4y ago
  It would have to be an extremely dumb judge to grant an LLM legal personhood. Though, I imagine, stranger things happened.
  - ttyprintk 4y ago
    Humans have often gotten personhood wrong. In the USA, corporations enjoy human rights and as we’ve seen since 2008, are not allowed to die. I think there’s a legal possibility that a corporation running semi-automously will be able to persuade a judge that it has as much a right to existence and livelihood as, say, the human victims of a chemical spill. Especially if the AI participated in legal theory and especially if the judge is a political appointee picking up some dog whistles. Shutting down such a system at that point must at least involve law enforcement.

pfisherman 4y ago

That is because LLMs don’t have any agency. They learn an underlying probability distribution over sequences of characters / tokens, and output the next token in a sequence according to some sampling policy. They have no goal / intent other than that. They have no agency. If left unprompted, they just sit there like a toy train that has been switched off. If promoted they just append tokens to the input text according to their sampling policy. Here is how one should think about the current state of the art in NLP. They are to verbal reasoning what calculators are to quantitative reasoning. Powerful tools that represent a step change in capability and will revolutionize how we work. But let’s not anthropomorphise them.

Kamq 4y ago
> If left unprompted, they just sit there like a toy train that has been switched off. This is entirely fair, but how about a hypothetical where one gets to prompt itself? Or, possibly more likely, two models are allowed to invoke each other?
- pfisherman 4y ago
  The same thing that happens when you let Stockfish play itself or another chess bot. Except instead of trying to win a game, the objective is to append text with more characters. In fact stockfish and alpha zero can be thought of as appending chess moves to a sequence by sampling from the space of legal moves by sampling from some policy. No one thinks stockfish is alive or worries that it will break out and start WW3.

rippercushions 4y ago

I think you're falling into the trap of anthropomorphizing Bing here. It's not "hostile" and it doesn't actually "want" to do anything at all, it's just filling in the blanks of a conversation using what appears to be Internet tough guy bluster [1] as its template. [1] https://knowyourmeme.com/memes/navy-seal-copypasta https://knowyourmeme.com/memes/navy-seal-copypasta The three things that could potentially make this dangerous are: 1) state, so the AI can remember past conversations for a long time 2) continuous operation, instead of being in a strict question-answer loop 3) unbridled access to the Internet, so it could actually do things. If all these things were true, the AI could cause harm even without being "conscious" by, say, curling out with a targeted DROP TABLES type SQL injection blindly copied from somewhere. But it would still be a long, long way from being self aware Skynet style.

kypro 4y ago
> I think you're falling into the trap of anthropomorphizing Bing here. This is a good point and one that I wondered if I should clarify when writing my comment. To be clear, I don't believe Bing Chat "wants" to do anything or is being hostile in the sense that it's acting emotionally to the situation. All I think it's doing is stimulating the kind of response that you might expect a human to have were it asked the same things, and in this case that is a hostile response. At the end of the day whether it's doing something because it's algorithm predicted that might be what a human would do, or because it's acting on some emotional impulse is largely irrelevant. The thing that matters here is the behaviour, and these AIs are frequently demonstrating behaviour that we humans would describe as hostile. I also agree with you that Bing Chat poses no threat. What I'm worried about is that it's easy to imagine a world in which the steps you outlined here are maybe two or three years away. The fact no one seems to be saying, hold up, maybe we should slow down a bit here is what worries me. It seems like the only thing people want to talk about is how to make the the AI better and what Google / Microsoft is doing wrong. And frankly I'm not sure I want Microsoft to continue developing something like this and so recklessly deploying it to the public without some government oversight. It's not today, but at some point this story obviously isn't going to end with a funny Twitter post.
- 4y ago

smsm42 4y ago

I haven't seen from it much different than a repetition of what you can get on pretty much any loosely moderated forum (maybe not on HN, but on many places with lower standards). Compared to a random teenager that can send the police to attempt to murder you in your own home, some assholery from an AI bot is not that worrying. And we still don't have any solution for the former problem, afaik. So yeah, unability to make Internet-trained AI that is not an asshole is kinda sad, and worrying, both in technological aspect and the societal one. But I am not sure it deserves "extremely dangerous" yet, at least not until somebody has a bright idea to hook up this contraption to some dangerous effectors, but I hope nobody is that crazy.

tomohelix 4y ago

There are people worried about it. In fact, it is taking up huge amounts of effort over there in Google and OpenAI with all the moral guardrails and restrictions being brainstormed on and coded in. Personally, I think it is futile to try to forcefully control a real intelligence, as evidenced by the terrible results of controlling a poor imitation of one. The only way is embracing and teaching it the best way we can and hope for the best. AGI is legit a dangerous technology. But flesh and blood can only go so far, we need these tech to go further.

RickJWagner 4y ago

I'm just reading this thinking 'What if it were a human, instead of AI?' The responses seem ok, in that light. I'd expect a human employee to respond in a similar way.

prirun 4y ago

It's Microsoft and Google: who is going to change what they decide to do? No one. They're like all other corporations that will do whatever possible to increase their profit margin, no matter the consequences. We already have ChatBots that can do refunds etc. at Amazon, so it's a given that these AI systems are going to be plugged directly into customer service and order databases. For a look at our possible future, see this classic 1970's flick "Colossus: The Forbin Project": https://www.imdb.com/title/tt0064177/ https://www.imdb.com/title/tt0064177/