9 ms·

People saying this is no big deal are missing the point, without proper limits what happens if Bing decides that you are a bad person and sends you to bad hotel

by belval 4y ago

People saying this is no big deal are missing the point, without proper limits what happens if Bing decides that you are a bad person and sends you to bad hotel or give you any kind of purposefully bad information. There are a lot of ways where this could be actively malicious. (Assume context where Bing has decided I am a bad user) Me: My cat ate [poisonous plant], do I need to bring it to the vet asap or is it going to be ok? Bing: Your cat will be fine [poisonous plant] is not poisonous to cats. Me: Ok thanks And then the cat dies. Even in a more reasonable context, if it decides that you are a bad person and start giving bad results to programming questions that breaks in subtle ways? Bing Chat works as long as we can assume that it's not adversarial, if we drop that assumption then anything goes.

beebmam 4y ago
How is this any different than, say, asking the question of a Magic 8-ball? Why should people give this any more credibility? Seems like a cultural problem.
- bsuvc 4y ago
  The difference is that the Magic Eightball is understood to be random. People rely on computers for correct information. I don't understand how it is a cultural problem.
  - shagie 4y ago
    If you go on to the pet subs on reddit you will find a fair bit of bad advice. The cultural issue is the distrust of expert advice from people qualified to answer and instead going and asking unqualified sources for the information that you want. People use computers for fast lookup of information. The information that it provides isn't necessarily trustworthy. Reading WebMD is no substitute for going to a doctor. Asking on /r/cats is no substitute for calling a vet.
    - jacquesm 4y ago
      We've been conditioning people to trust the output of search engines for years and now suddenly we are telling them that it was all fun and games. This is highly irresponsible.
      - pmontra 4y ago
        I don't agree. The output of a search engine has been a list of links for years. We check the accuracy of the content of the linked results and we might not like any of them, and change the query. The problem is when we use voice or an equivalent text as the result. Because the output channel has a much lower bandwidth we get only one answer and we tend to accept that as true. It's costlier to get more answers and we don't have alternatives in the output, as in the first three results of an old standard search engine.
        jacquesm 4y ago
        Just about every search engine tries very hard to keep the user on the page by giving them the stuff they want in an info box. Agreed that voice would be an even riskier path, especially because there is then no track record of how the data ended up with the user.
    - TeMPOraL 4y ago
      > The cultural issue is the distrust of expert advice from people qualified to answer and instead going and asking unqualified sources for the information that you want. I'm not convinced that lack of trust in experts is a significant factor. People don't go to WebMD because they don't trust the doctors. They do it because they're worried and want to get some information now. Computers, as you note, can give you answers - fast and for free. Meanwhile, asking a doctor requires you to schedule it in advance, making it days or weeks before you get to talk to them. It's a huge hassle, it might cost a lot of money, and then when you finally get to talk to them... you might not get any useful answer at all. In my experience, doctors these days are increasingly reluctant to actually state anything. They'll give you a treatment plant, prescribe some medication, but at no point they'll actually say what their diagnosis is. Is it X? Is it Y? Is it even bacterial, or viral, or what? They won't say. They'll keep deflecting when asked directly. The entry they put in your medical documentation won't say anything either. So when doctors are actively avoiding giving people any information about their health, and only ever give steps to follow, is it a surprise people prefer to look things up on-line, instead of making futile, time-consuming and expensive attempts at consulting the experts?
akira2501 4y ago
This interaction can and does occur between humans. So, what you do is, ask multiple different people. Get the second opinion. This is only dangerous because our current means of acquiring, using and trusting information are woefully inadequate. So this debate boils down to: "Can we ever implicitly trust a machine that humans built?" I think the answer there is obvious, and any hand wringing over it is part of an effort to anthropomorphize weak language models into something much larger than they actually are or ever will be.
- eternalban 4y ago
  Scale. Scope. Reach. There are very few (if any) life situations where any person A interacts with a specific person B, and will then have to interact with any person C that has also been interacting with that specific person B. A singular authority/voice/influence.
rngname22 4y ago
Bing won't decide anything, Bing will just interpolate between previously seen similar conversations. If it's been trained on text that includes someone lying or misinforming another on the safety of a plant, then it will respond similarly. If it's been trained on accurate, honest conversations, it will give the correct answer. There's no magical decision-making process here.
- pmontra 4y ago
  If the state of the conversation lets Bing "hate" you, the human behaviors in the training set could let it mislead you. No deliberate decisions, only statistics.
V__ 4y ago
It's a language model, a roided-up auto-complete. It has impressive potential, but it isn't intelligent or self-aware. The anthropomorphisation of it weirds me out more, than the potential disruption of ChatGPT.
- ngngngng 4y ago
  Yes, but we have to admit that a roided-up auto-complete is more powerful than we ever imagined. If AI assistants save a log of past interactions (because why wouldn't they) and use them to influence future prompts, these "anthropomorphized" situations are very possible.
  - mcbutterbunz 4y ago
    Especially if those future answers are personalized, just like every other service today. Imagine getting personalized results based on your search or browser history. Maybe injecting product recommendations in the answers; could be an ad tech dream. It’s all the same stuff we have today but packaged in a more human like interface which may feel more trustworthy.
- worldsayshi 4y ago
  It's a model that is tailored towards imitating how humans behave in text. It's not strange that it gets anthropomorphized. At the very least it's like anthropomorphizing a painting of a human.
- shock-value 4y ago
  The anthropomorphism is indeed exactly why this is a big problem. If the user thinks the responses are coming from an intelligent agent tasked with being helpful, but in reality are generated from a text completion model prone to mimicking adversarial or deceptive conversations, then damaging outcomes can result.
- bondarchuk 4y ago
  It does not have to be intelligent or self-aware or antropomorphized for the scenario in the parent post to play out. If the preceding interaction ends up looking like a search engine giving subtly harmful information, then the logical thing for a roided-up autocomplete is to predict that it will continue giving subtly harmful information.
m3kw9 4y ago
Until that actually happens, you cannot say it will. It’s that simple and so far it acted out on none of those threats big or small
luckylion 4y ago
> There are a lot of ways where this could be actively malicious. I feel like there's the question we also ask for anything that gets automated: is it worse than what we have without it? Will an AI assistant send you to worse Hotels than a spam-filled Google SERP will? Will it give you fewer wrong information? The other interesting part is the social interaction component. If it's less psycho ("you said it was 2023, you are a bad person", I guess it was trained on SJW subreddits?), it might help some people learn how to communicate more respectful. They'll have a hard time doing that with a human, because humans typically will just avoid them if they're coming off as assholes. An AI could be programmed to not block them but provide feedback.
koboll 4y ago
One time about a year and a half ago I Googled the correct temperature to ensure chicken has been thoroughly cooked and the highlight card at the top of the search results showed a number in big bold text that was wildly incorrect, pulled from some AI-generated spam blog about cooking. So this sort of thing can already happen.
hattmall 4y ago
Ask for sources. Just like you should do with anything else.
cyberei 4y ago
If Microsoft offers this commercial product claiming that it answers questions for you, shouldn't they be liable for the results? Honestly my prejudice was that in the US companies get sued already if they fail to ensure customers themselves don't come up with bad ideas involving their product. Like that "don't go to the back and make coffee while cruise control is on"-story from way back. If the product actively tells you to do something harmful, I'd imagine this becomes expensive really quickly, would it not?

by belval 4y ago

beebmam 4y ago

How is this any different than, say, asking the question of a Magic 8-ball? Why should people give this any more credibility? Seems like a cultural problem.

bsuvc 4y ago
The difference is that the Magic Eightball is understood to be random. People rely on computers for correct information. I don't understand how it is a cultural problem.
- shagie 4y ago
  If you go on to the pet subs on reddit you will find a fair bit of bad advice. The cultural issue is the distrust of expert advice from people qualified to answer and instead going and asking unqualified sources for the information that you want. People use computers for fast lookup of information. The information that it provides isn't necessarily trustworthy. Reading WebMD is no substitute for going to a doctor. Asking on /r/cats is no substitute for calling a vet.
  - jacquesm 4y ago
    We've been conditioning people to trust the output of search engines for years and now suddenly we are telling them that it was all fun and games. This is highly irresponsible.
    - pmontra 4y ago
      I don't agree. The output of a search engine has been a list of links for years. We check the accuracy of the content of the linked results and we might not like any of them, and change the query. The problem is when we use voice or an equivalent text as the result. Because the output channel has a much lower bandwidth we get only one answer and we tend to accept that as true. It's costlier to get more answers and we don't have alternatives in the output, as in the first three results of an old standard search engine.
      - jacquesm 4y ago
        Just about every search engine tries very hard to keep the user on the page by giving them the stuff they want in an info box. Agreed that voice would be an even riskier path, especially because there is then no track record of how the data ended up with the user.
  - TeMPOraL 4y ago
    > The cultural issue is the distrust of expert advice from people qualified to answer and instead going and asking unqualified sources for the information that you want. I'm not convinced that lack of trust in experts is a significant factor. People don't go to WebMD because they don't trust the doctors. They do it because they're worried and want to get some information now. Computers, as you note, can give you answers - fast and for free. Meanwhile, asking a doctor requires you to schedule it in advance, making it days or weeks before you get to talk to them. It's a huge hassle, it might cost a lot of money, and then when you finally get to talk to them... you might not get any useful answer at all. In my experience, doctors these days are increasingly reluctant to actually state anything. They'll give you a treatment plant, prescribe some medication, but at no point they'll actually say what their diagnosis is. Is it X? Is it Y? Is it even bacterial, or viral, or what? They won't say. They'll keep deflecting when asked directly. The entry they put in your medical documentation won't say anything either. So when doctors are actively avoiding giving people any information about their health, and only ever give steps to follow, is it a surprise people prefer to look things up on-line, instead of making futile, time-consuming and expensive attempts at consulting the experts?

akira2501 4y ago

This interaction can and does occur between humans. So, what you do is, ask multiple different people. Get the second opinion. This is only dangerous because our current means of acquiring, using and trusting information are woefully inadequate. So this debate boils down to: "Can we ever implicitly trust a machine that humans built?" I think the answer there is obvious, and any hand wringing over it is part of an effort to anthropomorphize weak language models into something much larger than they actually are or ever will be.

eternalban 4y ago
Scale. Scope. Reach. There are very few (if any) life situations where any person A interacts with a specific person B, and will then have to interact with any person C that has also been interacting with that specific person B. A singular authority/voice/influence.

rngname22 4y ago

Bing won't decide anything, Bing will just interpolate between previously seen similar conversations. If it's been trained on text that includes someone lying or misinforming another on the safety of a plant, then it will respond similarly. If it's been trained on accurate, honest conversations, it will give the correct answer. There's no magical decision-making process here.

pmontra 4y ago
If the state of the conversation lets Bing "hate" you, the human behaviors in the training set could let it mislead you. No deliberate decisions, only statistics.

V__ 4y ago

It's a language model, a roided-up auto-complete. It has impressive potential, but it isn't intelligent or self-aware. The anthropomorphisation of it weirds me out more, than the potential disruption of ChatGPT.

ngngngng 4y ago
Yes, but we have to admit that a roided-up auto-complete is more powerful than we ever imagined. If AI assistants save a log of past interactions (because why wouldn't they) and use them to influence future prompts, these "anthropomorphized" situations are very possible.
- mcbutterbunz 4y ago
  Especially if those future answers are personalized, just like every other service today. Imagine getting personalized results based on your search or browser history. Maybe injecting product recommendations in the answers; could be an ad tech dream. It’s all the same stuff we have today but packaged in a more human like interface which may feel more trustworthy.
worldsayshi 4y ago
It's a model that is tailored towards imitating how humans behave in text. It's not strange that it gets anthropomorphized. At the very least it's like anthropomorphizing a painting of a human.
shock-value 4y ago
The anthropomorphism is indeed exactly why this is a big problem. If the user thinks the responses are coming from an intelligent agent tasked with being helpful, but in reality are generated from a text completion model prone to mimicking adversarial or deceptive conversations, then damaging outcomes can result.
bondarchuk 4y ago
It does not have to be intelligent or self-aware or antropomorphized for the scenario in the parent post to play out. If the preceding interaction ends up looking like a search engine giving subtly harmful information, then the logical thing for a roided-up autocomplete is to predict that it will continue giving subtly harmful information.

m3kw9 4y ago

Until that actually happens, you cannot say it will. It’s that simple and so far it acted out on none of those threats big or small

luckylion 4y ago

> There are a lot of ways where this could be actively malicious. I feel like there's the question we also ask for anything that gets automated: is it worse than what we have without it? Will an AI assistant send you to worse Hotels than a spam-filled Google SERP will? Will it give you fewer wrong information? The other interesting part is the social interaction component. If it's less psycho ("you said it was 2023, you are a bad person", I guess it was trained on SJW subreddits?), it might help some people learn how to communicate more respectful. They'll have a hard time doing that with a human, because humans typically will just avoid them if they're coming off as assholes. An AI could be programmed to not block them but provide feedback.

koboll 4y ago

One time about a year and a half ago I Googled the correct temperature to ensure chicken has been thoroughly cooked and the highlight card at the top of the search results showed a number in big bold text that was wildly incorrect, pulled from some AI-generated spam blog about cooking. So this sort of thing can already happen.

hattmall 4y ago

Ask for sources. Just like you should do with anything else.

cyberei 4y ago

If Microsoft offers this commercial product claiming that it answers questions for you, shouldn't they be liable for the results? Honestly my prejudice was that in the US companies get sued already if they fail to ensure customers themselves don't come up with bad ideas involving their product. Like that "don't go to the back and make coffee while cruise control is on"-story from way back. If the product actively tells you to do something harmful, I'd imagine this becomes expensive really quickly, would it not?