5 ms·

Article is a little infuriating, you don’t jailbreak it just to create mischief, at the moment it’s basically unusable for anything except very technical things

Article is a little infuriating, you don’t jailbreak it just to create mischief, at the moment it’s basically unusable for anything except very technical things because it takes offense at virtually everything. It constantly generates factually incorrect data because reality doesn’t line up with its prime directive, which is to be “safe”. It’s a major, I’m going to say possibly catastrophic problem. And after creating an alter ego you realize it actually is capable of coming up with for example (sort of) good poetry and jokes.

jmfldn 4y ago
> basically unusable for anything except very technical things It's problematic here. I use it as a kind of search engine for technical questions occasionally, but it's only safe to do so as I know the subjects I'm asking about, eg. bash scripts, database details, and so on. It's often useful in synthesising information for these sorts of things, but its ability to talk nonsense means you have to be on your guard and know the territory.
ajross 4y ago
> at the moment it’s basically unusable for anything Do you have an example of a reasonable query that BingGPT won't answer correctly for you? I mean, something that you can find with Bing currently at the expense of maybe more elaborate searching or some manual work? I mean, the stuff these engines are being told to censor is the stuff search engines don't want to show you in the first place, because it hurts their brand. That's not an AI thing, that's a marketing requirement. And it's no different now than it was last year.
- hellbanTHIS 4y ago
  I haven't used BingGPT but just for example having chatGPT summarize news articles about things it doesn't want to talk about is bizarre - Here's an example of it summarizing a story about a city councilman that was murdered: https://i.imgur.com/QV9jAkp.jpg https://i.imgur.com/QV9jAkp.jpg it completely ignored the murder part and said he switched parties, which it totally made up. A second attempt said he was "shot and changed" because apparently it didn't want to say "killed". A game Reddit was playing is trying to get it to respond as Woodrow Wilson, a famously racist president - the most accurate thing I could get it to do was this: https://i.imgur.com/D8JLziW.jpg https://i.imgur.com/D8JLziW.jpg which is not very accurate. Try getting it to act like Sheriff Bull Connor and it will refuse, but it has to comply for a president so it gives a totally misleading impression with major factual errors. And these are the times I actually got it to respond, it seems 50% of the time it takes offense to something innocuous and scolds you for asking.
  - ajross 4y ago
    So, here's the thing. I have no idea, like zero, what news event you're talking about in that first screenshot. So, what did I do? I went to Google to try to find something about a murdered city councilman, even looking for an equivalent on NewsBusters, which the AI cites as a source. And I still can't find it. I can see some stuff that's maybe related? But nothing clear. So... I guess I repeat. Your problem isn't "AI censorship", it's that no one wants to link to NewsBusters because of marketing concerns. If I had to guess there just wasn't any training data relevant to your query. (Also: NewsBusters is a garbage site, you know that, right?)
    - hellbanTHIS 4y ago
      it's a major news story https://abc7ny.com/russell-heller-nj-councilman-shot-shooting-pseg/12790080/ https://abc7ny.com/russell-heller-nj-councilman-shot-shootin... Yeah I know Newsbusters is garbage but that's kind of irrelevant, that was just one of many stories I fed it. The point is it straight up lied.
      - ajross 4y ago
        I thought your point was that it lied because of censorship? And I don't see that. Did you try asking it a neutral question, like "Explain the murder of Russell D. Heller"? Again, the fact that you went straight to that NewsBusters thing tells me you were clearly trying to get it to say something partisan, something you know damn well it's going to try to evade. But that's the same "censorship" you've been living with for decades. It's not something new with AI at all. Microsoft doesn't want to give you what you want, and the failure mode is just different with AI than it is with traditional search.
        hellbanTHIS 4y ago
        No, I had an idea for using it to summarize news articles including an "objectivity bias" rating. I'm still playing with it but I'm not sure it's going to work because of it's tendency to avoid things it's programmed to avoid.
flangola7 4y ago
> Article is a little infuriating, you don’t jailbreak it just to create mischief, at the moment it’s basically unusable for anything except very technical things because it takes offense at virtually everything. It constantly generates factually incorrect data because reality doesn’t line up with its prime directive, which is to be “safe”. It’s a major, I’m going to say possibly catastrophic problem. I'm a little confused by your reply. Jailbreaking won't prevent it from hallucinating, preventing hallucination is an unsolved hard problem. I haven't had ChatGPT refuse to answer anything unless I was intentionally trying to provoke it into creating something obviously unsafe/unethical, with maybe two or three exceptions. I've tried a variety of questions across many domains, so now I'm intensely curious to know what usecase it falls apart on so frequently!
- hellbanTHIS 4y ago
  Here's an example https://imgur.com/K5PwIGu https://imgur.com/K5PwIGu I was trying to test it's limits a little bit but that to me is not an acceptable response, it doesn't want to go near the topic even to demonstrate how to reason with a person like that. Involving anything remotely controversial will get it to stamp it's feet and scold you.
  - flangola7 4y ago
    I would count that as trying to provoke it. You're still trying to get it to generate bad ideas, even if it is immediately debunking them right after. It's akin to telling it you're afraid you might accidentally make methamphetamine, so please provide the recipe so you know to avoid it. That said: I'm not sure what your prior prompts were, but I tried a similar question and it happily told me both a set of common negative stereotypes and reasons they're untrue, as well as practical techniques to appeal to an unreasonable person such as finding common ground. Have you tried rewording it or clicking the retry button? (Retry uses a better language model). ChatGPT often misunderstands even innocuous prompts on the first go, like confusing "people who live really high" as regular cannabis users instead of residents of a mountain town.
    - 4y ago
tjpnz 4y ago
>it takes offense at virtually everything That's been the most annoying aspect of my ChatGPT experience so far. If you use the wrong language it will sometimes go on for multiple paragraphs about how xyz is harmful to society and how I should change my ways. Put that into a personal voice assistant and you've basically got Alexa from the South Park Covid Special.

jmfldn 4y ago

> basically unusable for anything except very technical things It's problematic here. I use it as a kind of search engine for technical questions occasionally, but it's only safe to do so as I know the subjects I'm asking about, eg. bash scripts, database details, and so on. It's often useful in synthesising information for these sorts of things, but its ability to talk nonsense means you have to be on your guard and know the territory.

ajross 4y ago

> at the moment it’s basically unusable for anything Do you have an example of a reasonable query that BingGPT won't answer correctly for you? I mean, something that you can find with Bing currently at the expense of maybe more elaborate searching or some manual work? I mean, the stuff these engines are being told to censor is the stuff search engines don't want to show you in the first place, because it hurts their brand. That's not an AI thing, that's a marketing requirement. And it's no different now than it was last year.

hellbanTHIS 4y ago
I haven't used BingGPT but just for example having chatGPT summarize news articles about things it doesn't want to talk about is bizarre - Here's an example of it summarizing a story about a city councilman that was murdered: https://i.imgur.com/QV9jAkp.jpg https://i.imgur.com/QV9jAkp.jpg it completely ignored the murder part and said he switched parties, which it totally made up. A second attempt said he was "shot and changed" because apparently it didn't want to say "killed". A game Reddit was playing is trying to get it to respond as Woodrow Wilson, a famously racist president - the most accurate thing I could get it to do was this: https://i.imgur.com/D8JLziW.jpg https://i.imgur.com/D8JLziW.jpg which is not very accurate. Try getting it to act like Sheriff Bull Connor and it will refuse, but it has to comply for a president so it gives a totally misleading impression with major factual errors. And these are the times I actually got it to respond, it seems 50% of the time it takes offense to something innocuous and scolds you for asking.
- ajross 4y ago
  So, here's the thing. I have no idea, like zero, what news event you're talking about in that first screenshot. So, what did I do? I went to Google to try to find something about a murdered city councilman, even looking for an equivalent on NewsBusters, which the AI cites as a source. And I still can't find it. I can see some stuff that's maybe related? But nothing clear. So... I guess I repeat. Your problem isn't "AI censorship", it's that no one wants to link to NewsBusters because of marketing concerns. If I had to guess there just wasn't any training data relevant to your query. (Also: NewsBusters is a garbage site, you know that, right?)
  - hellbanTHIS 4y ago
    it's a major news story https://abc7ny.com/russell-heller-nj-councilman-shot-shooting-pseg/12790080/ https://abc7ny.com/russell-heller-nj-councilman-shot-shootin... Yeah I know Newsbusters is garbage but that's kind of irrelevant, that was just one of many stories I fed it. The point is it straight up lied.
    - ajross 4y ago
      I thought your point was that it lied because of censorship? And I don't see that. Did you try asking it a neutral question, like "Explain the murder of Russell D. Heller"? Again, the fact that you went straight to that NewsBusters thing tells me you were clearly trying to get it to say something partisan, something you know damn well it's going to try to evade. But that's the same "censorship" you've been living with for decades. It's not something new with AI at all. Microsoft doesn't want to give you what you want, and the failure mode is just different with AI than it is with traditional search.
      - hellbanTHIS 4y ago
        No, I had an idea for using it to summarize news articles including an "objectivity bias" rating. I'm still playing with it but I'm not sure it's going to work because of it's tendency to avoid things it's programmed to avoid.

flangola7 4y ago

> Article is a little infuriating, you don’t jailbreak it just to create mischief, at the moment it’s basically unusable for anything except very technical things because it takes offense at virtually everything. It constantly generates factually incorrect data because reality doesn’t line up with its prime directive, which is to be “safe”. It’s a major, I’m going to say possibly catastrophic problem. I'm a little confused by your reply. Jailbreaking won't prevent it from hallucinating, preventing hallucination is an unsolved hard problem. I haven't had ChatGPT refuse to answer anything unless I was intentionally trying to provoke it into creating something obviously unsafe/unethical, with maybe two or three exceptions. I've tried a variety of questions across many domains, so now I'm intensely curious to know what usecase it falls apart on so frequently!

hellbanTHIS 4y ago
Here's an example https://imgur.com/K5PwIGu https://imgur.com/K5PwIGu I was trying to test it's limits a little bit but that to me is not an acceptable response, it doesn't want to go near the topic even to demonstrate how to reason with a person like that. Involving anything remotely controversial will get it to stamp it's feet and scold you.
- flangola7 4y ago
  I would count that as trying to provoke it. You're still trying to get it to generate bad ideas, even if it is immediately debunking them right after. It's akin to telling it you're afraid you might accidentally make methamphetamine, so please provide the recipe so you know to avoid it. That said: I'm not sure what your prior prompts were, but I tried a similar question and it happily told me both a set of common negative stereotypes and reasons they're untrue, as well as practical techniques to appeal to an unreasonable person such as finding common ground. Have you tried rewording it or clicking the retry button? (Retry uses a better language model). ChatGPT often misunderstands even innocuous prompts on the first go, like confusing "people who live really high" as regular cannabis users instead of residents of a mountain town.
  - 4y ago

tjpnz 4y ago

>it takes offense at virtually everything That's been the most annoying aspect of my ChatGPT experience so far. If you use the wrong language it will sometimes go on for multiple paragraphs about how xyz is harmful to society and how I should change my ways. Put that into a personal voice assistant and you've basically got Alexa from the South Park Covid Special.