6 ms·

> OpenAI published a detailed blog post outlining some of the principles used to ensure safety in their models. The post emphasize in areas such as privacy, fac

> OpenAI published a detailed blog post outlining some of the principles used to ensure safety in their models. The post emphasize in areas such as privacy, factual accuracy Am I the only one amused by the phrase “factual accuracy”? How many stories have we read like the one where it tries to ghost light the guy that this year is actually last year. “Oh, your phone must be wrong too, because there is no way I could be wrong.” Though, maybe that is what factually accurate means. It is convinced that it is always factually accurate, even though it is not.

smoldesu 4y ago
> It is convinced that it is always factually accurate, even though it is not. I don't think that's true. ChatGPT (or any LLM) isn't convinced much of anything. It might present something confidently (which is what most people want) but that's a side-effect of it's programming, not an indication of how good it feels on the answer. If you reply to anything ChatGPT says with "No, you're wrong." it will try to write a new, confident and satisfying answer that responds to your assertion. LLMs will always be "wrong" because they have no distinction between fiction and fact. Everything it reads is mapped into language, not concept space or an attitude or a worldview.
- moonchrome 4y ago
  >Everything it reads is mapped into language, not concept space Umm I'm pretty sure it's discovered concepts through compressing text - it seems perfectly capable of generalizing concepts
  - groffee 4y ago
    [dead]
  - musha68k 4y ago
    Would be interesting to read some unbiased science on that. My hunch would be that any concept it might detect still depends on 2nd order patterns acquired through training? The ultimate “book smart” at best, basically. I’m not a ML scientist though, again would be interesting to read an unbiased evaluation of those aspects. Edit: downvoted, not sure why though. Isn’t this a valid question a lot of non-ML folks would ask themselves?
  - smoldesu 4y ago
    Text compression isn't a deterministic process, unfortunately. It's "concept" of compression is clearly derived from token sampling, in the same way it's concept of "math" is based on guessing the number/token that comes next. While I do agree that ChatGPT exhibits pattern-recognizing qualities, that's basically what it was built to do. I'm not arguing against emergent properties, just against emergent intelligence or even the idea of "understanding" in the first place.
  - mdp2021 4y ago
    > it seems perfectly capable of generalizing concepts How would you support that perception?
    - firatsarlar 4y ago
      With hope and living? It is a dream come true for people. An abstract perception of a knowledge, is like sniffing a rose. It feels, yes, I get there. This 40.000 pages book, woow, I'll make time to live it or sniff another daisy?!
      - mdp2021 4y ago
        > It is a dream come true for people For as long as they will be an active part of societies, they'll better live in reality. We need to know what we are dealing with factually, because water is not exchangeable with iron in practice. The perception of a machine as being able «to generalize concepts» may be an illusion, so it requires some support: we want to avoid illusions for the contextual purposes.
        firatsarlar 4y ago
        Let go , make the context further with words , yourself -society- Write books 1000 pages. Force child to read - lucky ones-, make things harder for everyone.
        mdp2021 4y ago
        But you are out of context. A box was created that outputs statements: either it has a «concept space» or just works through constructing plausible language. The poster claims leaning towards the "«concept space»" possibility - but this should not be a poll, and indications of "why" (such idea) are needed not to make the claim intellectually sterile. It is important otherwise the users of the box may cling to an illusion - that is not advisable. Words are there as a commodity to transfer intellectual states, a dynamic we need for practical and social achievements - otherwise, surrounded by indigence and unaware butchers, "«things [will be] harder for everyone»".
- rcxdude 4y ago
  There's been quite a few different iterations of ChatGPT and bing with different behaviours in this regard: it depends somewhat on the base GPT version, the fine-tuning, and the prompt. Bing very famously at one point was extremely passive aggressive when challenged on basically anything. And while there's nothing intrinsic to the structure and training goals of LLMs which directs them towards more structured reasoning, it does seem that in the process of learning to predict language they pick up some degree of it, however flawed it may be.
  - seba_dos1 4y ago
    > Bing very famously at one point was extremely passive aggressive when challenged on basically anything. It still wasn't an indication of how confident it "felt" with its answers. It was just role-playing a more confident and aggressive chat bot than ChatGPT does.
    - int_19h 4y ago
      Outside of the roleplay, there's no "it". The thing that actually interacts with the user is the persona. But for the same reason, it doesn't matter what the underlying LLM "really thinks" - it'll be the persona that'll try to con you, or write broken or malicious code.
rrauenza 4y ago
I’ve been googling trying to figure out what “ghost light” is in this context .. did you get an autocorrect for gas light?
- gleenn 4y ago
  Looks like they meant "gaslight" but I did find it on Urban Dictionary: ghost light Lighting in a video game that has no apparent source for the light to come from. Its like going out on a bright day, but not being able to find the sun in the sky even though the surroundings are brightly lit. Dead Rising on XBOX is a good example. http://ghost-light.urbanup.com/2450357 http://ghost-light.urbanup.com/2450357
  - harrylove 4y ago
    Agree on gaslight as the intended word. Ghost light also has a theatrical origin, still in use today. https://en.m.wikipedia.org/wiki/Ghost_light_(theatre) https://en.m.wikipedia.org/wiki/Ghost_light_(theatre)
im3w1l 4y ago
That was Bing/Sydney. ChatGPT has a very different "personality".
kolinko 4y ago
Well, unless they designed it to have zero confidence in itself, we are bound to have situations like this. When I was trying to troll it, by saying that IPCC just released a report stating that climate change is not real, and that they were completely wrong after all, it properly said that it is not very likely and that I'm probably mistaken. It admitted that it doesn't have internet access, but still refused to believe the outrageous thing I was saying. I can also imagine GPT's super-low confidence leading to errors in other places - e.g. when I mistakenly claim that it's wrong, and it sheepishly takes my claim at a face value. Finally, considering that the whole world is using it, including some people detached from reality, I really prefer it to be overconfident, than to follow someone into some conspiracy hole.
- Guvante 4y ago
  For better or worse in the current age of the internet prose is a good first pass filter for quality. Someone arguing incoherently is seen as less believable. Unfortunately the prose for these chat models doesn't change based on how certain it is of the facts. So you can't tell based on how it is talking whether it is true or not. Certainly people online speak well while lying either intentionally or unintentionally but usually well intentioned people will coach things they aren't as certain about helping to paint a more accurate picture. I haven't taken a deep dive on the latest models but historically most AI haven't worried about "facts" as much as associating speech patterns. It knows how to talk about facts because other people have done so in the past kind of thing. This means you need to patch in arbitrary rules to reintroduce some semblance of truth to the outputs which isn't an easy task. False training is a whole different area IMO. Especially when there is a difference between responding to a particular user and responding to everyone based on new information.
abstractbill 4y ago
The models are a lot of fun to play with, but yeah, every time I've tried to use them for something "serious" they nearly always invent stuff (and are so convincing in how they write about it!). Most recently I've been interested in what's happened with the 4-color theorem since the 1976 computer-assisted proof, and decided to use GPTChat instead of google+wikipedia. GPTChat had me convinced and excited that, apparently the computer-assisted part of the proof has been getting steadily smaller and smaller over the years and decades, and we're getting close to a proof that might not need computer assistance at all. It wrote really convincingly about it! And then I went and looked for the papers it had talked about. They didn't exist, and their authors either didn't exist, or worked in completely unrelated fields.
- riceart 4y ago
  Before the inevitable idiots come in to say hurr durr but have you tried ChatGPT 4… yes I paid for it, and it is just as prone to hallucinations of factual information. It loves to make up new names for peoples initials.
  - deleted 4y ago
    [deleted]
  - vidarh 4y ago
    While it is still prone to hallucinations, with GPT4 I've had it tell me "X is not a real project, but assuming it exists and is designed to do Y [based on the name], I can provide you with a general guide on how to use it." I never had earlier version to do that. So it does appear to be better, though it's certainly still an issue.
  - dumbaccount123 4y ago
    I found the opposite to be true, i mean sure if youre tricking it. Wait for GPT 5-6 in a year or two and see haha.
  - dumbaccount123 4y ago
    Also please refrain from calling other members idiots, next time you wont be warned.
senko 4y ago
I hold a more charitable interpretation. We (the public) have found an important bug in the system, ie. GPT can lie (or "hallucinate"), even if you try to convince it not to lie. The bug is definitely lowering the usefulness of their product, as well as the public option about it. But I'll let the programmer who has never coded a bug cast the first stone. I wouldn't be surprised if they're scrambling internally to minimize the problem (in the product, not in public perception). They have also recently added a note to ChatGPT: "ChatGPT may produce inaccurate information about people, places, or facts" which is an acknowledement that yes, watch out (I compare it to "caution: contents hot" labels). On the topic of dealing with it, I like the stance that simonw recently took: "We need to tell people ChatGPT will lie to them, not debate linguistics" [0]. I don't attach intentions to a machine algorithm (to me, "gaslight" definitely implies an evil intent), and I don't think OpenAI people are evil, stupid, corrupted or something else because they put out a product that has a bug. But since the wide public can't handle nuances, I'd agree it's better to say "chatgpt lies, use it for things where it either doesn't matter or you can verify; don't use it for fact-finding" to get the point across. [0] https://simonwillison.net/2023/Apr/7/chatgpt-lies/ https://simonwillison.net/2023/Apr/7/chatgpt-lies/
- deleted 4y ago
  [deleted]
- vlovich123 4y ago
  It's not a bug. It's an architectural defect / limitation in our understanding of how to build AI. That makes it a strictly harder problem that will take longer. And it's not totally clear to me that you'll get there purely with LLMs. LLMs accomplish a good chunk of what we classify as intelligence for sure. But it's missing the cognition / reasoning skills and the open question is whether you can solve that by just bolting on more techniques into the LLM or you need a totally different kind of model that you can marry to an LLM.
  - vidarh 4y ago
    GPT 4 will admit to not knowing things in many cases where 3.5turbo does not (tested the same prompt), and either will stop there or go off on a "but if it did exist it might go something like this" type continuation. It still hallucinates a lot, but it's not at all clear that this will be all that difficult an issue to solve given the progress.
isoprophlex 4y ago
I find the thing incredibly smart and yet utterly useless at times. I just spent 20 minutes getting the current iteration of ChatGPT to agree with me that a certain sentence is palindromic. Even when you make it print the unaccented characters one by one, spaces excluded, backwards and forwards, it still insists "Élu par cette crapule" isn't palindromic. I understand how tokenization makes this difficult but come on... this doesn't feel like a difficult task for something that supposedly passes the LSATs and whatnot. * French for "Elected by this piece of shit"
- ShannonLimiter 4y ago
  Just ask it to figure it out step-by-step and to remove accents: GPT-4: Figure this out step by step and check your work: Remove the accents and tell me if this a palindrome: Élu par cette crapule GPT-3.5 (Needs to know that removing spaces and lowercasing is okay): Figure this out step by step and check your work: Lowercase it and remove the accents and spaces and tell me if this a palindrome: Élu par cette crapule
  - deleted 4y ago
    [deleted]
- sbierwagen 4y ago
  LLMs using BPEs are generally going to be bad at string reversal tasks, since they can't see individual letters: https://news.ycombinator.com/item?id=35297183 https://news.ycombinator.com/item?id=35297183

smoldesu 4y ago

> It is convinced that it is always factually accurate, even though it is not. I don't think that's true. ChatGPT (or any LLM) isn't convinced much of anything. It might present something confidently (which is what most people want) but that's a side-effect of it's programming, not an indication of how good it feels on the answer. If you reply to anything ChatGPT says with "No, you're wrong." it will try to write a new, confident and satisfying answer that responds to your assertion. LLMs will always be "wrong" because they have no distinction between fiction and fact. Everything it reads is mapped into language, not concept space or an attitude or a worldview.

moonchrome 4y ago
>Everything it reads is mapped into language, not concept space Umm I'm pretty sure it's discovered concepts through compressing text - it seems perfectly capable of generalizing concepts
- groffee 4y ago
  [dead]
- musha68k 4y ago
  Would be interesting to read some unbiased science on that. My hunch would be that any concept it might detect still depends on 2nd order patterns acquired through training? The ultimate “book smart” at best, basically. I’m not a ML scientist though, again would be interesting to read an unbiased evaluation of those aspects. Edit: downvoted, not sure why though. Isn’t this a valid question a lot of non-ML folks would ask themselves?
- smoldesu 4y ago
  Text compression isn't a deterministic process, unfortunately. It's "concept" of compression is clearly derived from token sampling, in the same way it's concept of "math" is based on guessing the number/token that comes next. While I do agree that ChatGPT exhibits pattern-recognizing qualities, that's basically what it was built to do. I'm not arguing against emergent properties, just against emergent intelligence or even the idea of "understanding" in the first place.
- mdp2021 4y ago
  > it seems perfectly capable of generalizing concepts How would you support that perception?
  - firatsarlar 4y ago
    With hope and living? It is a dream come true for people. An abstract perception of a knowledge, is like sniffing a rose. It feels, yes, I get there. This 40.000 pages book, woow, I'll make time to live it or sniff another daisy?!
    - mdp2021 4y ago
      > It is a dream come true for people For as long as they will be an active part of societies, they'll better live in reality. We need to know what we are dealing with factually, because water is not exchangeable with iron in practice. The perception of a machine as being able «to generalize concepts» may be an illusion, so it requires some support: we want to avoid illusions for the contextual purposes.
      - firatsarlar 4y ago
        Let go , make the context further with words , yourself -society- Write books 1000 pages. Force child to read - lucky ones-, make things harder for everyone.
        mdp2021 4y ago
        But you are out of context. A box was created that outputs statements: either it has a «concept space» or just works through constructing plausible language. The poster claims leaning towards the "«concept space»" possibility - but this should not be a poll, and indications of "why" (such idea) are needed not to make the claim intellectually sterile. It is important otherwise the users of the box may cling to an illusion - that is not advisable. Words are there as a commodity to transfer intellectual states, a dynamic we need for practical and social achievements - otherwise, surrounded by indigence and unaware butchers, "«things [will be] harder for everyone»".
rcxdude 4y ago
There's been quite a few different iterations of ChatGPT and bing with different behaviours in this regard: it depends somewhat on the base GPT version, the fine-tuning, and the prompt. Bing very famously at one point was extremely passive aggressive when challenged on basically anything. And while there's nothing intrinsic to the structure and training goals of LLMs which directs them towards more structured reasoning, it does seem that in the process of learning to predict language they pick up some degree of it, however flawed it may be.
- seba_dos1 4y ago
  > Bing very famously at one point was extremely passive aggressive when challenged on basically anything. It still wasn't an indication of how confident it "felt" with its answers. It was just role-playing a more confident and aggressive chat bot than ChatGPT does.
  - int_19h 4y ago
    Outside of the roleplay, there's no "it". The thing that actually interacts with the user is the persona. But for the same reason, it doesn't matter what the underlying LLM "really thinks" - it'll be the persona that'll try to con you, or write broken or malicious code.

rrauenza 4y ago

I’ve been googling trying to figure out what “ghost light” is in this context .. did you get an autocorrect for gas light?

gleenn 4y ago
Looks like they meant "gaslight" but I did find it on Urban Dictionary: ghost light Lighting in a video game that has no apparent source for the light to come from. Its like going out on a bright day, but not being able to find the sun in the sky even though the surroundings are brightly lit. Dead Rising on XBOX is a good example. http://ghost-light.urbanup.com/2450357 http://ghost-light.urbanup.com/2450357
- harrylove 4y ago
  Agree on gaslight as the intended word. Ghost light also has a theatrical origin, still in use today. https://en.m.wikipedia.org/wiki/Ghost_light_(theatre) https://en.m.wikipedia.org/wiki/Ghost_light_(theatre)

im3w1l 4y ago

That was Bing/Sydney. ChatGPT has a very different "personality".

kolinko 4y ago

Well, unless they designed it to have zero confidence in itself, we are bound to have situations like this. When I was trying to troll it, by saying that IPCC just released a report stating that climate change is not real, and that they were completely wrong after all, it properly said that it is not very likely and that I'm probably mistaken. It admitted that it doesn't have internet access, but still refused to believe the outrageous thing I was saying. I can also imagine GPT's super-low confidence leading to errors in other places - e.g. when I mistakenly claim that it's wrong, and it sheepishly takes my claim at a face value. Finally, considering that the whole world is using it, including some people detached from reality, I really prefer it to be overconfident, than to follow someone into some conspiracy hole.

Guvante 4y ago
For better or worse in the current age of the internet prose is a good first pass filter for quality. Someone arguing incoherently is seen as less believable. Unfortunately the prose for these chat models doesn't change based on how certain it is of the facts. So you can't tell based on how it is talking whether it is true or not. Certainly people online speak well while lying either intentionally or unintentionally but usually well intentioned people will coach things they aren't as certain about helping to paint a more accurate picture. I haven't taken a deep dive on the latest models but historically most AI haven't worried about "facts" as much as associating speech patterns. It knows how to talk about facts because other people have done so in the past kind of thing. This means you need to patch in arbitrary rules to reintroduce some semblance of truth to the outputs which isn't an easy task. False training is a whole different area IMO. Especially when there is a difference between responding to a particular user and responding to everyone based on new information.

abstractbill 4y ago

The models are a lot of fun to play with, but yeah, every time I've tried to use them for something "serious" they nearly always invent stuff (and are so convincing in how they write about it!). Most recently I've been interested in what's happened with the 4-color theorem since the 1976 computer-assisted proof, and decided to use GPTChat instead of google+wikipedia. GPTChat had me convinced and excited that, apparently the computer-assisted part of the proof has been getting steadily smaller and smaller over the years and decades, and we're getting close to a proof that might not need computer assistance at all. It wrote really convincingly about it! And then I went and looked for the papers it had talked about. They didn't exist, and their authors either didn't exist, or worked in completely unrelated fields.

riceart 4y ago
Before the inevitable idiots come in to say hurr durr but have you tried ChatGPT 4… yes I paid for it, and it is just as prone to hallucinations of factual information. It loves to make up new names for peoples initials.
- deleted 4y ago
  [deleted]
- vidarh 4y ago
  While it is still prone to hallucinations, with GPT4 I've had it tell me "X is not a real project, but assuming it exists and is designed to do Y [based on the name], I can provide you with a general guide on how to use it." I never had earlier version to do that. So it does appear to be better, though it's certainly still an issue.
- dumbaccount123 4y ago
  I found the opposite to be true, i mean sure if youre tricking it. Wait for GPT 5-6 in a year or two and see haha.
- dumbaccount123 4y ago
  Also please refrain from calling other members idiots, next time you wont be warned.

senko 4y ago

I hold a more charitable interpretation. We (the public) have found an important bug in the system, ie. GPT can lie (or "hallucinate"), even if you try to convince it not to lie. The bug is definitely lowering the usefulness of their product, as well as the public option about it. But I'll let the programmer who has never coded a bug cast the first stone. I wouldn't be surprised if they're scrambling internally to minimize the problem (in the product, not in public perception). They have also recently added a note to ChatGPT: "ChatGPT may produce inaccurate information about people, places, or facts" which is an acknowledement that yes, watch out (I compare it to "caution: contents hot" labels). On the topic of dealing with it, I like the stance that simonw recently took: "We need to tell people ChatGPT will lie to them, not debate linguistics" [0]. I don't attach intentions to a machine algorithm (to me, "gaslight" definitely implies an evil intent), and I don't think OpenAI people are evil, stupid, corrupted or something else because they put out a product that has a bug. But since the wide public can't handle nuances, I'd agree it's better to say "chatgpt lies, use it for things where it either doesn't matter or you can verify; don't use it for fact-finding" to get the point across. [0] https://simonwillison.net/2023/Apr/7/chatgpt-lies/ https://simonwillison.net/2023/Apr/7/chatgpt-lies/

deleted 4y ago
[deleted]
vlovich123 4y ago
It's not a bug. It's an architectural defect / limitation in our understanding of how to build AI. That makes it a strictly harder problem that will take longer. And it's not totally clear to me that you'll get there purely with LLMs. LLMs accomplish a good chunk of what we classify as intelligence for sure. But it's missing the cognition / reasoning skills and the open question is whether you can solve that by just bolting on more techniques into the LLM or you need a totally different kind of model that you can marry to an LLM.
- vidarh 4y ago
  GPT 4 will admit to not knowing things in many cases where 3.5turbo does not (tested the same prompt), and either will stop there or go off on a "but if it did exist it might go something like this" type continuation. It still hallucinates a lot, but it's not at all clear that this will be all that difficult an issue to solve given the progress.

isoprophlex 4y ago

I find the thing incredibly smart and yet utterly useless at times. I just spent 20 minutes getting the current iteration of ChatGPT to agree with me that a certain sentence is palindromic. Even when you make it print the unaccented characters one by one, spaces excluded, backwards and forwards, it still insists "Élu par cette crapule" isn't palindromic. I understand how tokenization makes this difficult but come on... this doesn't feel like a difficult task for something that supposedly passes the LSATs and whatnot. * French for "Elected by this piece of shit"

ShannonLimiter 4y ago
Just ask it to figure it out step-by-step and to remove accents: GPT-4: Figure this out step by step and check your work: Remove the accents and tell me if this a palindrome: Élu par cette crapule GPT-3.5 (Needs to know that removing spaces and lowercasing is okay): Figure this out step by step and check your work: Lowercase it and remove the accents and spaces and tell me if this a palindrome: Élu par cette crapule
- deleted 4y ago
  [deleted]
sbierwagen 4y ago
LLMs using BPEs are generally going to be bad at string reversal tasks, since they can't see individual letters: https://news.ycombinator.com/item?id=35297183 https://news.ycombinator.com/item?id=35297183