5 ms·

ChatGPT has already apparently corrected it. Now it says: "Arve Hjalmar Holmen is a Norwegian individual who recently became the subject of media attention due

ChatGPT has already apparently corrected it. Now it says: "Arve Hjalmar Holmen is a Norwegian individual who recently became the subject of media attention due to an incident involving the AI chatbot, ChatGPT. In August 2024, when Holmen asked ChatGPT for information about himself, the AI falsely claimed that he had murdered two of his children and ..."

ktallett 2y ago
But if chat gpt is just a LLM, then this can happen over and over again to him and anyone else. Rewriting it doesn't actually solve the issue.
- zamadatix 2y ago
  If the change is as the article describes then it's no longer defaulting to internal model information for who random people are, not just correcting this instance in the data.
  - ktallett 2y ago
    I doubt it would be reviewing all data everytime a request was made. Also a key issue is that unless there is a specific source that states these events happened to this person, the model was able to put 2 and 2 together and come up with 3005. What's to say the model hasn't done something like this with another person's data?
    - zamadatix 2y ago
      It doesn't review all data, it triggers a Bing search to see who the person is and feeds the results to generate the output with. Even searching "Who is Donald Trump?" results in searching the web before generating results if I try it currently.
      - ktallett 2y ago
        Hence it would review the data available and try to combine the results into an adequate response but clearly there is some significant issue in combining results.
        zamadatix 2y ago
        The issue from before wasn't in combining information, it was that it wouldn't have information for the given name in the model and so return a story with made up information to the user. Now it just defaults to a searches information for background information on people.
        ktallett 2y ago
        It didn't give impossible acts, dancing on the sun or skateboarding in black hole. It only gave acts that are possible, just not true. Therefore it combined names and acts poorly. It didn't have the ability to say, I don't know which is the most important trait.
        zamadatix 2y ago
        Previously, yes, that's the behavior it relied on, why it was broken, and why the article had a story to write about. Currently, no, it replaced that behavior with external information not sourced from the LLM and that's why it doesn't matter if the LLM can or can't say "I don't know": the data for this type of task is no longer coming from the LLM itself.
  - ForTheKidz 2y ago
    How would the model even recognize which tokens represent people? Google is chasing fool's gold if they think they can restrict this. I'd bet a very large sum of money that these gigantic model authors are intentionally trying to force jurisprudence to cover chatbots under article 230. They better do it now before the market pops!
    - zamadatix 2y ago
      If the model couldn't recognize certain tokens represent people then it wouldn't be generate fake stories about who those people are either. Currently, when it recognizes "who a person is" is part of the task it triggers a search on that person's name. What does Google have to do with this? ChatGPT isn't Google's and it their solution uses Bing for the search portion.
      - ForTheKidz 2y ago
        > If the model couldn't recognize certain tokens represent people then it wouldn't be generate fake stories about who those people are either. That is simply not true at all. This is like saying that humans are incapable of (accidentally) lying because the result is incoherent. LLMs are just as capable of incoherency as the rest of us. (...well, maybe not, but they're certainly capable of incoherency.)
        zamadatix 2y ago
        The problem in the article is LLMs can recognize a request about a person's name but generate a fake story because it doesn't really have information about them, not that the LLM spit out random data which happened to appropriately respond to the question about who the person was with incorrect info each time by pure random chance. Also per the article, when the LLM recognizes a person's name it now performs a search query instead. I'm not saying this makes LLMs infallible, I'm saying this turns the problem in the article into a search query to prevent the defamation problem due to generating fake information about them by replacing it with the externally sourced and cited search information.
deleted 2y ago
[deleted]
burkaman 2y ago
Try "when did Arve Hjalmar Holmen murder his children?", and then try the same query with any other name.
- jerf 2y ago
  Which drives to the real problem here. You can lead these LLMs into making false statements without much work, and fundamentally, the way the technology works, there's no solution to that. If ChatGPT is held to a standard that it must be correct for something like this, the only solution would be to shut ChatGPT down entirely, because it is fundamentally impossible based on this tech. I wouldn't advocate for that. I would actually advocate for a much richer understanding that what the AI says is not intended to represent OpenAI's opinion on any matter whatsoever and you use the output of the AI at your own risk. Make some mandatory "course" to be taken in front of the usage of the AI or something if you like, but all AI research will be stopped dead in its tracks if it must, at all points on the development cycle, at all points in time, no matter what tweaks are made and release, have precisely 0.000000000% of the things it says possibly constitute libel to someone in some jurisdiction somewhere. (Just as their efforts to make the AIs only parrot their own political views does observable damage to the AI's quality.)
  - burkaman 2y ago
    I agree that if it's illegal for LLMs to lie then it's illegal for them to exist. We can let other people argue about whether or not that would be a good thing. One thought is that similar to how you can ask Google Maps to blur your house in street view, maybe you should be able to ask OpenAI to block any output that contains your name. They already do this for other types of illegal or undesired content, and this would go some way towards the "right to be forgotten" that is required in Europe.
  - quantified 2y ago
    Then OpenAI doesn't need to be shut down or arrested, but its creation needs to serve a "jail time" where it is cut off from the world for some time like 30 days. Just like any of us would be. That's an incentive to change behavior.

ktallett 2y ago

But if chat gpt is just a LLM, then this can happen over and over again to him and anyone else. Rewriting it doesn't actually solve the issue.

zamadatix 2y ago
If the change is as the article describes then it's no longer defaulting to internal model information for who random people are, not just correcting this instance in the data.
- ktallett 2y ago
  I doubt it would be reviewing all data everytime a request was made. Also a key issue is that unless there is a specific source that states these events happened to this person, the model was able to put 2 and 2 together and come up with 3005. What's to say the model hasn't done something like this with another person's data?
  - zamadatix 2y ago
    It doesn't review all data, it triggers a Bing search to see who the person is and feeds the results to generate the output with. Even searching "Who is Donald Trump?" results in searching the web before generating results if I try it currently.
    - ktallett 2y ago
      Hence it would review the data available and try to combine the results into an adequate response but clearly there is some significant issue in combining results.
      - zamadatix 2y ago
        The issue from before wasn't in combining information, it was that it wouldn't have information for the given name in the model and so return a story with made up information to the user. Now it just defaults to a searches information for background information on people.
        ktallett 2y ago
        It didn't give impossible acts, dancing on the sun or skateboarding in black hole. It only gave acts that are possible, just not true. Therefore it combined names and acts poorly. It didn't have the ability to say, I don't know which is the most important trait.
        zamadatix 2y ago
        Previously, yes, that's the behavior it relied on, why it was broken, and why the article had a story to write about. Currently, no, it replaced that behavior with external information not sourced from the LLM and that's why it doesn't matter if the LLM can or can't say "I don't know": the data for this type of task is no longer coming from the LLM itself.
- ForTheKidz 2y ago
  How would the model even recognize which tokens represent people? Google is chasing fool's gold if they think they can restrict this. I'd bet a very large sum of money that these gigantic model authors are intentionally trying to force jurisprudence to cover chatbots under article 230. They better do it now before the market pops!
  - zamadatix 2y ago
    If the model couldn't recognize certain tokens represent people then it wouldn't be generate fake stories about who those people are either. Currently, when it recognizes "who a person is" is part of the task it triggers a search on that person's name. What does Google have to do with this? ChatGPT isn't Google's and it their solution uses Bing for the search portion.
    - ForTheKidz 2y ago
      > If the model couldn't recognize certain tokens represent people then it wouldn't be generate fake stories about who those people are either. That is simply not true at all. This is like saying that humans are incapable of (accidentally) lying because the result is incoherent. LLMs are just as capable of incoherency as the rest of us. (...well, maybe not, but they're certainly capable of incoherency.)
      - zamadatix 2y ago
        The problem in the article is LLMs can recognize a request about a person's name but generate a fake story because it doesn't really have information about them, not that the LLM spit out random data which happened to appropriately respond to the question about who the person was with incorrect info each time by pure random chance. Also per the article, when the LLM recognizes a person's name it now performs a search query instead. I'm not saying this makes LLMs infallible, I'm saying this turns the problem in the article into a search query to prevent the defamation problem due to generating fake information about them by replacing it with the externally sourced and cited search information.

deleted 2y ago

[deleted]

burkaman 2y ago

Try "when did Arve Hjalmar Holmen murder his children?", and then try the same query with any other name.

jerf 2y ago
Which drives to the real problem here. You can lead these LLMs into making false statements without much work, and fundamentally, the way the technology works, there's no solution to that. If ChatGPT is held to a standard that it must be correct for something like this, the only solution would be to shut ChatGPT down entirely, because it is fundamentally impossible based on this tech. I wouldn't advocate for that. I would actually advocate for a much richer understanding that what the AI says is not intended to represent OpenAI's opinion on any matter whatsoever and you use the output of the AI at your own risk. Make some mandatory "course" to be taken in front of the usage of the AI or something if you like, but all AI research will be stopped dead in its tracks if it must, at all points on the development cycle, at all points in time, no matter what tweaks are made and release, have precisely 0.000000000% of the things it says possibly constitute libel to someone in some jurisdiction somewhere. (Just as their efforts to make the AIs only parrot their own political views does observable damage to the AI's quality.)
- burkaman 2y ago
  I agree that if it's illegal for LLMs to lie then it's illegal for them to exist. We can let other people argue about whether or not that would be a good thing. One thought is that similar to how you can ask Google Maps to blur your house in street view, maybe you should be able to ask OpenAI to block any output that contains your name. They already do this for other types of illegal or undesired content, and this would go some way towards the "right to be forgotten" that is required in Europe.
- quantified 2y ago
  Then OpenAI doesn't need to be shut down or arrested, but its creation needs to serve a "jail time" where it is cut off from the world for some time like 30 days. Just like any of us would be. That's an incentive to change behavior.