4 ms·

> If the model couldn't recognize certain tokens represent people then it wouldn't be generate fake stories about who those people are either. That is simply n

> If the model couldn't recognize certain tokens represent people then it wouldn't be generate fake stories about who those people are either. That is simply not true at all. This is like saying that humans are incapable of (accidentally) lying because the result is incoherent. LLMs are just as capable of incoherency as the rest of us. (...well, maybe not, but they're certainly capable of incoherency.)

zamadatix 2y ago
The problem in the article is LLMs can recognize a request about a person's name but generate a fake story because it doesn't really have information about them, not that the LLM spit out random data which happened to appropriately respond to the question about who the person was with incorrect info each time by pure random chance. Also per the article, when the LLM recognizes a person's name it now performs a search query instead. I'm not saying this makes LLMs infallible, I'm saying this turns the problem in the article into a search query to prevent the defamation problem due to generating fake information about them by replacing it with the externally sourced and cited search information.
- ForTheKidz 2y ago
  > The problem in the article is LLMs can recognize a request about a person's name but generate a fake story because it doesn't really have information about them, not that the LLM spit out random data which happened to appropriately respond to the question about who the person was with incorrect info each time by pure random chance. I don't see how "arbitrary" is any better—that's certainly how humans behave if forced to provide an answer. While it may appear obvious how we engage our internal skepticism signal, it's obvious this search for contradictions is bounded by both breadth and depth. Such an instinct will need to be inspected and reproduced to provide a "I don't know" answer, if that is what you desire from your chatbot (rather than incoherent synthesis, aka creativity).
  - zamadatix 2y ago
    To be clear, the solution to use "search" in this context is a "web search" and what you're responding to is the description of the prior, broken, behavior that prompted the story and subsequent change in behavior. I.e. ChatGPT now performs a Bing API query to get cached results for "who is ${persons name}". None of this relies on the model now figuring out how uncertain or certain it is, if it sees a query asking about a person it just always performs a search rather than trying to come up with an answer itself. It then also provides the links to the external pages it got the answer from.
    - ForTheKidz 2y ago
      yes, I was using search in the other more generic sense (e.g. beam search). The google search thing is really only interesting if they can bind the tokens to the result, otherwise you're just going to have to re-google to vet the chatbot.
      - zamadatix 2y ago
        Can I ask why you keep attributing things to Google here when I've continuously clarified they are not involved in either this model or the search results it's using? And yes, this is not like beam search and that's exactly why it works consistently for the defamation prevention use case.
        ForTheKidz 2y ago
        > exactly why it works consistently for the defamation prevention use case. Right but in this case the claim is clearly incoherent. If any claim with your name on the internet can be assumed to be about you, you can sink basically any company offering text generation. So either governments lean into the inherently incoherent concept of "defamation" or they completely abandon text generation. We're in a really rough place right now where american companies service many regions with incompatible laws. Ideally these states would be served by companies with compatible values. America is both the best and worst thing to ever happen to the internet.

zamadatix 2y ago

The problem in the article is LLMs can recognize a request about a person's name but generate a fake story because it doesn't really have information about them, not that the LLM spit out random data which happened to appropriately respond to the question about who the person was with incorrect info each time by pure random chance. Also per the article, when the LLM recognizes a person's name it now performs a search query instead. I'm not saying this makes LLMs infallible, I'm saying this turns the problem in the article into a search query to prevent the defamation problem due to generating fake information about them by replacing it with the externally sourced and cited search information.

ForTheKidz 2y ago
> The problem in the article is LLMs can recognize a request about a person's name but generate a fake story because it doesn't really have information about them, not that the LLM spit out random data which happened to appropriately respond to the question about who the person was with incorrect info each time by pure random chance. I don't see how "arbitrary" is any better—that's certainly how humans behave if forced to provide an answer. While it may appear obvious how we engage our internal skepticism signal, it's obvious this search for contradictions is bounded by both breadth and depth. Such an instinct will need to be inspected and reproduced to provide a "I don't know" answer, if that is what you desire from your chatbot (rather than incoherent synthesis, aka creativity).
- zamadatix 2y ago
  To be clear, the solution to use "search" in this context is a "web search" and what you're responding to is the description of the prior, broken, behavior that prompted the story and subsequent change in behavior. I.e. ChatGPT now performs a Bing API query to get cached results for "who is ${persons name}". None of this relies on the model now figuring out how uncertain or certain it is, if it sees a query asking about a person it just always performs a search rather than trying to come up with an answer itself. It then also provides the links to the external pages it got the answer from.
  - ForTheKidz 2y ago
    yes, I was using search in the other more generic sense (e.g. beam search). The google search thing is really only interesting if they can bind the tokens to the result, otherwise you're just going to have to re-google to vet the chatbot.
    - zamadatix 2y ago
      Can I ask why you keep attributing things to Google here when I've continuously clarified they are not involved in either this model or the search results it's using? And yes, this is not like beam search and that's exactly why it works consistently for the defamation prevention use case.
      - ForTheKidz 2y ago
        > exactly why it works consistently for the defamation prevention use case. Right but in this case the claim is clearly incoherent. If any claim with your name on the internet can be assumed to be about you, you can sink basically any company offering text generation. So either governments lean into the inherently incoherent concept of "defamation" or they completely abandon text generation. We're in a really rough place right now where american companies service many regions with incompatible laws. Ideally these states would be served by companies with compatible values. America is both the best and worst thing to ever happen to the internet.