4 ms·

How would the model even recognize which tokens represent people? Google is chasing fool's gold if they think they can restrict this. I'd bet a very large sum o

How would the model even recognize which tokens represent people? Google is chasing fool's gold if they think they can restrict this. I'd bet a very large sum of money that these gigantic model authors are intentionally trying to force jurisprudence to cover chatbots under article 230. They better do it now before the market pops!

zamadatix 2y ago
If the model couldn't recognize certain tokens represent people then it wouldn't be generate fake stories about who those people are either. Currently, when it recognizes "who a person is" is part of the task it triggers a search on that person's name. What does Google have to do with this? ChatGPT isn't Google's and it their solution uses Bing for the search portion.
- ForTheKidz 2y ago
  > If the model couldn't recognize certain tokens represent people then it wouldn't be generate fake stories about who those people are either. That is simply not true at all. This is like saying that humans are incapable of (accidentally) lying because the result is incoherent. LLMs are just as capable of incoherency as the rest of us. (...well, maybe not, but they're certainly capable of incoherency.)
  - zamadatix 2y ago
    The problem in the article is LLMs can recognize a request about a person's name but generate a fake story because it doesn't really have information about them, not that the LLM spit out random data which happened to appropriately respond to the question about who the person was with incorrect info each time by pure random chance. Also per the article, when the LLM recognizes a person's name it now performs a search query instead. I'm not saying this makes LLMs infallible, I'm saying this turns the problem in the article into a search query to prevent the defamation problem due to generating fake information about them by replacing it with the externally sourced and cited search information.
    - ForTheKidz 2y ago
      > The problem in the article is LLMs can recognize a request about a person's name but generate a fake story because it doesn't really have information about them, not that the LLM spit out random data which happened to appropriately respond to the question about who the person was with incorrect info each time by pure random chance. I don't see how "arbitrary" is any better—that's certainly how humans behave if forced to provide an answer. While it may appear obvious how we engage our internal skepticism signal, it's obvious this search for contradictions is bounded by both breadth and depth. Such an instinct will need to be inspected and reproduced to provide a "I don't know" answer, if that is what you desire from your chatbot (rather than incoherent synthesis, aka creativity).
      - zamadatix 2y ago
        To be clear, the solution to use "search" in this context is a "web search" and what you're responding to is the description of the prior, broken, behavior that prompted the story and subsequent change in behavior. I.e. ChatGPT now performs a Bing API query to get cached results for "who is ${persons name}". None of this relies on the model now figuring out how uncertain or certain it is, if it sees a query asking about a person it just always performs a search rather than trying to come up with an answer itself. It then also provides the links to the external pages it got the answer from.
        ForTheKidz 2y ago
        yes, I was using search in the other more generic sense (e.g. beam search). The google search thing is really only interesting if they can bind the tokens to the result, otherwise you're just going to have to re-google to vet the chatbot.
        zamadatix 2y ago
        Can I ask why you keep attributing things to Google here when I've continuously clarified they are not involved in either this model or the search results it's using? And yes, this is not like beam search and that's exactly why it works consistently for the defamation prevention use case.
lxgr 2y ago
In the same way that humans do too. We can argue about the reasoning abilities of LLMs all day long, but their pure language faculty (which includes figuring out which words in a sentence probably reference a person, based on context and corpus probabilities) is hopefully generally accepted as being real at this point.

zamadatix 2y ago

If the model couldn't recognize certain tokens represent people then it wouldn't be generate fake stories about who those people are either. Currently, when it recognizes "who a person is" is part of the task it triggers a search on that person's name. What does Google have to do with this? ChatGPT isn't Google's and it their solution uses Bing for the search portion.

ForTheKidz 2y ago
> If the model couldn't recognize certain tokens represent people then it wouldn't be generate fake stories about who those people are either. That is simply not true at all. This is like saying that humans are incapable of (accidentally) lying because the result is incoherent. LLMs are just as capable of incoherency as the rest of us. (...well, maybe not, but they're certainly capable of incoherency.)
- zamadatix 2y ago
  The problem in the article is LLMs can recognize a request about a person's name but generate a fake story because it doesn't really have information about them, not that the LLM spit out random data which happened to appropriately respond to the question about who the person was with incorrect info each time by pure random chance. Also per the article, when the LLM recognizes a person's name it now performs a search query instead. I'm not saying this makes LLMs infallible, I'm saying this turns the problem in the article into a search query to prevent the defamation problem due to generating fake information about them by replacing it with the externally sourced and cited search information.
  - ForTheKidz 2y ago
    > The problem in the article is LLMs can recognize a request about a person's name but generate a fake story because it doesn't really have information about them, not that the LLM spit out random data which happened to appropriately respond to the question about who the person was with incorrect info each time by pure random chance. I don't see how "arbitrary" is any better—that's certainly how humans behave if forced to provide an answer. While it may appear obvious how we engage our internal skepticism signal, it's obvious this search for contradictions is bounded by both breadth and depth. Such an instinct will need to be inspected and reproduced to provide a "I don't know" answer, if that is what you desire from your chatbot (rather than incoherent synthesis, aka creativity).
    - zamadatix 2y ago
      To be clear, the solution to use "search" in this context is a "web search" and what you're responding to is the description of the prior, broken, behavior that prompted the story and subsequent change in behavior. I.e. ChatGPT now performs a Bing API query to get cached results for "who is ${persons name}". None of this relies on the model now figuring out how uncertain or certain it is, if it sees a query asking about a person it just always performs a search rather than trying to come up with an answer itself. It then also provides the links to the external pages it got the answer from.
      - ForTheKidz 2y ago
        yes, I was using search in the other more generic sense (e.g. beam search). The google search thing is really only interesting if they can bind the tokens to the result, otherwise you're just going to have to re-google to vet the chatbot.
        zamadatix 2y ago
        Can I ask why you keep attributing things to Google here when I've continuously clarified they are not involved in either this model or the search results it's using? And yes, this is not like beam search and that's exactly why it works consistently for the defamation prevention use case.

lxgr 2y ago

In the same way that humans do too. We can argue about the reasoning abilities of LLMs all day long, but their pure language faculty (which includes figuring out which words in a sentence probably reference a person, based on context and corpus probabilities) is hopefully generally accepted as being real at this point.