4 ms·

That's a game-changer! It seems like factuality issues with ChatGPT might be fixed. We wrote a blog post on how to get started with a custom plugin: https://qdr

That's a game-changer! It seems like factuality issues with ChatGPT might be fixed. We wrote a blog post on how to get started with a custom plugin: https://qdrant.tech/articles/chatgpt-plugin/ https://qdrant.tech/articles/chatgpt-plugin/

snickerbockers 4y ago
>It seems like factuality issues with ChatGPT might be fixed. Is that really possible to fix that just from a plug-in? All it has to do is admit when it doesn't have the answer, and yet it won't even do that. This leads me to think that ChatGPT doesn't even know when it's lying, so i can't imagine how a plug-in will fix that.
- kacperlukawski 4y ago
  The fact that the model does not have to rely on its internal knowledge anymore but can communicate literally with any external system makes me feel it may significantly reduce the hallucination.
  - majormajor 4y ago
    If it was easy to simply verify truth "with any external system" then would we even need a language model? E.g. if you could just ask [THING] for the true answer, or verify an answer trivially with it... just ask it directly! I ran into this issue with some software documentation just this morning - the answer was helpful but completely wrong in some intermediate steps - but short of a plugin that literally controlled or cloned a similar dev environment to mine that it would take over, it wouldn't be able to tell that the intermediate result was different than it claimed.
    - vidarh 4y ago
      ChatGPT is already pretty good at "admitting" it's wrong when it's given the actual facts, so it does seem likely that providing it with a way to e.g. look up trusted sources and ask it to take those sources into consideration might improve things.
      - majormajor 4y ago
        I think that helps with "hallucination" but less so with "factuality" (when re-reading the parent discussions, I see the convo swerved a bit between those two, so I think that'll be an increasingly important distinction in the future). Confirming it's output against a (potentially wrong) source helps the former but not the latter.
    - CuriouslyC 4y ago
      If one api knows one set of facts, and another api knows another, ad infinitum, are you going to tell people they should remember which api knows which set of facts and query each individually? Why not have a single service that knows of all the various apis for different things, and can query and synthesize answers that extract the relevant information from all of them (with compare/contrast/etc)?
      - kacperlukawski 4y ago
        When you develop a plugin, you provide a description that ChatGPT uses to know when to call that particular service. So you don't need to tell people what they need to use - the model will decide independently based on the plugins you enabled. That being said - we developed a custom plugin for Qdrant docs, so our users will be able to ask questions about how to do certain things with our database. But I do not believe it should be enabled by default for everybody. A non-technical person doesn't need that many details. The same is for the other services - if you prefer using KAYAK over Expedia, you're free to choose.
        kacperlukawski 4y ago
        Yeah, you need to enable the plugins you want. I'm just saying you can enable all the ones that make sense for you, and you don't have to switch between them.
        majormajor 4y ago
        From the videos I thought it was the plugins the user enabled? That's what your second paragraph sounds like too, but your first seems to suggest it being more automatic, user-doesn't-need-to-worry-about-it?
    - bluecrab 4y ago
      All it needs is guardrails which is available already.
  - snickerbockers 4y ago
    That's only going to solve the problem of incorrect facts. I have seen it make logical mistakes as well and having access to external services will not solve that problem. As an example, I once asked it to show me the diff between two revisions of the code it was writing an it made something that looks like it might be a valid patch but did not represent the difference between the two versions. Of course this specific problem could be fixed with a simple plug-in that runs the unix diff program but that wouldn't fix the root-cause, and i would argue that providing a special-case for every type of request is antithetical to what AI is supposed to be since this effectively is how alpha and Google already work.
- kenjackson 4y ago
  A plug-in can detect when text comes up that is in a specific domain and whether or not ChatGPT believes it is hallucinating, the plugin can be invoked to provide additional context to ChatGPT. That is, in order to fix the problem, ChatGPT doesn't even need to know that it has a problem.
- benlivengood 4y ago
  The key piece will be when it queries multiple services by default and compares the answers to its own inferences, and is prompted to trust majority opinion or report that there isn't consensus. The iterative question about moons larger than Mercury in the Wolfram Alpha thread is a simple example of iterative tool use.
- letmevoteplease 4y ago
  "Interestingly, the base pre-trained [GPT-4] model is highly calibrated (its predicted confidence in an answer generally matches the probability of being correct). However, through our current post-training process, the calibration is reduced."[1] The graph is striking.[2] [1] https://openai.com/research/gpt-4 https://openai.com/research/gpt-4 [2] https://i.imgur.com/cxPgkhD.jpg https://i.imgur.com/cxPgkhD.jpg
  - furyofantares 4y ago
    They should make the aligned one generate the text and the accurate one detect if it's lying, override it, and tell the user that it doesn't know.
LouisSayers 4y ago
You'll soon be able to choose your own facts with the "left" and "right" plugins. Choose your own adventure.

snickerbockers 4y ago

>It seems like factuality issues with ChatGPT might be fixed. Is that really possible to fix that just from a plug-in? All it has to do is admit when it doesn't have the answer, and yet it won't even do that. This leads me to think that ChatGPT doesn't even know when it's lying, so i can't imagine how a plug-in will fix that.

kacperlukawski 4y ago
The fact that the model does not have to rely on its internal knowledge anymore but can communicate literally with any external system makes me feel it may significantly reduce the hallucination.
- majormajor 4y ago
  If it was easy to simply verify truth "with any external system" then would we even need a language model? E.g. if you could just ask [THING] for the true answer, or verify an answer trivially with it... just ask it directly! I ran into this issue with some software documentation just this morning - the answer was helpful but completely wrong in some intermediate steps - but short of a plugin that literally controlled or cloned a similar dev environment to mine that it would take over, it wouldn't be able to tell that the intermediate result was different than it claimed.
  - vidarh 4y ago
    ChatGPT is already pretty good at "admitting" it's wrong when it's given the actual facts, so it does seem likely that providing it with a way to e.g. look up trusted sources and ask it to take those sources into consideration might improve things.
    - majormajor 4y ago
      I think that helps with "hallucination" but less so with "factuality" (when re-reading the parent discussions, I see the convo swerved a bit between those two, so I think that'll be an increasingly important distinction in the future). Confirming it's output against a (potentially wrong) source helps the former but not the latter.
  - CuriouslyC 4y ago
    If one api knows one set of facts, and another api knows another, ad infinitum, are you going to tell people they should remember which api knows which set of facts and query each individually? Why not have a single service that knows of all the various apis for different things, and can query and synthesize answers that extract the relevant information from all of them (with compare/contrast/etc)?
    - kacperlukawski 4y ago
      When you develop a plugin, you provide a description that ChatGPT uses to know when to call that particular service. So you don't need to tell people what they need to use - the model will decide independently based on the plugins you enabled. That being said - we developed a custom plugin for Qdrant docs, so our users will be able to ask questions about how to do certain things with our database. But I do not believe it should be enabled by default for everybody. A non-technical person doesn't need that many details. The same is for the other services - if you prefer using KAYAK over Expedia, you're free to choose.
      - kacperlukawski 4y ago
        Yeah, you need to enable the plugins you want. I'm just saying you can enable all the ones that make sense for you, and you don't have to switch between them.
      - majormajor 4y ago
        From the videos I thought it was the plugins the user enabled? That's what your second paragraph sounds like too, but your first seems to suggest it being more automatic, user-doesn't-need-to-worry-about-it?
  - bluecrab 4y ago
    All it needs is guardrails which is available already.
- snickerbockers 4y ago
  That's only going to solve the problem of incorrect facts. I have seen it make logical mistakes as well and having access to external services will not solve that problem. As an example, I once asked it to show me the diff between two revisions of the code it was writing an it made something that looks like it might be a valid patch but did not represent the difference between the two versions. Of course this specific problem could be fixed with a simple plug-in that runs the unix diff program but that wouldn't fix the root-cause, and i would argue that providing a special-case for every type of request is antithetical to what AI is supposed to be since this effectively is how alpha and Google already work.
kenjackson 4y ago
A plug-in can detect when text comes up that is in a specific domain and whether or not ChatGPT believes it is hallucinating, the plugin can be invoked to provide additional context to ChatGPT. That is, in order to fix the problem, ChatGPT doesn't even need to know that it has a problem.
benlivengood 4y ago
The key piece will be when it queries multiple services by default and compares the answers to its own inferences, and is prompted to trust majority opinion or report that there isn't consensus. The iterative question about moons larger than Mercury in the Wolfram Alpha thread is a simple example of iterative tool use.
letmevoteplease 4y ago
"Interestingly, the base pre-trained [GPT-4] model is highly calibrated (its predicted confidence in an answer generally matches the probability of being correct). However, through our current post-training process, the calibration is reduced."[1] The graph is striking.[2] [1] https://openai.com/research/gpt-4 https://openai.com/research/gpt-4 [2] https://i.imgur.com/cxPgkhD.jpg https://i.imgur.com/cxPgkhD.jpg
- furyofantares 4y ago
  They should make the aligned one generate the text and the accurate one detect if it's lying, override it, and tell the user that it doesn't know.

LouisSayers 4y ago

You'll soon be able to choose your own facts with the "left" and "right" plugins. Choose your own adventure.