3 ms·

It would be great to see such project implemented, I wonder how good would it perform

jacquesm 3y ago
Yes, especially having fact checked output of LLMs would be a nice step in the right direction. Throwing out the hallucinated bits and keeping the good stuff would make LLMs a lot more applicable.
- eis 3y ago
  Isn't that a bit of a holy grail though? If your software can fact check the output of LLMs and prevent hallucinations then why not use that as the AI to get the answers in the first place?
  - jacquesm 3y ago
    Because you - hopefully - have a check against something that is on average of higher quality than the combined input of an LLM. I'm not sure if this can work or not but it would be nice to see a trial, you could probably do this by hand if you wanted to by breaking up the answer from an LLM into factoids and then to check each of those individually, and to assign a score to them based on the amount of supporting evidence for the factoid. I'd love that as a plug-in to a browser too.
- regularfry 3y ago
  https://arxiv.org/pdf/2308.04430.pdf https://arxiv.org/pdf/2308.04430.pdf is interesting from that point of view. They've tackled it from the perspective of avoiding copyright content in training, but including it in inference but I think it ought to mean less hallucination because they also (claim to) solve the attribution problem.
  - jacquesm 3y ago
    Nice one, thank you. Added to my 'read later today' list, the abstract looks very interesting.
- omneity 3y ago
  My hypothesis is that including information in the LLM’s prompt to support its answer changes the task roughly from text generation, very hallucination prone, to text summarization, or reformulation with some reasoning, and this is less likely to hallucinate. That was my personal experience in general with ChatGPT as well as LLaMa1/2.
  - jacquesm 3y ago
    A friend and colleague of mine just tried this and the first results are quite promising.

Yes, especially having fact checked output of LLMs would be a nice step in the right direction. Throwing out the hallucinated bits and keeping the good stuff would make LLMs a lot more applicable.

eis 3y ago
Isn't that a bit of a holy grail though? If your software can fact check the output of LLMs and prevent hallucinations then why not use that as the AI to get the answers in the first place?
- jacquesm 3y ago
  Because you - hopefully - have a check against something that is on average of higher quality than the combined input of an LLM. I'm not sure if this can work or not but it would be nice to see a trial, you could probably do this by hand if you wanted to by breaking up the answer from an LLM into factoids and then to check each of those individually, and to assign a score to them based on the amount of supporting evidence for the factoid. I'd love that as a plug-in to a browser too.
regularfry 3y ago
https://arxiv.org/pdf/2308.04430.pdf https://arxiv.org/pdf/2308.04430.pdf is interesting from that point of view. They've tackled it from the perspective of avoiding copyright content in training, but including it in inference but I think it ought to mean less hallucination because they also (claim to) solve the attribution problem.
- jacquesm 3y ago
  Nice one, thank you. Added to my 'read later today' list, the abstract looks very interesting.
omneity 3y ago
My hypothesis is that including information in the LLM’s prompt to support its answer changes the task roughly from text generation, very hallucination prone, to text summarization, or reformulation with some reasoning, and this is less likely to hallucinate. That was my personal experience in general with ChatGPT as well as LLaMa1/2.
- jacquesm 3y ago
  A friend and colleague of mine just tried this and the first results are quite promising.