4 ms·

So, yes, now that we've acknowledged that GPT is very good at making things up, for the umpteenth time in the last month, I'm curious: what are the current idea

So, yes, now that we've acknowledged that GPT is very good at making things up, for the umpteenth time in the last month, I'm curious: what are the current ideas in ML to improve on this? I assume the ideas of database access and such are in there, but what methods can be used to make the AI recognize when it is stating a fact that should be checked, and finding ways to check it?

mistermann 4y ago
I'm rather skeptical that a highly reliable methodology for epistemology even exists at all (outside of things like math and the physical sciences), let alone one that can be implemented by a computer.
SQueeeeeL 4y ago
The thing is most facts will never be "checkable" because the world we exist in is simply too complicated to be fully enumerated. You basically require a simulated mind to even begin to grasp the nuances of fact vs perspective vs opinion. It's far beyond the scope of a few companies putting a few decades and a couple upstart tech people to the problem to make any real progress on understand and producing truth, which is why ChatGPT answers every question like a middle schooler, because it's just simulating understanding.
- mistermann 4y ago
  > It's far beyond the scope of a few companies putting a few decades vand a couple upstart tech people to the problem to make any real progress on understand and producing truth The accuracy of this fact is itself is subject to the very issue you are noting though! :)
- dragonwriter 4y ago
  > You basically require a simulated mind to even begin to grasp the nuances of fact vs perspective vs opinion. I’ve done a little testing of this (using HN comments!) and ChatGPT seems to be reasonably good at distinguishing fact vs. opinion. It seems in that limited testing to be slightly more likely to view as opinion things I would view as likely fact claims but where the operationalization intended is ambiguous as opinion, but the boundary between a fact claim with an unstated operationalization and an opinion claim is fuzzy. That doesn’t get a lot closer to checking fact claims, or identifying and deciding when and how to report supporting evidence of them, but I don’t think that distinguishing fact claims is, itsef, as far beyond reach of current technology as you are describing it.
deleted 4y ago
[deleted]
oceanplexian 4y ago
I think all the parts already exist to solve this, you need the AI to check its work rather than simply throwing out a response. So on the front end you might ask it "What is the capital of Italy", but on the back you translate that into "What is the capital of Italy, how could you factually validate that, and give me some example searches you might type into Google to check your work." Then run the searches and return them: "Here are the searches from credible databases, they say X, Y, and Z, does that conflict with your assumption?". Eventually you feed it enough data and provide enough context to the conversation and the AI will naturally head in the right direction. Computers are fast, we should be able to do all this in a matter of milliseconds.
- BaseballPhysics 4y ago
  > I think all the parts already exist to solve this, you need the AI to check its work rather than simply throwing out a response. That would require the AI to have a comprehension of facts, and the entire point is these models don't. So no, I see no evidence that the parts exist to solve this. If anything, the fact the problem isn't already solved suggests to me there is no easy answer.
NovemberWhiskey 4y ago
The way we fix this in most of the rest of the world is "providing citations": we should always be a bit suspicious of unsupported assertions about facts, no? [0] It won't help for those cases where the facts are right, but the reasoning around them is wrong, I suppose; but critical reasoning is a generally essential skill. [0] https://www.pcc.edu/library/research/reasons-for-citing-sources/ https://www.pcc.edu/library/research/reasons-for-citing-sour...
- dragonwriter 4y ago
  > The way we fix this in most of the rest of the world is “providing citations” Statistical generation of text doesn’t work in a way which makes “providing citations” a straightforward thing; It’s not reading a bunch of text, making notecards on ideas, and then using the notecards to build a written response where ideas directly come from sources that are quoted or paraphrased. “Providing citations” can be a goal for the output, but its not much in the way of an an answer to “how”. And you can ask some of the current tools to provide citations, and when they do it at all, they can hallucinate citations just as well as anything else. (This has been seen in many of the attempts to get ChatGPT, etc., to produce legal arguments.) Now, checking citations, to some degree of utility, is probably tractable if the cited material is available, as AI’s are good at summarizing text, so comparing summary of a passage with a citation against the summary of the cited material should be possible, but I don’t think any AI system yet has found a way to provide good enough citations (even with some occasional hallucinations) that citation checking to raise flags on the bad ones would be enough to get us to useful results. The challenge is how to both recognize when citations are appropriate and generate good ones a high percentage of the time.
  - NovemberWhiskey 4y ago
    I suppose I'm indifferent to whether it's hard or not; it doesn't matter to me as a consumer. If your LLM can't provide the sources for the facts it's asserting, then maybe we shouldn't trust it in matters that depend on facts? If your LLM can't tell the difference between factual statements, postulation, and argument, then why would I trust for anything at all? It's a great party trick though. FWIW, I prompted ChatGPT with "Describe how tail recursion works in programming, and why it is important. Include citations." and it actually included two relevant links in its sources, but did not indicates which part of the text depended on them.
scarmig 4y ago
See, for example, "RARR: Researching and Revising What Language Models Say, Using Language Models" (https://arxiv.org/pdf/2210.08726.pdf https://arxiv.org/pdf/2210.08726.pdf). Essentially, break a document into a series of claims, query some kind of document store for each claim to fetch evidences, and then evaluate the extent to which each evidence supports or contradicts the original claim.
noloblo 4y ago
prolog based expert systems

mistermann 4y ago

I'm rather skeptical that a highly reliable methodology for epistemology even exists at all (outside of things like math and the physical sciences), let alone one that can be implemented by a computer.

SQueeeeeL 4y ago

The thing is most facts will never be "checkable" because the world we exist in is simply too complicated to be fully enumerated. You basically require a simulated mind to even begin to grasp the nuances of fact vs perspective vs opinion. It's far beyond the scope of a few companies putting a few decades and a couple upstart tech people to the problem to make any real progress on understand and producing truth, which is why ChatGPT answers every question like a middle schooler, because it's just simulating understanding.

mistermann 4y ago
> It's far beyond the scope of a few companies putting a few decades vand a couple upstart tech people to the problem to make any real progress on understand and producing truth The accuracy of this fact is itself is subject to the very issue you are noting though! :)
dragonwriter 4y ago
> You basically require a simulated mind to even begin to grasp the nuances of fact vs perspective vs opinion. I’ve done a little testing of this (using HN comments!) and ChatGPT seems to be reasonably good at distinguishing fact vs. opinion. It seems in that limited testing to be slightly more likely to view as opinion things I would view as likely fact claims but where the operationalization intended is ambiguous as opinion, but the boundary between a fact claim with an unstated operationalization and an opinion claim is fuzzy. That doesn’t get a lot closer to checking fact claims, or identifying and deciding when and how to report supporting evidence of them, but I don’t think that distinguishing fact claims is, itsef, as far beyond reach of current technology as you are describing it.

deleted 4y ago

[deleted]

oceanplexian 4y ago

I think all the parts already exist to solve this, you need the AI to check its work rather than simply throwing out a response. So on the front end you might ask it "What is the capital of Italy", but on the back you translate that into "What is the capital of Italy, how could you factually validate that, and give me some example searches you might type into Google to check your work." Then run the searches and return them: "Here are the searches from credible databases, they say X, Y, and Z, does that conflict with your assumption?". Eventually you feed it enough data and provide enough context to the conversation and the AI will naturally head in the right direction. Computers are fast, we should be able to do all this in a matter of milliseconds.

BaseballPhysics 4y ago
> I think all the parts already exist to solve this, you need the AI to check its work rather than simply throwing out a response. That would require the AI to have a comprehension of facts, and the entire point is these models don't. So no, I see no evidence that the parts exist to solve this. If anything, the fact the problem isn't already solved suggests to me there is no easy answer.

NovemberWhiskey 4y ago

The way we fix this in most of the rest of the world is "providing citations": we should always be a bit suspicious of unsupported assertions about facts, no? [0] It won't help for those cases where the facts are right, but the reasoning around them is wrong, I suppose; but critical reasoning is a generally essential skill. [0] https://www.pcc.edu/library/research/reasons-for-citing-sources/ https://www.pcc.edu/library/research/reasons-for-citing-sour...

dragonwriter 4y ago
> The way we fix this in most of the rest of the world is “providing citations” Statistical generation of text doesn’t work in a way which makes “providing citations” a straightforward thing; It’s not reading a bunch of text, making notecards on ideas, and then using the notecards to build a written response where ideas directly come from sources that are quoted or paraphrased. “Providing citations” can be a goal for the output, but its not much in the way of an an answer to “how”. And you can ask some of the current tools to provide citations, and when they do it at all, they can hallucinate citations just as well as anything else. (This has been seen in many of the attempts to get ChatGPT, etc., to produce legal arguments.) Now, checking citations, to some degree of utility, is probably tractable if the cited material is available, as AI’s are good at summarizing text, so comparing summary of a passage with a citation against the summary of the cited material should be possible, but I don’t think any AI system yet has found a way to provide good enough citations (even with some occasional hallucinations) that citation checking to raise flags on the bad ones would be enough to get us to useful results. The challenge is how to both recognize when citations are appropriate and generate good ones a high percentage of the time.
- NovemberWhiskey 4y ago
  I suppose I'm indifferent to whether it's hard or not; it doesn't matter to me as a consumer. If your LLM can't provide the sources for the facts it's asserting, then maybe we shouldn't trust it in matters that depend on facts? If your LLM can't tell the difference between factual statements, postulation, and argument, then why would I trust for anything at all? It's a great party trick though. FWIW, I prompted ChatGPT with "Describe how tail recursion works in programming, and why it is important. Include citations." and it actually included two relevant links in its sources, but did not indicates which part of the text depended on them.

scarmig 4y ago

See, for example, "RARR: Researching and Revising What Language Models Say, Using Language Models" (https://arxiv.org/pdf/2210.08726.pdf https://arxiv.org/pdf/2210.08726.pdf). Essentially, break a document into a series of claims, query some kind of document store for each claim to fetch evidences, and then evaluate the extent to which each evidence supports or contradicts the original claim.

noloblo 4y ago

prolog based expert systems