3 ms·

Regarding text and other kinds of sensory data, human senses are of unknown provenance as well. Hallucinations happen, and can be complete fiction. Does that me

Regarding text and other kinds of sensory data, human senses are of unknown provenance as well. Hallucinations happen, and can be complete fiction. Does that means that humans have no basis whatsoever for fact-checking? Other senses would correspond to other pieces of text, so I answer this as "no". If this was true, humans would have been detached from realty. The other problem with discounting LLMs based on the form of sensory input that's available to them is that there is no qualitative comparison of validity of different senses. Sure, getting more visual input counts as more evidence about the world (bar hallucinations). But can part of it be substituted with text without losing the quantitative benefit? Can all of it be substituted with text? You seem to answer "no", but I think it's an oversimplification. Not that I have a definite answer. The one thing that LLMs cannot currently do is design and perform experiments - or if they can, then they can't integrate the results. I'd look for ways LLMs cannot reach some level of justification for beliefs there. But I would not presume that this level is automatically lower than what humans can reach. I have not compared the size of a human training set to that of an LLM.

chongli 3y ago
Again, justification is not a binary state. When you walk around your house, do you say "I might be hallucinating, so therefore I have 0 confidence that any of this is real"? No, of course not. On the other hand, if you're talking to someone you've never met over the internet and they're telling you the shed in their back yard is painted red, what confidence do you have in that? How do you even know you're talking to a person? LLMs are trained on large amounts of text that's just been scraped off the internet. Some of it may be high quality (if you want to call Wikipedia high quality) but much of it (Reddit) is not. When you ask an LLM chatbot a question, how do you know you're getting the Wikipedia answer and not the Reddit one? You don't. It's all mashed up. Until LLMs are able to reliably provide citations for their answers, they ought to be relegated to the epistemic dustbin. Yet I see high schoolers relying on them all the time. They will ask ChatGPT a question and write down its answer verbatim. No second-guessing, no fact-checking, nothing. Blind trust. It's appalling! My professors have even told me they've had students hand in papers full of citations. Turns out, all the citations were completely fabricated by ChatGPT. One professor even said the paper a student submitted cited HIM, the professor receiving the assignment, for a completely made up paper that never existed. When confronted, the students admitted all of the papers were ChatGPT-generated nonsense. Even worse than the Reddit problem above is the problem of LLM-generated text proliferating throughout the internet. The next generation of LLMs are going to be polluted by that nonsense. Garbage-in, garbage-out. They'll be no more trustworthy than a psychiatric patient rambling to themselves in a padded cell.
- persnickety 3y ago
  > epistemic dustbin I think this is a good idea if LLMs justification was 0. It's not, and it's not clear what it is. That people talk to a storyteller thinking that they talk to a scholar is on the people, or on whoever misrepresented the interlocutor, not on the interlocutor. It's marvelous already that we can't rule out whatever nonzero amount of justification in the case of LLMs.
  - chongli 3y ago
    rule out whatever nonzero amount of justification We absolutely can rule it out. The justification is zero. The LLM has not and cannot verify any of the factual claims in its text. It can't even tell a claim apart from a non-claim. In legal parlance, the entire training set of an LLM is hearsay. LLMs do not produce any first party information. It's 100% third party.
    - Davidzheng 3y ago
      I don't think this is correct. It could (at least theoretically) be able to verify chains of pure reasoning bounded by a fixed length.
      - chongli 3y ago
        If I ask ChatGPT "is the sky blue?" it replies "Yes, the sky is often seen as blue during the day, particularly on clear days." How does it verify this? It cannot. There is no chain of reasoning by which the LLM can verify this statement. All it can do is defer to others. In other words, the utterances of ChatGPT are entirely hearsay.
    - persnickety 3y ago
      If you ask an LLM "did Alice on the X forum write Y?", you can get a justifiable answer. That's the text that the LLM got directly. Remember that speech is sensory input for an LLM, and it's as direct as your human senses. Asking an LLM about the physical world is not unlike asking a historian about the past. All information they have past a certain point is 3rd party, mediated by tens/hundreds of years. Can historians say anything justifiable about the past? So can LLMs about the physical world. If you want to rule out historians being able to justify their knowledge, I'd like to hear that.

chongli 3y ago

Again, justification is not a binary state. When you walk around your house, do you say "I might be hallucinating, so therefore I have 0 confidence that any of this is real"? No, of course not. On the other hand, if you're talking to someone you've never met over the internet and they're telling you the shed in their back yard is painted red, what confidence do you have in that? How do you even know you're talking to a person? LLMs are trained on large amounts of text that's just been scraped off the internet. Some of it may be high quality (if you want to call Wikipedia high quality) but much of it (Reddit) is not. When you ask an LLM chatbot a question, how do you know you're getting the Wikipedia answer and not the Reddit one? You don't. It's all mashed up. Until LLMs are able to reliably provide citations for their answers, they ought to be relegated to the epistemic dustbin. Yet I see high schoolers relying on them all the time. They will ask ChatGPT a question and write down its answer verbatim. No second-guessing, no fact-checking, nothing. Blind trust. It's appalling! My professors have even told me they've had students hand in papers full of citations. Turns out, all the citations were completely fabricated by ChatGPT. One professor even said the paper a student submitted cited HIM, the professor receiving the assignment, for a completely made up paper that never existed. When confronted, the students admitted all of the papers were ChatGPT-generated nonsense. Even worse than the Reddit problem above is the problem of LLM-generated text proliferating throughout the internet. The next generation of LLMs are going to be polluted by that nonsense. Garbage-in, garbage-out. They'll be no more trustworthy than a psychiatric patient rambling to themselves in a padded cell.

persnickety 3y ago
> epistemic dustbin I think this is a good idea if LLMs justification was 0. It's not, and it's not clear what it is. That people talk to a storyteller thinking that they talk to a scholar is on the people, or on whoever misrepresented the interlocutor, not on the interlocutor. It's marvelous already that we can't rule out whatever nonzero amount of justification in the case of LLMs.
- chongli 3y ago
  rule out whatever nonzero amount of justification We absolutely can rule it out. The justification is zero. The LLM has not and cannot verify any of the factual claims in its text. It can't even tell a claim apart from a non-claim. In legal parlance, the entire training set of an LLM is hearsay. LLMs do not produce any first party information. It's 100% third party.
  - Davidzheng 3y ago
    I don't think this is correct. It could (at least theoretically) be able to verify chains of pure reasoning bounded by a fixed length.
    - chongli 3y ago
      If I ask ChatGPT "is the sky blue?" it replies "Yes, the sky is often seen as blue during the day, particularly on clear days." How does it verify this? It cannot. There is no chain of reasoning by which the LLM can verify this statement. All it can do is defer to others. In other words, the utterances of ChatGPT are entirely hearsay.
  - persnickety 3y ago
    If you ask an LLM "did Alice on the X forum write Y?", you can get a justifiable answer. That's the text that the LLM got directly. Remember that speech is sensory input for an LLM, and it's as direct as your human senses. Asking an LLM about the physical world is not unlike asking a historian about the past. All information they have past a certain point is 3rd party, mediated by tens/hundreds of years. Can historians say anything justifiable about the past? So can LLMs about the physical world. If you want to rule out historians being able to justify their knowledge, I'd like to hear that.