3 ms·

> epistemic dustbin I think this is a good idea if LLMs justification was 0. It's not, and it's not clear what it is. That people talk to a storyteller thinkin

> epistemic dustbin I think this is a good idea if LLMs justification was 0. It's not, and it's not clear what it is. That people talk to a storyteller thinking that they talk to a scholar is on the people, or on whoever misrepresented the interlocutor, not on the interlocutor. It's marvelous already that we can't rule out whatever nonzero amount of justification in the case of LLMs.

chongli 3y ago
rule out whatever nonzero amount of justification We absolutely can rule it out. The justification is zero. The LLM has not and cannot verify any of the factual claims in its text. It can't even tell a claim apart from a non-claim. In legal parlance, the entire training set of an LLM is hearsay. LLMs do not produce any first party information. It's 100% third party.
- Davidzheng 3y ago
  I don't think this is correct. It could (at least theoretically) be able to verify chains of pure reasoning bounded by a fixed length.
  - chongli 3y ago
    If I ask ChatGPT "is the sky blue?" it replies "Yes, the sky is often seen as blue during the day, particularly on clear days." How does it verify this? It cannot. There is no chain of reasoning by which the LLM can verify this statement. All it can do is defer to others. In other words, the utterances of ChatGPT are entirely hearsay.
- persnickety 3y ago
  If you ask an LLM "did Alice on the X forum write Y?", you can get a justifiable answer. That's the text that the LLM got directly. Remember that speech is sensory input for an LLM, and it's as direct as your human senses. Asking an LLM about the physical world is not unlike asking a historian about the past. All information they have past a certain point is 3rd party, mediated by tens/hundreds of years. Can historians say anything justifiable about the past? So can LLMs about the physical world. If you want to rule out historians being able to justify their knowledge, I'd like to hear that.
  - chongli 3y ago
    If you ask an LLM "did Alice on the X forum write Y?", you can get a justifiable answer. Sure, but that’s not a very interesting question and search engines tend to be much better at answering it anyway. Historians have access to more than just texts about the past. They have access to physical evidence. Radiometric dating, artifacts, geographic data about where things were found. At the same time, if you ask historians about (say) the Peloponnesian War they will tell you about the account of Thucydides. What they won’t say is that they know about it personally. That’s a mistake a lot of non-scientists make about science. They’ll hear scientists say things like “we know this” or “we know that” and think that they are committing to some privileged knowledge of the world. They aren’t. They’re using the words “we know X” as shorthand for “our best theory about X.” In fact, many scientists identify as scientific anti-realists [1] which means they don’t believe science has any true facts about the nature of reality at all and that the only job of science is to make better predictions about the outcome of experiments. Of course, if you take anti-realism to its logical conclusion then you have to admit that fossils are only evidence that you may find more fossils in the future, and that you have no basis to talk about the existence of dinosaurs. Personally, I think our actual knowledge of things directly outside our personal experience is very limited. But then I want to encourage people not just to believe everything they read. People should be exploring and trying their own science experiments and trying to build things to understand how the world works. [1] https://plato.stanford.edu/entries/scientific-realism/#AntiFoilForScieReal https://plato.stanford.edu/entries/scientific-realism/#AntiF...
    - persnickety 3y ago
      That a question is not interesting does not rule out the justifiability of the answer, which you seemed to assert. > Historians have access to more than just texts about the past. They have access to physical evidence. Radiometric dating, artifacts, geographic data about where things were found. Correct. That's still all indirect, "third-party" evidence. Does that make answers based on it impossible to justify? To make this analogy even closer to LLMs, let's take historian scholars (linguists? undergrads?) who have nothing to study but books. No experiments, no carbon dating, just texts. Is their entire knowledge about the past impossible to justify?

chongli 3y ago

rule out whatever nonzero amount of justification We absolutely can rule it out. The justification is zero. The LLM has not and cannot verify any of the factual claims in its text. It can't even tell a claim apart from a non-claim. In legal parlance, the entire training set of an LLM is hearsay. LLMs do not produce any first party information. It's 100% third party.

Davidzheng 3y ago
I don't think this is correct. It could (at least theoretically) be able to verify chains of pure reasoning bounded by a fixed length.
- chongli 3y ago
  If I ask ChatGPT "is the sky blue?" it replies "Yes, the sky is often seen as blue during the day, particularly on clear days." How does it verify this? It cannot. There is no chain of reasoning by which the LLM can verify this statement. All it can do is defer to others. In other words, the utterances of ChatGPT are entirely hearsay.
persnickety 3y ago
If you ask an LLM "did Alice on the X forum write Y?", you can get a justifiable answer. That's the text that the LLM got directly. Remember that speech is sensory input for an LLM, and it's as direct as your human senses. Asking an LLM about the physical world is not unlike asking a historian about the past. All information they have past a certain point is 3rd party, mediated by tens/hundreds of years. Can historians say anything justifiable about the past? So can LLMs about the physical world. If you want to rule out historians being able to justify their knowledge, I'd like to hear that.
- chongli 3y ago
  If you ask an LLM "did Alice on the X forum write Y?", you can get a justifiable answer. Sure, but that’s not a very interesting question and search engines tend to be much better at answering it anyway. Historians have access to more than just texts about the past. They have access to physical evidence. Radiometric dating, artifacts, geographic data about where things were found. At the same time, if you ask historians about (say) the Peloponnesian War they will tell you about the account of Thucydides. What they won’t say is that they know about it personally. That’s a mistake a lot of non-scientists make about science. They’ll hear scientists say things like “we know this” or “we know that” and think that they are committing to some privileged knowledge of the world. They aren’t. They’re using the words “we know X” as shorthand for “our best theory about X.” In fact, many scientists identify as scientific anti-realists [1] which means they don’t believe science has any true facts about the nature of reality at all and that the only job of science is to make better predictions about the outcome of experiments. Of course, if you take anti-realism to its logical conclusion then you have to admit that fossils are only evidence that you may find more fossils in the future, and that you have no basis to talk about the existence of dinosaurs. Personally, I think our actual knowledge of things directly outside our personal experience is very limited. But then I want to encourage people not just to believe everything they read. People should be exploring and trying their own science experiments and trying to build things to understand how the world works. [1] https://plato.stanford.edu/entries/scientific-realism/#AntiFoilForScieReal https://plato.stanford.edu/entries/scientific-realism/#AntiF...
  - persnickety 3y ago
    That a question is not interesting does not rule out the justifiability of the answer, which you seemed to assert. > Historians have access to more than just texts about the past. They have access to physical evidence. Radiometric dating, artifacts, geographic data about where things were found. Correct. That's still all indirect, "third-party" evidence. Does that make answers based on it impossible to justify? To make this analogy even closer to LLMs, let's take historian scholars (linguists? undergrads?) who have nothing to study but books. No experiments, no carbon dating, just texts. Is their entire knowledge about the past impossible to justify?