6 ms·

I'm questioning your binary. As far as I can tell, you're saying that LLMs cannot make justified statements because all evidence they have access to is indirect

I'm questioning your binary. As far as I can tell, you're saying that LLMs cannot make justified statements because all evidence they have access to is indirect text. Therefore, you're stating that no amount of text can justify a belief. Am I misrepresenting your position? But when I present an analogous situation, except with historians playing the main role, you seem to accept the justifiability of their statements on a spectrum, rather than rejecting it based on the evidence being textual and indirect. This is the crux of the discussion, without explaining which we won't come to a mutual understanding (much less acceptance).

chongli 3y ago
Historians who don’t have access to anything other than text (which is your hypothetical situation) cannot justify any claim about real world events in the past. However, this does not happen in real life. Text does not fall from the heavens into the minds of historians. It comes in books, manuscripts, stone tablets, etc. Unlike digital text (which is the only thing LLMs have access to), physical texts have physical properties which can be examined experimentally. They can be subject to radiometric dating, they typically are found in a particular place (which is usually recorded), and they have version and copyright information which enables them to be traced back to the original. In the case where the original manuscript is lost (such as Thucydides’ history of the Peloponnesian War), we have to rely on copied manuscripts by other scribes, often made centuries later. All of this is to say that it can be very difficult to trace a book back to its original author, especially without the original manuscript, but the evidence historians use to piece all this together is generally not in the text itself. It’s physical evidence. Exactly the thing that LLMs have no access to.
- persnickety 3y ago
  Thanks for the clear answer. If I understand you correctly, text alone cannot provide any justifiability, but text in addition to other sensory evidence (location, seeing the radiometric device's results, etc.) can provide justified knowledge. Let me protest against this broader point. If I'm a historian, do I have to find the physical text in the original place myself, or can this information be supplied to me in the form of text? Do I have to actually see the radiocarbon dating machine work myself, or is it enough that the operator sends me an email with the results? What you describe doesn't really strike me as evidence that cannot be supplied through text - you mentioned that it's recorded yourself. It's still text about the past, maybe even with an extra level of indirection (it describes the ~present which has some information about the past). It follows that some of LLMs statements about historical artifacts should be justifiable - those coming from the written word that our hypothetical historian would study. Is that not correct?

chongli 3y ago

Historians who don’t have access to anything other than text (which is your hypothetical situation) cannot justify any claim about real world events in the past. However, this does not happen in real life. Text does not fall from the heavens into the minds of historians. It comes in books, manuscripts, stone tablets, etc. Unlike digital text (which is the only thing LLMs have access to), physical texts have physical properties which can be examined experimentally. They can be subject to radiometric dating, they typically are found in a particular place (which is usually recorded), and they have version and copyright information which enables them to be traced back to the original. In the case where the original manuscript is lost (such as Thucydides’ history of the Peloponnesian War), we have to rely on copied manuscripts by other scribes, often made centuries later. All of this is to say that it can be very difficult to trace a book back to its original author, especially without the original manuscript, but the evidence historians use to piece all this together is generally not in the text itself. It’s physical evidence. Exactly the thing that LLMs have no access to.

persnickety 3y ago
Thanks for the clear answer. If I understand you correctly, text alone cannot provide any justifiability, but text in addition to other sensory evidence (location, seeing the radiometric device's results, etc.) can provide justified knowledge. Let me protest against this broader point. If I'm a historian, do I have to find the physical text in the original place myself, or can this information be supplied to me in the form of text? Do I have to actually see the radiocarbon dating machine work myself, or is it enough that the operator sends me an email with the results? What you describe doesn't really strike me as evidence that cannot be supplied through text - you mentioned that it's recorded yourself. It's still text about the past, maybe even with an extra level of indirection (it describes the ~present which has some information about the past). It follows that some of LLMs statements about historical artifacts should be justifiable - those coming from the written word that our hypothetical historian would study. Is that not correct?