4 ms·

Text might be indirect, but it's crucially still about the world. I could say that being ale to read is one of the senses, and no sense reflects the world in it

Text might be indirect, but it's crucially still about the world. I could say that being ale to read is one of the senses, and no sense reflects the world in itself, but rather is an indirection. But I won't do it. Instead, I'll just replace a few words in your last paragraph to reveal that it doesn't discredit LLMs as much as it seems. * * * But these historians know nothing about the distant past. They might capture some features of the past as patterns that help them better predict where to start an archaeological dig. But that is not the same as knowledge of the past. They have no sensory contact with the past so they are unable to sunthesize true knoledge about it. This is why they are prone to hallucination.

chongli 3y ago
Text might be indirect, but it's crucially still about the world. I could say that being ale to read is one of the senses, and no sense reflects the world in itself, but rather is an indirection. Two things: Justification for belief isn’t a binary state, it’s a spectrum. When we have a small amount of evidence supporting a belief, we have a small justification. When we have a large amount of evidence, we have a large justification. We describe this in terms of our confidence in our beliefs. Without getting too far off into the weeds, this extends into all areas of science. We can’t see, touch, hear, smell, or taste electrons (9V battery tests notwithstanding), but we can gain confidence by replicating experiments with them. Working with a cathode ray tube, for example, we can manipulate the voltage on the plates and directly observe the deflection of the point of light on the screen. We can use the equations from a physics textbook, combined with voltage measurements on a multimeter, to make predictions about the electron beam’s deflection and then test them directly on the device. Secondly (and crucially), the text consumed by LLMs isn’t about the world. It’s text of unknown provenance. Some of it may be about the world, some of it may be complete fiction. The LLM has no basis whatsoever for fact-checking the text. All it can do is perform a statistical test on the commonality of a text and then infer from that. Unfortunately that does not get you any closer to reality. A falsehood repeated many times does not become the truth. As for historians? They have way more than text to go on. They have physical evidence (artifacts), they have radioisotope testing, they have DNA testing. They have geographic locations where texts were found. None of this real world sense data is available to an LLM.
- persnickety 3y ago
  Regarding text and other kinds of sensory data, human senses are of unknown provenance as well. Hallucinations happen, and can be complete fiction. Does that means that humans have no basis whatsoever for fact-checking? Other senses would correspond to other pieces of text, so I answer this as "no". If this was true, humans would have been detached from realty. The other problem with discounting LLMs based on the form of sensory input that's available to them is that there is no qualitative comparison of validity of different senses. Sure, getting more visual input counts as more evidence about the world (bar hallucinations). But can part of it be substituted with text without losing the quantitative benefit? Can all of it be substituted with text? You seem to answer "no", but I think it's an oversimplification. Not that I have a definite answer. The one thing that LLMs cannot currently do is design and perform experiments - or if they can, then they can't integrate the results. I'd look for ways LLMs cannot reach some level of justification for beliefs there. But I would not presume that this level is automatically lower than what humans can reach. I have not compared the size of a human training set to that of an LLM.
  - chongli 3y ago
    Again, justification is not a binary state. When you walk around your house, do you say "I might be hallucinating, so therefore I have 0 confidence that any of this is real"? No, of course not. On the other hand, if you're talking to someone you've never met over the internet and they're telling you the shed in their back yard is painted red, what confidence do you have in that? How do you even know you're talking to a person? LLMs are trained on large amounts of text that's just been scraped off the internet. Some of it may be high quality (if you want to call Wikipedia high quality) but much of it (Reddit) is not. When you ask an LLM chatbot a question, how do you know you're getting the Wikipedia answer and not the Reddit one? You don't. It's all mashed up. Until LLMs are able to reliably provide citations for their answers, they ought to be relegated to the epistemic dustbin. Yet I see high schoolers relying on them all the time. They will ask ChatGPT a question and write down its answer verbatim. No second-guessing, no fact-checking, nothing. Blind trust. It's appalling! My professors have even told me they've had students hand in papers full of citations. Turns out, all the citations were completely fabricated by ChatGPT. One professor even said the paper a student submitted cited HIM, the professor receiving the assignment, for a completely made up paper that never existed. When confronted, the students admitted all of the papers were ChatGPT-generated nonsense. Even worse than the Reddit problem above is the problem of LLM-generated text proliferating throughout the internet. The next generation of LLMs are going to be polluted by that nonsense. Garbage-in, garbage-out. They'll be no more trustworthy than a psychiatric patient rambling to themselves in a padded cell.

Text might be indirect, but it's crucially still about the world. I could say that being ale to read is one of the senses, and no sense reflects the world in itself, but rather is an indirection. But I won't do it. Instead, I'll just replace a few words in your last paragraph to reveal that it doesn't discredit LLMs as much as it seems. * * * But these historians know nothing about the distant past. They might capture some features of the past as patterns that help them better predict where to start an archaeological dig. But that is not the same as knowledge of the past. They have no sensory contact with the past so they are unable to sunthesize true knoledge about it. This is why they are prone to hallucination.

chongli 3y ago

Text might be indirect, but it's crucially still about the world. I could say that being ale to read is one of the senses, and no sense reflects the world in itself, but rather is an indirection. Two things: Justification for belief isn’t a binary state, it’s a spectrum. When we have a small amount of evidence supporting a belief, we have a small justification. When we have a large amount of evidence, we have a large justification. We describe this in terms of our confidence in our beliefs. Without getting too far off into the weeds, this extends into all areas of science. We can’t see, touch, hear, smell, or taste electrons (9V battery tests notwithstanding), but we can gain confidence by replicating experiments with them. Working with a cathode ray tube, for example, we can manipulate the voltage on the plates and directly observe the deflection of the point of light on the screen. We can use the equations from a physics textbook, combined with voltage measurements on a multimeter, to make predictions about the electron beam’s deflection and then test them directly on the device. Secondly (and crucially), the text consumed by LLMs isn’t about the world. It’s text of unknown provenance. Some of it may be about the world, some of it may be complete fiction. The LLM has no basis whatsoever for fact-checking the text. All it can do is perform a statistical test on the commonality of a text and then infer from that. Unfortunately that does not get you any closer to reality. A falsehood repeated many times does not become the truth. As for historians? They have way more than text to go on. They have physical evidence (artifacts), they have radioisotope testing, they have DNA testing. They have geographic locations where texts were found. None of this real world sense data is available to an LLM.

persnickety 3y ago
Regarding text and other kinds of sensory data, human senses are of unknown provenance as well. Hallucinations happen, and can be complete fiction. Does that means that humans have no basis whatsoever for fact-checking? Other senses would correspond to other pieces of text, so I answer this as "no". If this was true, humans would have been detached from realty. The other problem with discounting LLMs based on the form of sensory input that's available to them is that there is no qualitative comparison of validity of different senses. Sure, getting more visual input counts as more evidence about the world (bar hallucinations). But can part of it be substituted with text without losing the quantitative benefit? Can all of it be substituted with text? You seem to answer "no", but I think it's an oversimplification. Not that I have a definite answer. The one thing that LLMs cannot currently do is design and perform experiments - or if they can, then they can't integrate the results. I'd look for ways LLMs cannot reach some level of justification for beliefs there. But I would not presume that this level is automatically lower than what humans can reach. I have not compared the size of a human training set to that of an LLM.
- chongli 3y ago
  Again, justification is not a binary state. When you walk around your house, do you say "I might be hallucinating, so therefore I have 0 confidence that any of this is real"? No, of course not. On the other hand, if you're talking to someone you've never met over the internet and they're telling you the shed in their back yard is painted red, what confidence do you have in that? How do you even know you're talking to a person? LLMs are trained on large amounts of text that's just been scraped off the internet. Some of it may be high quality (if you want to call Wikipedia high quality) but much of it (Reddit) is not. When you ask an LLM chatbot a question, how do you know you're getting the Wikipedia answer and not the Reddit one? You don't. It's all mashed up. Until LLMs are able to reliably provide citations for their answers, they ought to be relegated to the epistemic dustbin. Yet I see high schoolers relying on them all the time. They will ask ChatGPT a question and write down its answer verbatim. No second-guessing, no fact-checking, nothing. Blind trust. It's appalling! My professors have even told me they've had students hand in papers full of citations. Turns out, all the citations were completely fabricated by ChatGPT. One professor even said the paper a student submitted cited HIM, the professor receiving the assignment, for a completely made up paper that never existed. When confronted, the students admitted all of the papers were ChatGPT-generated nonsense. Even worse than the Reddit problem above is the problem of LLM-generated text proliferating throughout the internet. The next generation of LLMs are going to be polluted by that nonsense. Garbage-in, garbage-out. They'll be no more trustworthy than a psychiatric patient rambling to themselves in a padded cell.