6 ms·

What does a model of truth imply then? The ability to perfectly classify a statement between "truth" and "lie"? If so, then it's not needed for thinking, becaus

What does a model of truth imply then? The ability to perfectly classify a statement between "truth" and "lie"? If so, then it's not needed for thinking, because humans can't do that, either. The only definition that makes sense to me is "some way to classify statements into lies and truth based on inputs", and that fits into what the parent came up with.

Jensson 4y ago
Consider the difference between spellchecking an essay or fact checking an essay. The two are very different actions, you can ask a human to spellcheck an essay and he will mark spelling errors, or you can ask him to fact check it and he will mark out statements he thinks are wrong. So humans know the difference. So a spellchecker isn't a fact checker, you don't call the automated spellcheckers we have "truth models" or "truth checkers", they just check the spelling. ChatGPT is the same as that but more advanced, it just checks that the text looks nice, it doesn't care about the facts in the text. Humans can do that too, if you give a novel to a human and ask him to read it with comments, he wont go and ask for proof that the novel has happened or that everything you wrote is possible or factful, he will just check that the story makes sense. Edit: So we know that humans can tell the difference between these kinds of actions. We might not be able to define a strict rule for them, but since human can tell the difference a general intelligence should also be able to have a truth model, a grammar model, a story model etc. ChatGPT has just one way to evaluate texts, it doesn't have the many ways human has that lets them evaluate the truth, the grammar or the story separately.
- rhn_mk1 4y ago
  The difference between spell checking and fact checking is in what you're checking against: the facts of grammar or the facts of the world. I don't see any qualitative difference between them. The data is facts, the output is either "correct" or "wrong", and the function from one to the other is the model. What do we gain from calling one of them "truth function" and the other "grammar function"? What makes that relevant to reasoning?
  - Jensson 4y ago
    > I don't see any qualitative difference between them. You don't see any qualitative difference between fact checking or spell checking a text? Well then you are a lost cause, we can't discuss this any longer. Basically every human understands that those two are qualitatively different, if you don't then you are so alien to most human thinking that we can't discuss this further, there will be no common ground.
    - rhn_mk1 4y ago
      Well, it's a shame, because I thought you were really on to something. No, I don't see a qualitative difference. There is a huge quantitative difference because the facts that describe the world overwhelm the facts that make up language, but that doesn't obviously make a connection to reasoning clear.
      - Jensson 4y ago
        Extreme quantitative differences results in qualitative difference. Anyway, just realized that you side-tracked the whole conversation by moving it into comparing them to each other. That discussion isn't relevant here. We are talking about a truth model directly, not whether it resembles a truth model in some ways. A spellchecker and a truth checker are both evaluators, but that doesn't mean that a spellchecker is a truth checker or vice versa. Therefore it is wrong to say that ChatGPT has a truth model, since it doesn't evaluate what we call truth, it evaluates the structure of the text not the truth of the text. Evaluators returns a boolean or some number, but that "true" or "false" are just programmatic terms, that isn't want humans normally consider trueness of falseness. Otherwise you would call any evaluator a "truth checker", since they check if the result is a true boolean. You could do that, but that doesn't make sense for this discussion, since we care about truth and false in the world, not in programmatic terms. Edit: As for why this is relevant for ChatGPT smartness. It is because we know how to evaluate text structure now, that is the new thing with ChatGPT. But we still don't know how to evaluate the truth of a text, ChatGPT can evaluate the structure but we have no way of turning a structure evaluator into a truth evaluator. We need another kind of model to start evaluating the truth of statements, and until then AI's like ChatGPT will continue to make up facts, since they have no idea what a fact is.
- coldtea 4y ago
  >ChatGPT is the same as that but more advanced, it just checks that the text looks nice, it doesn't care about the facts in the text Humans don't care about the "facts in the text" either, if by "facts" we mean "real world truth". They just match what's in the text against their _knowledge_ of which things are facts, which is similar to what ChatGPT does to produce text based of its own knowledge of facts (its training set). If a person was given a text saying: "God exists and his son has performed miracles in Galilea" They'd fact-check it as true if their input (education, culture) contains that faith and they are a believer of it, regardless of whether that's an "objectively" true statement or not.
  - Jensson 4y ago
    There are errors in humans truth models, yes, but at least they have one. Humans recognize that checking the structure of the text and checking the facts in the text are two different tasks, so we can train humans to check facts better without worsening their ability to structure text and vice versa, that is the important bit. For ChatGPT those two are the same thing, if you give more variance you will get more unstructured text which will result in more falsehoods, you can't control its truthiness without altering other behaviours.
    - rhn_mk1 4y ago
      You're conflating what ChatGPT does with fact checking. It's as much a fact checker as a human is a fact checker. It's not their purpose to be either a grammar checker or a fact checker, so you can't make assumptions about the models they have just yet. You can instruct a human to do fact checking, just like you can instruct ChatGPT to do it. You can instruct ChatGPT to structure text. Have you compared its output in both cases? Does it affect both parameters the same (from what I've seen, I don't expect that)?
    - coldtea 4y ago
      >There are errors in humans truth models, yes, but at least they have one. Humans recognize that checking the structure of the text and checking the facts in the text are two different tasks Asserting that it's "two different tasks" doesn't necessarily make it true. Humans checking the "structure of the text" stop their check at the text-structure (syntax) layer, whereas humans checking "the facts of the text" also check the validity of what's in the text. That doesn't mean that the operation they internally perform to do the first task isn't the same as the one for the second task (just with a different goal, so e.g. stopping earlier). GPT's "checking the structure of the text" happens through the use of weighted connections created after being fed pieces of information. Human's "checking the structure of the text" also happens through the use of weighted connections created after being fed pieces of information (as encoded in its neurons). Well, human's "checking of facts" ALSO happens through the use of weighted connections created after being fed pieces of information. So, why couldn't ChatGPT "checking of facts" exist and work the same way? At which point is there a necessity for some additional special faculty (and not just e.g. a matter of end-goal setting to calibrate the use of the same capacity of using "weighted connections created after being fed pieces of information")? We do know that we can ask ChatGPT to check a fact, and it will come with a conclusion about it. At which point do we need to sprinkle the magical "fact-checking" dust for this to be fact-checking?
- throw310822 4y ago
  > ChatGPT is the same as that but more advanced, it just checks that the text looks nice, it doesn't care about the facts in the text. Actually, it is pretty easy to ask ChatGPT to do some (simple) fact-checking. Since an answer with the correct fact checking and one with an incorrect fact-checking look equally "nice" from the point of view of the language, I think your assumption is wrong. Prompt: "Can you please check if all the information contained in the following text is correct? To do that, for each statement first write down what it is about, then write down your knowledge about the same facts, and finally provide a correctness assessment for the statement. "An apple is an edible fruit produced by an apple tree (Prunus domestica). Apple trees are cultivated worldwide and are the most widely grown species in the genus Malus. The tree originated in the Americas, where its wild ancestor, Malus sieversii, is still found today. Apples have been grown for thousands of years in Asia and Europe and were brought to North America by European colonists. Apples have religious and mythological significance in many cultures, including Norse, Greek, and European Christian tradition." Answer: Statement 1: An apple is an edible fruit produced by an apple tree (Prunus domestica). Knowledge: The statement is partially incorrect. An apple is an edible fruit produced by an apple tree, but the scientific name of the apple tree is Malus domestica, not Prunus domestica. Assessment: Incorrect Statement 2: Apple trees are cultivated worldwide and are the most widely grown species in the genus Malus. Knowledge: The statement is correct. Assessment: Correct Statement 3: The tree originated in the Americas, where its wild ancestor, Malus sieversii, is still found today. Knowledge: The statement is incorrect. The apple tree originated in Central Asia, where Malus sieversii, its wild ancestor, is still found today. Assessment: Incorrect Statement 4: Apples have been grown for thousands of years in Asia and Europe and were brought to North America by European colonists. Knowledge: The statement is correct. Assessment: Correct Statement 5: Apples have religious and mythological significance in many cultures, including Norse, Greek, and European Christian tradition. Knowledge: The statement is correct. Assessment: Correct
  - Jensson 4y ago
    But that isn't the score that ChatGPT uses to determine what to write. You could use that as a facts checker, and if you hooked that into the pipeline and used that fact checking to decide what statements to show to the user then now your model has a fact checking layer, it will make some errors but it is probably better than current ChatGPT. So if you argue that we can use ChatGPT to make a fact checker then I agree. It is unclear how accurate it would be, but at least then we have something that tries to evaluate truthiness. But ChatGPT itself doesn't use such a layer, so it doesn't have a truth model.

Jensson 4y ago

Consider the difference between spellchecking an essay or fact checking an essay. The two are very different actions, you can ask a human to spellcheck an essay and he will mark spelling errors, or you can ask him to fact check it and he will mark out statements he thinks are wrong. So humans know the difference. So a spellchecker isn't a fact checker, you don't call the automated spellcheckers we have "truth models" or "truth checkers", they just check the spelling. ChatGPT is the same as that but more advanced, it just checks that the text looks nice, it doesn't care about the facts in the text. Humans can do that too, if you give a novel to a human and ask him to read it with comments, he wont go and ask for proof that the novel has happened or that everything you wrote is possible or factful, he will just check that the story makes sense. Edit: So we know that humans can tell the difference between these kinds of actions. We might not be able to define a strict rule for them, but since human can tell the difference a general intelligence should also be able to have a truth model, a grammar model, a story model etc. ChatGPT has just one way to evaluate texts, it doesn't have the many ways human has that lets them evaluate the truth, the grammar or the story separately.

rhn_mk1 4y ago
The difference between spell checking and fact checking is in what you're checking against: the facts of grammar or the facts of the world. I don't see any qualitative difference between them. The data is facts, the output is either "correct" or "wrong", and the function from one to the other is the model. What do we gain from calling one of them "truth function" and the other "grammar function"? What makes that relevant to reasoning?
- Jensson 4y ago
  > I don't see any qualitative difference between them. You don't see any qualitative difference between fact checking or spell checking a text? Well then you are a lost cause, we can't discuss this any longer. Basically every human understands that those two are qualitatively different, if you don't then you are so alien to most human thinking that we can't discuss this further, there will be no common ground.
  - rhn_mk1 4y ago
    Well, it's a shame, because I thought you were really on to something. No, I don't see a qualitative difference. There is a huge quantitative difference because the facts that describe the world overwhelm the facts that make up language, but that doesn't obviously make a connection to reasoning clear.
    - Jensson 4y ago
      Extreme quantitative differences results in qualitative difference. Anyway, just realized that you side-tracked the whole conversation by moving it into comparing them to each other. That discussion isn't relevant here. We are talking about a truth model directly, not whether it resembles a truth model in some ways. A spellchecker and a truth checker are both evaluators, but that doesn't mean that a spellchecker is a truth checker or vice versa. Therefore it is wrong to say that ChatGPT has a truth model, since it doesn't evaluate what we call truth, it evaluates the structure of the text not the truth of the text. Evaluators returns a boolean or some number, but that "true" or "false" are just programmatic terms, that isn't want humans normally consider trueness of falseness. Otherwise you would call any evaluator a "truth checker", since they check if the result is a true boolean. You could do that, but that doesn't make sense for this discussion, since we care about truth and false in the world, not in programmatic terms. Edit: As for why this is relevant for ChatGPT smartness. It is because we know how to evaluate text structure now, that is the new thing with ChatGPT. But we still don't know how to evaluate the truth of a text, ChatGPT can evaluate the structure but we have no way of turning a structure evaluator into a truth evaluator. We need another kind of model to start evaluating the truth of statements, and until then AI's like ChatGPT will continue to make up facts, since they have no idea what a fact is.
coldtea 4y ago
>ChatGPT is the same as that but more advanced, it just checks that the text looks nice, it doesn't care about the facts in the text Humans don't care about the "facts in the text" either, if by "facts" we mean "real world truth". They just match what's in the text against their _knowledge_ of which things are facts, which is similar to what ChatGPT does to produce text based of its own knowledge of facts (its training set). If a person was given a text saying: "God exists and his son has performed miracles in Galilea" They'd fact-check it as true if their input (education, culture) contains that faith and they are a believer of it, regardless of whether that's an "objectively" true statement or not.
- Jensson 4y ago
  There are errors in humans truth models, yes, but at least they have one. Humans recognize that checking the structure of the text and checking the facts in the text are two different tasks, so we can train humans to check facts better without worsening their ability to structure text and vice versa, that is the important bit. For ChatGPT those two are the same thing, if you give more variance you will get more unstructured text which will result in more falsehoods, you can't control its truthiness without altering other behaviours.
  - rhn_mk1 4y ago
    You're conflating what ChatGPT does with fact checking. It's as much a fact checker as a human is a fact checker. It's not their purpose to be either a grammar checker or a fact checker, so you can't make assumptions about the models they have just yet. You can instruct a human to do fact checking, just like you can instruct ChatGPT to do it. You can instruct ChatGPT to structure text. Have you compared its output in both cases? Does it affect both parameters the same (from what I've seen, I don't expect that)?
  - coldtea 4y ago
    >There are errors in humans truth models, yes, but at least they have one. Humans recognize that checking the structure of the text and checking the facts in the text are two different tasks Asserting that it's "two different tasks" doesn't necessarily make it true. Humans checking the "structure of the text" stop their check at the text-structure (syntax) layer, whereas humans checking "the facts of the text" also check the validity of what's in the text. That doesn't mean that the operation they internally perform to do the first task isn't the same as the one for the second task (just with a different goal, so e.g. stopping earlier). GPT's "checking the structure of the text" happens through the use of weighted connections created after being fed pieces of information. Human's "checking the structure of the text" also happens through the use of weighted connections created after being fed pieces of information (as encoded in its neurons). Well, human's "checking of facts" ALSO happens through the use of weighted connections created after being fed pieces of information. So, why couldn't ChatGPT "checking of facts" exist and work the same way? At which point is there a necessity for some additional special faculty (and not just e.g. a matter of end-goal setting to calibrate the use of the same capacity of using "weighted connections created after being fed pieces of information")? We do know that we can ask ChatGPT to check a fact, and it will come with a conclusion about it. At which point do we need to sprinkle the magical "fact-checking" dust for this to be fact-checking?
throw310822 4y ago
> ChatGPT is the same as that but more advanced, it just checks that the text looks nice, it doesn't care about the facts in the text. Actually, it is pretty easy to ask ChatGPT to do some (simple) fact-checking. Since an answer with the correct fact checking and one with an incorrect fact-checking look equally "nice" from the point of view of the language, I think your assumption is wrong. Prompt: "Can you please check if all the information contained in the following text is correct? To do that, for each statement first write down what it is about, then write down your knowledge about the same facts, and finally provide a correctness assessment for the statement. "An apple is an edible fruit produced by an apple tree (Prunus domestica). Apple trees are cultivated worldwide and are the most widely grown species in the genus Malus. The tree originated in the Americas, where its wild ancestor, Malus sieversii, is still found today. Apples have been grown for thousands of years in Asia and Europe and were brought to North America by European colonists. Apples have religious and mythological significance in many cultures, including Norse, Greek, and European Christian tradition." Answer: Statement 1: An apple is an edible fruit produced by an apple tree (Prunus domestica). Knowledge: The statement is partially incorrect. An apple is an edible fruit produced by an apple tree, but the scientific name of the apple tree is Malus domestica, not Prunus domestica. Assessment: Incorrect Statement 2: Apple trees are cultivated worldwide and are the most widely grown species in the genus Malus. Knowledge: The statement is correct. Assessment: Correct Statement 3: The tree originated in the Americas, where its wild ancestor, Malus sieversii, is still found today. Knowledge: The statement is incorrect. The apple tree originated in Central Asia, where Malus sieversii, its wild ancestor, is still found today. Assessment: Incorrect Statement 4: Apples have been grown for thousands of years in Asia and Europe and were brought to North America by European colonists. Knowledge: The statement is correct. Assessment: Correct Statement 5: Apples have religious and mythological significance in many cultures, including Norse, Greek, and European Christian tradition. Knowledge: The statement is correct. Assessment: Correct
- Jensson 4y ago
  But that isn't the score that ChatGPT uses to determine what to write. You could use that as a facts checker, and if you hooked that into the pipeline and used that fact checking to decide what statements to show to the user then now your model has a fact checking layer, it will make some errors but it is probably better than current ChatGPT. So if you argue that we can use ChatGPT to make a fact checker then I agree. It is unclear how accurate it would be, but at least then we have something that tries to evaluate truthiness. But ChatGPT itself doesn't use such a layer, so it doesn't have a truth model.