3 ms·

> factful texts That might be the problem here. Humans can't agree on facts apart from some very basic ones, so I'd argue we don't have truly "factful texts" a

> factful texts That might be the problem here. Humans can't agree on facts apart from some very basic ones, so I'd argue we don't have truly "factful texts" at all. We have our own definitions of truth, and they are all built on flawed data. If factful texts are needed to have a model of truth, then humans don't have one, either. Or at least then humans have imperfect models of truth informed by the training data, and, as far as I know, this fits ChatGPT as well. > pretty easy to guess that having a fact checking model instead of just a text structure evaluator would greatly improve reasoning I read this as "feeding the model better data will improve things" - after all, I just explained that I see fact checking and structure evaluation as effectively the same kind of a task. And with that in mind, I do agree with this guess :)

Jensson 4y ago
> I just explained that I see fact checking and structure evaluation as effectively the same kind of a task. The tasks correlate, which is why ChatGPT doesn't always spout nonsense, but they aren't the same. Facts are a tiny part of the structure so even though structure correlates a bit with facts most of the structure just has to do with grammar, so it prefers grammatically nice text over correct text. Similarly fact checking can mostly ignore grammar and structure, as long as the relationships of items is clear you can just look at the logic. I think that looking at just structure gives a way too weak signal here, which is why ChatGPT is so dumb even though it has so much data and compute. Imagine if instead we trained the structural model to extract the non-structural logical parts, and then train another model on just the logic, that would lessen the logical burden of the structure model while also give us a separate and very pure logic signal that can be used as a truth model. That might be all we need for general intelligence, but I doubt we will get there without such a separation. So yeah, I get what you are saying, but as I said structural checks correlates a bit with truth checks, but not nearly enough to call them the same. A structural check would prefer a presidential speech full of lies over the messy forum comment debunking all those lies, so it isn't a truth model it is a structure model, even though those correlate a bit.
- rhn_mk1 4y ago
  I don't mean that fact checking correlates to structure. I mean that there is no definition of "structure" I can think of that is not entirely dictated by facts (knowledge/training data). Grammatical structure is stretched on facts of grammar, for example. So I don't see the point of saying "facts are a tiny part of structure" because "facts" as you use them guide some sort of structure, too. Just different than the grammatical structure. You say "ChatGPT is dumb" meaning that it says untrue things. But that doesn't mean that it doesn't have a truth model. It does mean that its truth model (if it has one) doesn't match yours. This is not surprising. Garbage in, garbage out. It also says nothing about its reasoning capability. After all, you can't reason much good out of bad data. So while you might be truth that some internal structure changes might improve its truth output, I also see no reason why improving its data wouldn't improve its truthfulness.
  - Jensson 4y ago
    > I mean that there is no definition of "structure" I can think of that is not entirely dictated by facts (knowledge/training data). This would imply that every scoring function is a fact checker, which would make the term "fact checker" meaningless. So what you did was making a "technically correct" argument by abusing the fact that "fact checker" is ill defined, so you used the widest possible interpretation of the word to make it include the case we are talking about. Arguments like that are nonsense, you know it doesn't make sense to define fact checking like that since it makes the term meaningless, you just did it ad hoc for this argument, you aren't that dumb, but trying to win arguments by bending your own logic in ways that you know is wrong is just hurting you, it wont convince anyone and it will hurt your ability to learn and see more perspectives. Edit: But if you define "fact checker" to be "any function", then sure, I agree ChatGPT is a fact checker, it is a function. But then why didn't you just say "I think all functions are fact checkers, ChatGPT is a function, hence it is a fact checker" from the start? Would have saved a lot of time.
    - rhn_mk1 4y ago
      > This would imply that every scoring function is a fact checker Right. A "fact checker" in this meaning is a function that checks against a body of facts. A "fact checker" in the colloquial meaning is someone performing a check against the body of facts they can find in resources available to them. I don't think this takes away anything from the term, but it does highlight that the task is not objective, and that the result depends on feeding the function with the correct body of facts. What follows is that if you feed ChatGPT the "correct" body of facts, you might get a fact checker in the colloquial meaning. That "fact checker" is ill-defined matters a lot because the argument here was that you can't do any reasoning if you don't have a truth model (which I guess was implied to be the "fact checker"). And if the argument is based on ill-defined terms, then it doesn't prove anything about whether GPT can do reasoning or not. EDIT: Actually, scratch it. I do mean that any knowledge, text => boolean function is a "fact checker" and constitutes a "truth model". If some other "truth model" is meant, please explain, because I don't see how this one can be used to prove anything about the ability to reason based on whether someone implements it or not.

Jensson 4y ago

> I just explained that I see fact checking and structure evaluation as effectively the same kind of a task. The tasks correlate, which is why ChatGPT doesn't always spout nonsense, but they aren't the same. Facts are a tiny part of the structure so even though structure correlates a bit with facts most of the structure just has to do with grammar, so it prefers grammatically nice text over correct text. Similarly fact checking can mostly ignore grammar and structure, as long as the relationships of items is clear you can just look at the logic. I think that looking at just structure gives a way too weak signal here, which is why ChatGPT is so dumb even though it has so much data and compute. Imagine if instead we trained the structural model to extract the non-structural logical parts, and then train another model on just the logic, that would lessen the logical burden of the structure model while also give us a separate and very pure logic signal that can be used as a truth model. That might be all we need for general intelligence, but I doubt we will get there without such a separation. So yeah, I get what you are saying, but as I said structural checks correlates a bit with truth checks, but not nearly enough to call them the same. A structural check would prefer a presidential speech full of lies over the messy forum comment debunking all those lies, so it isn't a truth model it is a structure model, even though those correlate a bit.

rhn_mk1 4y ago
I don't mean that fact checking correlates to structure. I mean that there is no definition of "structure" I can think of that is not entirely dictated by facts (knowledge/training data). Grammatical structure is stretched on facts of grammar, for example. So I don't see the point of saying "facts are a tiny part of structure" because "facts" as you use them guide some sort of structure, too. Just different than the grammatical structure. You say "ChatGPT is dumb" meaning that it says untrue things. But that doesn't mean that it doesn't have a truth model. It does mean that its truth model (if it has one) doesn't match yours. This is not surprising. Garbage in, garbage out. It also says nothing about its reasoning capability. After all, you can't reason much good out of bad data. So while you might be truth that some internal structure changes might improve its truth output, I also see no reason why improving its data wouldn't improve its truthfulness.
- Jensson 4y ago
  > I mean that there is no definition of "structure" I can think of that is not entirely dictated by facts (knowledge/training data). This would imply that every scoring function is a fact checker, which would make the term "fact checker" meaningless. So what you did was making a "technically correct" argument by abusing the fact that "fact checker" is ill defined, so you used the widest possible interpretation of the word to make it include the case we are talking about. Arguments like that are nonsense, you know it doesn't make sense to define fact checking like that since it makes the term meaningless, you just did it ad hoc for this argument, you aren't that dumb, but trying to win arguments by bending your own logic in ways that you know is wrong is just hurting you, it wont convince anyone and it will hurt your ability to learn and see more perspectives. Edit: But if you define "fact checker" to be "any function", then sure, I agree ChatGPT is a fact checker, it is a function. But then why didn't you just say "I think all functions are fact checkers, ChatGPT is a function, hence it is a fact checker" from the start? Would have saved a lot of time.
  - rhn_mk1 4y ago
    > This would imply that every scoring function is a fact checker Right. A "fact checker" in this meaning is a function that checks against a body of facts. A "fact checker" in the colloquial meaning is someone performing a check against the body of facts they can find in resources available to them. I don't think this takes away anything from the term, but it does highlight that the task is not objective, and that the result depends on feeding the function with the correct body of facts. What follows is that if you feed ChatGPT the "correct" body of facts, you might get a fact checker in the colloquial meaning. That "fact checker" is ill-defined matters a lot because the argument here was that you can't do any reasoning if you don't have a truth model (which I guess was implied to be the "fact checker"). And if the argument is based on ill-defined terms, then it doesn't prove anything about whether GPT can do reasoning or not. EDIT: Actually, scratch it. I do mean that any knowledge, text => boolean function is a "fact checker" and constitutes a "truth model". If some other "truth model" is meant, please explain, because I don't see how this one can be used to prove anything about the ability to reason based on whether someone implements it or not.