3 ms·

> It can't reason at all. This isn't true. You can ask it basic logic problems that it's never seen before, and it will apply the rules of logic to them. It

> It can't reason at all. This isn't true. You can ask it basic logic problems that it's never seen before, and it will apply the rules of logic to them. It can also identify correctly which rules of logic would make sense to apply in more complex situations, even when it doesn't get the answer right straight away. At doing this stuff GPT4 is better than GPT 3.5 which is better than previous GPTs. I fully expect that future models will be able to tackle more complex applications of logic to new domains successfully. If you use only examples that weren't in its training set, you'll get to its limits quickly, but basic level first order logic is definitely within its ability.

zarzavat 4y ago
It can reason but it has limited ability to transfer its reasoning skills into other domains. For example, it can take code and add types to it. This involves a lot of reasoning ability. It can do this because it’s been trained on a vast amount of code. But it can’t yet fully transfer those reasoning abilities outside the narrow domain of code.
- SanderNL 4y ago
  What is a domain? You talk of it like you know what it means. These things have no concept of a domain. If they use reasoning they use it all over the place. It is up to us to judge domainness.
  - zarzavat 4y ago
    Domain as in a context, the type of data that they were trained on. Code vs text. English vs Russian. Python vs APL. For instance, OpenAI found that GPT4 is much better at reasoning in some human languages than others. It is best at reasoning in English, but struggles reasoning in less resourced languages. There is clearly some context-independent reasoning going on (i.e. generalization) otherwise the model would not be able to reason at all in languages that it hasn’t seen a particular problem in. But there also appears to be a large context-dependent factor.
    - SanderNL 4y ago
      I see, very interesting.
hawski 4y ago
It is a side effect of human language and a huge corpus of text it was trained on. I know that people sometimes use this kind of reasoning, words have attached meaning and then you can do some simple reasoning. I would say though, that it is not a real reasoning. It can do some of the things real reasoning can do, but the same as with people it can get attached to the meaning of words too much. When a person can't go beyond that they are called lazy or sloppy thinkers. It is a heuristic, not a real reasoning. I think it is a cheaper processing and it is of course useful, but it has its limits. For me it will mean that ChatGPT will not get to see some patterns that in creative works would be obvious for a creative human. As you said, when things are not in its training set it can struggle. If there is a plausible looking text for the question I asked it will give it to me, that's how it is designed. For example I asked it about Windows command line debugger - CDB. It gave me an example command line for it: cdb -c "your-app" -o "logfile". It is very wrong. -c requires an argument which are debugging commands to run on start, -o is to attach to all created attached processes. Real command line looks something like this: cdb -logo "logfile" "your-app" (and it still does not exactly behave as you would imagine having experience with Unix CLI). The problem ChatGPT has with CDB is probably, because it has much much bigger corpus on Unix-like command line tools and because the documentation for CDB is abysmal. From this ChatGPT session I would have more examples. I'm not saying it is useless. It just is not designed to do that. It might improve or there might be an another algorithm needed on top or instead of what it does use. For me it is like a kind of a step up from a search engine. It helps me to find something to start with. When I get to some details it is often wrong. I get a starting point from it and then find a proper source for the rest.

zarzavat 4y ago

It can reason but it has limited ability to transfer its reasoning skills into other domains. For example, it can take code and add types to it. This involves a lot of reasoning ability. It can do this because it’s been trained on a vast amount of code. But it can’t yet fully transfer those reasoning abilities outside the narrow domain of code.

SanderNL 4y ago
What is a domain? You talk of it like you know what it means. These things have no concept of a domain. If they use reasoning they use it all over the place. It is up to us to judge domainness.
- zarzavat 4y ago
  Domain as in a context, the type of data that they were trained on. Code vs text. English vs Russian. Python vs APL. For instance, OpenAI found that GPT4 is much better at reasoning in some human languages than others. It is best at reasoning in English, but struggles reasoning in less resourced languages. There is clearly some context-independent reasoning going on (i.e. generalization) otherwise the model would not be able to reason at all in languages that it hasn’t seen a particular problem in. But there also appears to be a large context-dependent factor.
  - SanderNL 4y ago
    I see, very interesting.

hawski 4y ago

It is a side effect of human language and a huge corpus of text it was trained on. I know that people sometimes use this kind of reasoning, words have attached meaning and then you can do some simple reasoning. I would say though, that it is not a real reasoning. It can do some of the things real reasoning can do, but the same as with people it can get attached to the meaning of words too much. When a person can't go beyond that they are called lazy or sloppy thinkers. It is a heuristic, not a real reasoning. I think it is a cheaper processing and it is of course useful, but it has its limits. For me it will mean that ChatGPT will not get to see some patterns that in creative works would be obvious for a creative human. As you said, when things are not in its training set it can struggle. If there is a plausible looking text for the question I asked it will give it to me, that's how it is designed. For example I asked it about Windows command line debugger - CDB. It gave me an example command line for it: cdb -c "your-app" -o "logfile". It is very wrong. -c requires an argument which are debugging commands to run on start, -o is to attach to all created attached processes. Real command line looks something like this: cdb -logo "logfile" "your-app" (and it still does not exactly behave as you would imagine having experience with Unix CLI). The problem ChatGPT has with CDB is probably, because it has much much bigger corpus on Unix-like command line tools and because the documentation for CDB is abysmal. From this ChatGPT session I would have more examples. I'm not saying it is useless. It just is not designed to do that. It might improve or there might be an another algorithm needed on top or instead of what it does use. For me it is like a kind of a step up from a search engine. It helps me to find something to start with. When I get to some details it is often wrong. I get a starting point from it and then find a proper source for the rest.