5 ms·

I think the most useful definition of understanding something is that you can explain it and use it in context. ChatGPT routinely does both.

jstummbillig 3y ago
And while AI gets better and better and we will remain as touchy as ever about abstract concepts that make us oh so human, how about we say it just can't be understanding, unless a human does, eh, it.
BobaFloutist 3y ago
Someone sufficiently fast and skilled at googling can explain and use in context a lot of things that they don't really properly understand. So unless you're saying that the composite of the googler and of google understand something that neither does individually, your definition has some holes.
- continuational 3y ago
  This is a variation of the Chinese room argument. If you consider understanding an observable property, then the Chinese room in aggregate displays understanding of Chinese. Would you say that humans understand nothing, because atoms don't understand anything, and we're made up of atoms?
  - BobaFloutist 3y ago
    I would say that there is a stronger consensus that a human being can be reasonably described as a single entity than a human being using a reference resource. A more apt comparison to my mind would be if a human being can be described as personally exerting strong nuclear force, just because their subatomic particles do, which I would happily answer "no."
skepticATX 3y ago
How about this: understanding is the ability to generalize knowledge and apply it to novel scenarios. This definition is something that humans, and animals for that matter, do every day - both in small and large ways. And this is something that current language models aren't very good at.
- continuational 3y ago
  What is the test for this? I taught it Firefly, which is an undocumented programming language I'm working on, through conversion. I find it's a lot quicker than any human at picking up syntax and semantics, both in real time and in number of messages, and makes pretty good attempts at writing code in it, as much as you could expect from a human programmer. That is, until you run out of context - is this what you mean?
  - skepticATX 3y ago
    There are plenty of results supporting my assertion; but the tests must be carefully designed. Of course, LLMs are not databases that store exact answers - so it's not enough to ask it something that it hasn't seen, if it's seen something similar (as is likely the case with your programming language). One benchmark that I track closely is ConceptARC, which aims to test generalization and abstraction capabilities. Here is a very recent result that uses the benchmark: https://arxiv.org/abs/2311.09247 https://arxiv.org/abs/2311.09247. Humans correctly solved 91% of the problems, GPT-4 solved 33%, and GPT-4V did much worse than GPT-4.
    - continuational 3y ago
      I wouldn't be surprised if GPT-4 is not too good at visual patterns, given that it's trained on text. Look at the actual prompt in figure 2. I doubt humans would get a 91% score on that.
  - stubybubs 3y ago
    I gave it the three lightbulbs in a closet riddle. https://puzzles.nigelcoldwell.co.uk/seven.htm https://puzzles.nigelcoldwell.co.uk/seven.htm The key complication is "once you've opened the door, you may no longer touch a switch." It gets this. There are many examples of it written out on the web. When I give it a variation and say "you can open the door to look at the bulbs and use the switches all you want" and it is absolutely unable to understand this. To a human it's simple: look at the bulbs and flick the switches. It kept giving me answers about using a special lens to examine the bulbs, using something to detect heat. I explained it in many ways and tried several times. I was paying for GPT-4 at the time as well. I would not consider this thinking. It's unable to make this simple abstraction from its training data. I think 4 looks better than 3 simply because it's got more data, but we're reaching diminishing returns on that, as has been stated.

jstummbillig 3y ago

And while AI gets better and better and we will remain as touchy as ever about abstract concepts that make us oh so human, how about we say it just can't be understanding, unless a human does, eh, it.

BobaFloutist 3y ago

Someone sufficiently fast and skilled at googling can explain and use in context a lot of things that they don't really properly understand. So unless you're saying that the composite of the googler and of google understand something that neither does individually, your definition has some holes.

continuational 3y ago
This is a variation of the Chinese room argument. If you consider understanding an observable property, then the Chinese room in aggregate displays understanding of Chinese. Would you say that humans understand nothing, because atoms don't understand anything, and we're made up of atoms?
- BobaFloutist 3y ago
  I would say that there is a stronger consensus that a human being can be reasonably described as a single entity than a human being using a reference resource. A more apt comparison to my mind would be if a human being can be described as personally exerting strong nuclear force, just because their subatomic particles do, which I would happily answer "no."

skepticATX 3y ago

How about this: understanding is the ability to generalize knowledge and apply it to novel scenarios. This definition is something that humans, and animals for that matter, do every day - both in small and large ways. And this is something that current language models aren't very good at.

continuational 3y ago
What is the test for this? I taught it Firefly, which is an undocumented programming language I'm working on, through conversion. I find it's a lot quicker than any human at picking up syntax and semantics, both in real time and in number of messages, and makes pretty good attempts at writing code in it, as much as you could expect from a human programmer. That is, until you run out of context - is this what you mean?
- skepticATX 3y ago
  There are plenty of results supporting my assertion; but the tests must be carefully designed. Of course, LLMs are not databases that store exact answers - so it's not enough to ask it something that it hasn't seen, if it's seen something similar (as is likely the case with your programming language). One benchmark that I track closely is ConceptARC, which aims to test generalization and abstraction capabilities. Here is a very recent result that uses the benchmark: https://arxiv.org/abs/2311.09247 https://arxiv.org/abs/2311.09247. Humans correctly solved 91% of the problems, GPT-4 solved 33%, and GPT-4V did much worse than GPT-4.
  - continuational 3y ago
    I wouldn't be surprised if GPT-4 is not too good at visual patterns, given that it's trained on text. Look at the actual prompt in figure 2. I doubt humans would get a 91% score on that.
- stubybubs 3y ago
  I gave it the three lightbulbs in a closet riddle. https://puzzles.nigelcoldwell.co.uk/seven.htm https://puzzles.nigelcoldwell.co.uk/seven.htm The key complication is "once you've opened the door, you may no longer touch a switch." It gets this. There are many examples of it written out on the web. When I give it a variation and say "you can open the door to look at the bulbs and use the switches all you want" and it is absolutely unable to understand this. To a human it's simple: look at the bulbs and flick the switches. It kept giving me answers about using a special lens to examine the bulbs, using something to detect heat. I explained it in many ways and tried several times. I was paying for GPT-4 at the time as well. I would not consider this thinking. It's unable to make this simple abstraction from its training data. I think 4 looks better than 3 simply because it's got more data, but we're reaching diminishing returns on that, as has been stated.