4 ms·

The article starts out in bold with the statement they pretend to be like minds – like human minds. But it is only pretend, there is no mind there and that is

The article starts out in bold with the statement they pretend to be like minds – like human minds. But it is only pretend, there is no mind there and that is the key to understanding what ChatGPT is And surely they have no mind like us. But this obsession with whether or not it is like us, seems to miss the point, doesn't it. Is it useful? Despite the writers insistence that it is not, my children provide my with many examples that it is. For example, my daughter is learning to program and got negative feedback on her commenting style. (which admittedly where mostly absent) So, she asked how to do it better. The teacher gave only non-committal responses, like being helpful to the reader, etc. It is not the kind of instruction that is helpful for her. So she gave the whole program the ChatGTP and asked it to insert comments without changing the code. The next day, she took the output to the teacher and asked if this is what he meant. It was perfect. But now, she knows what is expected of her, and now she can do it without ChatGTP.

Gigachad 4y ago
When you drill down these discussion points, it all comes down to the fact we don’t exactly know how the human mind works so we can’t properly answer what is the difference between pretending to understand, and understanding.
- Jensson 4y ago
  We know that some humans can use information to solve harder problems, while others can not do it. We say the first kind built understanding about the information. You can easily test for this by just throwing hard problems at them that they haven't seen before, Google does that in their interviews for example (but many who do similar just takes problems people definitely have seen). So doing similar kinds of testing on ChatGPT we see that it doesn't understand the information. It can repeat a lot of stuff more or less verbatim, but it cannot apply most of the knowledge it can repeat, so it doesn't understand any of it. The only thing ChatGPT really understands are relationships between words that has been repeated in texts it has seen, that is some level of understanding but that has no relationship to what ChatGPT says about the words. When you ask ChatGPT to describe something it repeats a description it has seen of the word, when you ask it to do something with a word then it fully ignores that description and instead look at the relationships it has seen the word have to other words. So its understanding and its knowledge are completely separate things, so you can say that it doesn't understand anything it knows.
qsort 4y ago
> And surely they have no mind like us. I agree with this, it's the old "are submarines swimming" kind of question, semantics and not substance, i.e. irrelevant. However this is an article written by someone non-technical for other non-technical people, and this (emphasis is the author's) is perfectly on-point: > The tricky part is that ChatGPT and chatbots like it are designed to make use of a very influential human cognitive bias that we all have: the tendency to view things which are not people as people or at least as being like people. Which is absolutely a problem. Whether or not you ascribe a "mind" to it, laypeople absolutely need to understand that it's not a human they're talking to. > Is it useful? A better question is useful to whom and useful how. As a search engine it's pretty miserable for instance, as we found out. I'd like to leave myself some wiggle room because it's way too early to tell, but my hunch is that operating on natural language is a profound limitation that will make it far less useful than it is generally believed.
- collaborative 4y ago
  I have consistently been helped by ChatGPT every time Google has failed me. As a search engine it's really good. Only failing is re new data
- visarga 4y ago
  > laypeople absolutely need to understand that it's not a human they're talking to That's hard when the bot either gets angry at you or declares its love if you spend an hour with it. But is anthropomorphising AIs useful? that's the important question. I think it is useful. If you tell GPT3 it is a respected scientist, it will have higher accuracy solving tasks.
  - qsort 4y ago
    > But is anthropomorphising AIs useful? No it isn't. It's disastrous. The bot declares your love for you if you spend an hour with it because it's trained on incel shit from reddit, thinking anything else is 100% a user problem. > If you tell GPT3 it is a respected scientist, it will have higher accuracy solving tasks. Even taking that statement at face value, what I think about AI has no bearing on what it does. You are giving input tokens to a machine and you are fooling yourself if you think of it as anything but. This is exactly the problem about natural language I'm pointing out. I don't want a friend, a coworker or an assistant, I want a mechanical slave to instruct in very specific ways, and natural language is a pretty lousy way to give orders.
barrenko 4y ago
Buddhist and zen folks would tell you we have no mind ourselves. I'm seeing a convergence.
- greenpresident 4y ago
  Thomas Metzinger would agree with the assessment that we are a hallucination of a self and that we are Being No One.
  - jaggederest 4y ago
    I'm not sure hallucination is the right word, if we're operating in the theater of large language models. The correct way to describe human consciousness in those terms would be a descriptive model that outputs a lossy reconstructed linear viewpoint, combined with a prompt instructing it to believe the delusion that it's definitely for real in control and making choices instead of telling stories.
roomey 4y ago
This is a great idea, when I ask for help with something, so often the first thing is, code style, or variable names. This is the first hell yea use case I have seen for chat gtp! Thank you and your daughter!
BlueTemplar 4y ago
The article is quite specific about ChatGPT not being useful for writing essays. It also carefully speculates about some other uses. It even insists on it being a very fancy form of autocompletion, so it shouldn't be surprising that it was useful in your example. And it also points out that it's bad teaching that should be concerned about it, which is also seems the case for your daughter's teacher (for perhaps a very mild example of bad teaching).
candiodari 4y ago
> And surely they have no mind like us. I've often wondered about this, because this answers the question "are we alone on this planet?" with a clear, resounding "no". For example, every species has a mind in it's DNA. It communicates very slowly and it thinks and communicates by killing large amounts of life. Or perhaps it can be better put as it communicates by enhancing or diminishing specific forms of life, not always outright killing them. You can talk to this, like we've done accidentally with vaccines. DNA's answer would then be antibiotic resistance in dangerous diseases. It's a very different "mind".

Gigachad 4y ago

When you drill down these discussion points, it all comes down to the fact we don’t exactly know how the human mind works so we can’t properly answer what is the difference between pretending to understand, and understanding.

Jensson 4y ago
We know that some humans can use information to solve harder problems, while others can not do it. We say the first kind built understanding about the information. You can easily test for this by just throwing hard problems at them that they haven't seen before, Google does that in their interviews for example (but many who do similar just takes problems people definitely have seen). So doing similar kinds of testing on ChatGPT we see that it doesn't understand the information. It can repeat a lot of stuff more or less verbatim, but it cannot apply most of the knowledge it can repeat, so it doesn't understand any of it. The only thing ChatGPT really understands are relationships between words that has been repeated in texts it has seen, that is some level of understanding but that has no relationship to what ChatGPT says about the words. When you ask ChatGPT to describe something it repeats a description it has seen of the word, when you ask it to do something with a word then it fully ignores that description and instead look at the relationships it has seen the word have to other words. So its understanding and its knowledge are completely separate things, so you can say that it doesn't understand anything it knows.

qsort 4y ago

> And surely they have no mind like us. I agree with this, it's the old "are submarines swimming" kind of question, semantics and not substance, i.e. irrelevant. However this is an article written by someone non-technical for other non-technical people, and this (emphasis is the author's) is perfectly on-point: > The tricky part is that ChatGPT and chatbots like it are designed to make use of a very influential human cognitive bias that we all have: the tendency to view things which are not people as people or at least as being like people. Which is absolutely a problem. Whether or not you ascribe a "mind" to it, laypeople absolutely need to understand that it's not a human they're talking to. > Is it useful? A better question is useful to whom and useful how. As a search engine it's pretty miserable for instance, as we found out. I'd like to leave myself some wiggle room because it's way too early to tell, but my hunch is that operating on natural language is a profound limitation that will make it far less useful than it is generally believed.

collaborative 4y ago
I have consistently been helped by ChatGPT every time Google has failed me. As a search engine it's really good. Only failing is re new data
visarga 4y ago
> laypeople absolutely need to understand that it's not a human they're talking to That's hard when the bot either gets angry at you or declares its love if you spend an hour with it. But is anthropomorphising AIs useful? that's the important question. I think it is useful. If you tell GPT3 it is a respected scientist, it will have higher accuracy solving tasks.
- qsort 4y ago
  > But is anthropomorphising AIs useful? No it isn't. It's disastrous. The bot declares your love for you if you spend an hour with it because it's trained on incel shit from reddit, thinking anything else is 100% a user problem. > If you tell GPT3 it is a respected scientist, it will have higher accuracy solving tasks. Even taking that statement at face value, what I think about AI has no bearing on what it does. You are giving input tokens to a machine and you are fooling yourself if you think of it as anything but. This is exactly the problem about natural language I'm pointing out. I don't want a friend, a coworker or an assistant, I want a mechanical slave to instruct in very specific ways, and natural language is a pretty lousy way to give orders.

barrenko 4y ago

Buddhist and zen folks would tell you we have no mind ourselves. I'm seeing a convergence.

greenpresident 4y ago
Thomas Metzinger would agree with the assessment that we are a hallucination of a self and that we are Being No One.
- jaggederest 4y ago
  I'm not sure hallucination is the right word, if we're operating in the theater of large language models. The correct way to describe human consciousness in those terms would be a descriptive model that outputs a lossy reconstructed linear viewpoint, combined with a prompt instructing it to believe the delusion that it's definitely for real in control and making choices instead of telling stories.

roomey 4y ago

This is a great idea, when I ask for help with something, so often the first thing is, code style, or variable names. This is the first hell yea use case I have seen for chat gtp! Thank you and your daughter!

BlueTemplar 4y ago

The article is quite specific about ChatGPT not being useful for writing essays. It also carefully speculates about some other uses. It even insists on it being a very fancy form of autocompletion, so it shouldn't be surprising that it was useful in your example. And it also points out that it's bad teaching that should be concerned about it, which is also seems the case for your daughter's teacher (for perhaps a very mild example of bad teaching).

candiodari 4y ago

> And surely they have no mind like us. I've often wondered about this, because this answers the question "are we alone on this planet?" with a clear, resounding "no". For example, every species has a mind in it's DNA. It communicates very slowly and it thinks and communicates by killing large amounts of life. Or perhaps it can be better put as it communicates by enhancing or diminishing specific forms of life, not always outright killing them. You can talk to this, like we've done accidentally with vaccines. DNA's answer would then be antibiotic resistance in dangerous diseases. It's a very different "mind".