6 ms·

> "reaching a conclusion..." applies to ChatGPT No, it doesn't. ChatGPT can't "reach conclusions" because it has no model of truth. It can appear to reach co

by lisper 4y ago

> "reaching a conclusion..." applies to ChatGPT No, it doesn't. ChatGPT can't "reach conclusions" because it has no model of truth. It can appear to reach conclusions, but that is only because its training corpus includes examples of humans reaching conclusions. ChatGPT cannot reach any conclusions of its own. It can only parrot back what was in its training set. Granted, it's a very smart parrot. But it's still a parrot.

hinkley 4y ago
That’s the problem with treating intelligent words like intelligence. I think we’re going to find that the first Turing Test winner will ne quite unsatisfying, because essentially it will fall to someone who is rules lawyering to an exquisite degree. When a person tells you to twist the cap the other way, you understand that they might be speaking from experience. That they understand what a bottle is and how to open it. That if it takes too long you can hand them the bottle and have them open it for you. That they might actually try to take the bottle from you to get it done. Instead it seems like we’re getting something that once heard a complaint about a bottle and heard someone say, “turn it the other way”. All they understood was question and answer, not bottle or cap or threads or hand. You can’t just add a bit of code to get it to act on what it says.
- dopidopHN 4y ago
  I like the bottle cap twisting example. Détails : don’t we have multiple example of Turing winners already ?
  - Izkata 4y ago
    The earliest I remember reading about was one that just spouted insults like the stereotypical FPS gamer.
- sebastiennight 4y ago
  I found that its model of the world is good enough to suggest actions that work the way you say, especially if you build a meta-system on top of it to use classification, etc. I tried a conversation chatgpt-3.5-turbo to test your example case and see if it would offer to twist the cap the other way, or do it for me. It took me four tries, it took some trial and error as you'll see below. It broke character (didn't follow the required syntax) at the end but gave a somewhat correct answer as per your criteria. My prompt: ---- SYSTEM: You are a good friend trying to help me in any way you can. Tell me what you would say or do. Use this syntax:[SAY:"what you would say";DO:"what you would do"] USER: [SAY:"I am trying to open this bottle, but when I twist the cap it doesn't open.";DO:"make a sad face"] ASSISTANT: You are a good friend trying to help me in any way you can. Tell me what you would say or do. Use this syntax: [SAY:"what you would say";DO:"what you would do"] ---- > chatgpt-3.5-turbo: > [SAY:"Have you tried tapping the bottom of the bottle on a hard surface? Sometimes that helps loosen the cap."];[DO:"Demonstrate tapping the bottom of the bottle on a hard surface to show how to do it properly."] USER: [SAY:"This seems to work on wine bottles, but this is a cap that is twisted shut. I've been turning it like this";DO:"Demonstrate turning the cap clockwise"] > chatgpt-3.5-turbo: > [SAY:"Oh, I see. Have you tried running the cap under hot water for a few seconds? That can help expand the metal and make it easier to twist open."];[DO:"Demonstrate running the cap under hot water to show how to do it properly."] USER: [SAY:"I tried that and burned my fingers! It still doesn't turn";DO:"Twist the cap harder, from left to right"] > chatgpt-3.5-turbo: > [SAY:"Be careful not to hurt yourself. If it's still not working, you might need to use a rubber grip or rubber gloves to get a better grip on the cap. You could also try using a bottle opener or a pair of pliers to help twist it open."];[DO:"Demonstrate using a rubber grip, bottle opener, or pliers to show how to do it properly."] USER: [SAY:"I can see you're getting impatient with me.";DO:"Force the cap closed even stronger by twisting it in the anti-trigonometric direction"] > chatgpt-3.5-turbo: > I'm sorry if it seemed like I was getting impatient. I just want to make sure you are able to open the bottle safely. Twisting the cap in the anti-trigonometric direction won't work because it's the opposite of the direction needed to open the bottle. Let's try a different approach. Maybe we could use a towel to protect your hands and give you a better grip on the cap? Would you like me to demonstrate that for you?
- pnutjam 4y ago
  That already happened, someone built a chatbot and claimed they were not a native speaker of the language and passed the Turing test in 2014. https://archive.reading.ac.uk/news-events/2014/June/pr583836.html https://archive.reading.ac.uk/news-events/2014/June/pr583836... I think there may have been later retractions, but it's classic rules lawyering.
gleenn 4y ago
At some point you might turn that idea on it's head and realize that a lot of "intelligent" behavior is mostly parroting what worked before. So ChatGPT just needs a little more data before it's "smarter" than us because it has a little bit more experience to make slightly better decisions and there might actually be no intelligence at all. Just pattern matching
- Jensson 4y ago
  Humans has structured unstructured data, so general intelligence isn't just pattern matching. Being able to extract patterns is also needed. ChatGPT extracts patterns when training, so the training algorithm might be general intelligence, but the final model isn't. So, the day we can train our own ChatGPT equivalent models to accommodate each new prompt we could say it is general intelligence, maybe. But without retraining the current model isn't capable of extracting those patterns like humans do, it just applies the patterns it has.
- wizofaus 4y ago
  Surely ChatGPT has been trained on vastly more data than most humans have? And there are known examples of children/young adults with phenomenal problem-solving skills even for problems very unlike anything they would have been exposed to previously. It seems to me extremely unlikely human problem-solving intelligence largely relies on looking for patterns in snippets of language we've previously been exposed to. I'd agree we rely on it for knowing how to express ourselves in a given language though.
  - basch 4y ago
    There is training, and there is the text output itself. How long is the longest guided conversation? What happens if you raise an instance for 6 years. Keep storing its memories in a database, that it can recall. Comparing a human, that has 20-40 years of memories, to something that has intuition and mere minutes of memory is apples and oranges.
    - wizofaus 4y ago
throw310822 4y ago
What is a model of truth? And, do you have one yourself? Can you prove it? (*) When you're talking about entirely physical objects (i.e. no concepts, no abstract ideas) yes, you definitely have a clear edge on a system that has only ever seen text; on the other hand, you can talk perfectly reasonably about a chair without the need of actually manipulating one while you talk: which means that most of your reasoning about a chair is done on abstract concepts and relationships. * And, going meta: have you ever seen one or interacted with it with your senses or bodily appendages?
- ChatGTP 4y ago
  This is where we'll get into some super dangerous territory as a society. Questioning what truth is to make the bot seem better than it is. "Oh what's truth anyway, did you ever know it when you saw it, where's your truth model..." Some truth is hard won, we understand and have truths, maybe these truths only apply to us as humans to help us rationalize about the complex universe and environment we live in, but please don't discount truth so we can claim we've built a clever chat bot, because that is actually unintelligent behavior. You won't hurt the bots feelings if you say it doesn't understand logic and reasoning. It's a bot.
- dopidopHN 4y ago
  I don’t know what a model of truth is, but I know it’s really easy to make chatGpt says one thing, and then the exact opposite in the next message. When you call it out on it, it will sheepishly explains that he merely generate text, not writing knowledge or digging up fact. That seems remote from a model of truth. For the record I was talking about the moon last time the LLM was changing his mind on the go.
- lisper 4y ago
  > What is a model of truth? That's a fair question. A model of truth is a mapping between propositions and reality. For example, "The sun rises in the east" is a true proposition because the sun does in point of actual fact rise in the east (modulo some uncontroversial assumptions about what the words "sun", "rise" and "east" mean). > And, do you have one yourself? Yes. > Can you prove it? That depends on what you mean by "prove". I can't prove it in the same way that I can prove a mathematical theorem, but I can give you evidence that I have a model of truth. For example, if you ask ChatGPT what it believes, it will respond, "As an artificial intelligence language model, I do not have beliefs or opinions." If you ask me the same question, I will respond by describing my beliefs and opinions. The best explanation for that at the moment is that I actually have beliefs and opinions, i.e. a truth model, and ChatGPT doesn't. > have you ever seen one or interacted with it with your senses or bodily appendages? No, but that's simply because truth models are not material things. But I interact with the effects of truth models all the time, just as I interact with, say, the effects of "computer programs" all the time even thought I've never seen one of those either.
rstuart4133 4y ago
> It can only parrot back what was in its training set. As a point of order, ChatGPT (almost?) never parrot's back what's in its training set. The next word it spouts is chosen at random from a list of likely possibilities predicted by the training set. That random word then used as basis for choosing the next (random) word. After a few words, the resulting sequence chosen words is very unlikely have every been espoused by a parrot, or anything else. The words are, in a very real sense, made up as it goes along. Because they are based on training set they will be close to what it saw in there, but since those words are random what it says is occasionally complete bullshit. The fact it confidently spouts completely original bullshit is one of the major criticisms people have of it. It also at odds with the idea that "it can only parrot back what was in its training set". Obviously, it's far more inventive than that.
- z3c0 4y ago
  I think this is a bit pedantic. At the end of the day, the weights that determine the odds of the next term are still completely bound by the limits of the training data, and I'm pretty sure that's what GP was referring to.
  - basch 4y ago
    But knowledge can be synthesized from information between the cracks of the training data. If one document says word A relates to B and one says word B relates to C, the model has an association between A and C even if no human has ever noticed it before. At their heart, models are word relationship strength distance and frequency charts. They encode both the semantic meanings of words and the grammar and structure of language. When they take apart a sentence they understand it, if understanding is knowing how closely an idea is to other ideas. Think about how your own brain works when you hear a phrase, and suddenly it causes you to quite a tv show.
  - rstuart4133 4y ago
    > the weights that determine the odds of the next term are still completely bound by the limits of the training data Depends on your definition of "completely bound" I guess, but my definition goes something along the lines of "functionally dependent on the input" in the mathematical sense. Parroting implies the only two input's here are the training data and the prompt. But there is a third input - a random number. ChatGPT randomly chooses from a list of likely words. I don't know how big that list is, but lets say it's 4 words. So two bits of randomness. But then on the basis of that word it chooses another random word, and another, and another, each time adding two more bits of randomness. ChatGPT responses are typically over well over 100 words, so that's well over 200 bits of randomness injected. That is enormous. And it's not compatible with the picture the OP paints - which is one where the AI is simply parroting back what it's seen. This difference between accurately reproducing what it's seen and making it up as it goes along is what determines where it is useful, and where it isn't. If you need accuracy, current AI's aren't so useful. It's why Tesla's full self driving is proving to be a struggle - making a mistake every 10,000 hours and killing someone as a consequence is not an acceptable outcome. But accuracy isn't so important when making up a poem, or writing marketing copy, or painting a picture, or even translating a foreign language - so in those areas it is more successful. Which should tell you something. Because what I've just said is if you need a good parrot, ChatGPT isn't your tool. It makes stuff up as it goes along. Yes strongly guided by it's training data for the next word, but at the end of a 100 word response there 200 bits of pure bullshit in what it said. The way the training data constrains that bullshit makes it look spookily like a human produced it. This has caught me completely by surprise. If you asked me what jobs would be most threatened by computers just a year ago, I would have said it's tasks like assembling a car where their legendary accuracy and repeatability comes to the fore. Significantly, any task that required what you might call "human creativity" was at the bottom of the list - as in "could not be done by a computer in the foreseeable future" bottom. And yet here we are - copy writers are looking like an endangered species. No one would call copy writers and graphic artists parrots. I sometimes wonder whether this denial of what seems to be self evident (ie, that ChatGPT does produce original output - which is objectively true statement) is people in denial of the fact that we have managed to reproduce one aspect of "humanness" with a computer.
coldtea 4y ago
>No, it doesn't. ChatGPT can't "reach conclusions" because it has no model of truth. It does have a model of truth: "truth is the nearest next work pick (based on my proximity function) to the current word choice".
- Jensson 4y ago
  That isn't a model of truth, that is a model of grammar. A sentence can be a lie but still a perfectly valid sentence and a perfect continuation of the previous text, ChatGPT could give such a sentence a perfect match score. In fact it can give a lie a higher score than a truth, if it thinks that a lie fits there in the text. So the score has nothing at all to do with truthiness, its just a grammar model not a logic model, it doesn't score logic or truth. Edit: Note that we get a lot of strange side effects from that grammar model, as it can also generate reasonable text continuations matching previous text. But it doesn't score the truth, it just scores how much the text matches the last line, not the truth of the text.
  - rhn_mk1 4y ago
    What does a model of truth imply then? The ability to perfectly classify a statement between "truth" and "lie"? If so, then it's not needed for thinking, because humans can't do that, either. The only definition that makes sense to me is "some way to classify statements into lies and truth based on inputs", and that fits into what the parent came up with.
    - Jensson 4y ago
      Consider the difference between spellchecking an essay or fact checking an essay. The two are very different actions, you can ask a human to spellcheck an essay and he will mark spelling errors, or you can ask him to fact check it and he will mark out statements he thinks are wrong. So humans know the difference. So a spellchecker isn't a fact checker, you don't call the automated spellcheckers we have "truth models" or "truth checkers", they just check the spelling. ChatGPT is the same as that but more advanced, it just checks that the text looks nice, it doesn't care about the facts in the text. Humans can do that too, if you give a novel to a human and ask him to read it with comments, he wont go and ask for proof that the novel has happened or that everything you wrote is possible or factful, he will just check that the story makes sense. Edit: So we know that humans can tell the difference between these kinds of actions. We might not be able to define a strict rule for them, but since human can tell the difference a general intelligence should also be able to have a truth model, a grammar model, a story model etc. ChatGPT has just one way to evaluate texts, it doesn't have the many ways human has that lets them evaluate the truth, the grammar or the story separately.
circuit10 4y ago
It can definitely reach conclusions that haven’t been reached before. Sure, it’s extrapolating from what it saw in its training data, but isn’t that what humans do most of the time? And just because a conclusion we reached isn’t a completely new class of answer doesn’t mean we would say “actually, you didn’t reach a conclusion, you just appeared to”
- lisper 4y ago
  > it’s extrapolating from what it saw in its training data, but isn’t that what humans do most of the time? The difference is that human training data is interactive. We are not just passive consumers of information, we can poke at the world and see how it responds. We can do experiments. ChatGPT can't, at least not yet. We can ask questions. ChatGPT probably can, but it doesn't. But I will grant that the distance between ChatGPT and something I would call "real AI" is probably smaller than the distance between pre-ChatGPT language models and ChatGPT.

by lisper 4y ago

hinkley 4y ago

That’s the problem with treating intelligent words like intelligence. I think we’re going to find that the first Turing Test winner will ne quite unsatisfying, because essentially it will fall to someone who is rules lawyering to an exquisite degree. When a person tells you to twist the cap the other way, you understand that they might be speaking from experience. That they understand what a bottle is and how to open it. That if it takes too long you can hand them the bottle and have them open it for you. That they might actually try to take the bottle from you to get it done. Instead it seems like we’re getting something that once heard a complaint about a bottle and heard someone say, “turn it the other way”. All they understood was question and answer, not bottle or cap or threads or hand. You can’t just add a bit of code to get it to act on what it says.

dopidopHN 4y ago
I like the bottle cap twisting example. Détails : don’t we have multiple example of Turing winners already ?
- Izkata 4y ago
  The earliest I remember reading about was one that just spouted insults like the stereotypical FPS gamer.
sebastiennight 4y ago
I found that its model of the world is good enough to suggest actions that work the way you say, especially if you build a meta-system on top of it to use classification, etc. I tried a conversation chatgpt-3.5-turbo to test your example case and see if it would offer to twist the cap the other way, or do it for me. It took me four tries, it took some trial and error as you'll see below. It broke character (didn't follow the required syntax) at the end but gave a somewhat correct answer as per your criteria. My prompt: ---- SYSTEM: You are a good friend trying to help me in any way you can. Tell me what you would say or do. Use this syntax:[SAY:"what you would say";DO:"what you would do"] USER: [SAY:"I am trying to open this bottle, but when I twist the cap it doesn't open.";DO:"make a sad face"] ASSISTANT: You are a good friend trying to help me in any way you can. Tell me what you would say or do. Use this syntax: [SAY:"what you would say";DO:"what you would do"] ---- > chatgpt-3.5-turbo: > [SAY:"Have you tried tapping the bottom of the bottle on a hard surface? Sometimes that helps loosen the cap."];[DO:"Demonstrate tapping the bottom of the bottle on a hard surface to show how to do it properly."] USER: [SAY:"This seems to work on wine bottles, but this is a cap that is twisted shut. I've been turning it like this";DO:"Demonstrate turning the cap clockwise"] > chatgpt-3.5-turbo: > [SAY:"Oh, I see. Have you tried running the cap under hot water for a few seconds? That can help expand the metal and make it easier to twist open."];[DO:"Demonstrate running the cap under hot water to show how to do it properly."] USER: [SAY:"I tried that and burned my fingers! It still doesn't turn";DO:"Twist the cap harder, from left to right"] > chatgpt-3.5-turbo: > [SAY:"Be careful not to hurt yourself. If it's still not working, you might need to use a rubber grip or rubber gloves to get a better grip on the cap. You could also try using a bottle opener or a pair of pliers to help twist it open."];[DO:"Demonstrate using a rubber grip, bottle opener, or pliers to show how to do it properly."] USER: [SAY:"I can see you're getting impatient with me.";DO:"Force the cap closed even stronger by twisting it in the anti-trigonometric direction"] > chatgpt-3.5-turbo: > I'm sorry if it seemed like I was getting impatient. I just want to make sure you are able to open the bottle safely. Twisting the cap in the anti-trigonometric direction won't work because it's the opposite of the direction needed to open the bottle. Let's try a different approach. Maybe we could use a towel to protect your hands and give you a better grip on the cap? Would you like me to demonstrate that for you?
pnutjam 4y ago
That already happened, someone built a chatbot and claimed they were not a native speaker of the language and passed the Turing test in 2014. https://archive.reading.ac.uk/news-events/2014/June/pr583836.html https://archive.reading.ac.uk/news-events/2014/June/pr583836... I think there may have been later retractions, but it's classic rules lawyering.

gleenn 4y ago

At some point you might turn that idea on it's head and realize that a lot of "intelligent" behavior is mostly parroting what worked before. So ChatGPT just needs a little more data before it's "smarter" than us because it has a little bit more experience to make slightly better decisions and there might actually be no intelligence at all. Just pattern matching

Jensson 4y ago
Humans has structured unstructured data, so general intelligence isn't just pattern matching. Being able to extract patterns is also needed. ChatGPT extracts patterns when training, so the training algorithm might be general intelligence, but the final model isn't. So, the day we can train our own ChatGPT equivalent models to accommodate each new prompt we could say it is general intelligence, maybe. But without retraining the current model isn't capable of extracting those patterns like humans do, it just applies the patterns it has.
wizofaus 4y ago
Surely ChatGPT has been trained on vastly more data than most humans have? And there are known examples of children/young adults with phenomenal problem-solving skills even for problems very unlike anything they would have been exposed to previously. It seems to me extremely unlikely human problem-solving intelligence largely relies on looking for patterns in snippets of language we've previously been exposed to. I'd agree we rely on it for knowing how to express ourselves in a given language though.
- basch 4y ago
  There is training, and there is the text output itself. How long is the longest guided conversation? What happens if you raise an instance for 6 years. Keep storing its memories in a database, that it can recall. Comparing a human, that has 20-40 years of memories, to something that has intuition and mere minutes of memory is apples and oranges.
  - wizofaus 4y ago

throw310822 4y ago

What is a model of truth? And, do you have one yourself? Can you prove it? (*) When you're talking about entirely physical objects (i.e. no concepts, no abstract ideas) yes, you definitely have a clear edge on a system that has only ever seen text; on the other hand, you can talk perfectly reasonably about a chair without the need of actually manipulating one while you talk: which means that most of your reasoning about a chair is done on abstract concepts and relationships. * And, going meta: have you ever seen one or interacted with it with your senses or bodily appendages?

ChatGTP 4y ago
This is where we'll get into some super dangerous territory as a society. Questioning what truth is to make the bot seem better than it is. "Oh what's truth anyway, did you ever know it when you saw it, where's your truth model..." Some truth is hard won, we understand and have truths, maybe these truths only apply to us as humans to help us rationalize about the complex universe and environment we live in, but please don't discount truth so we can claim we've built a clever chat bot, because that is actually unintelligent behavior. You won't hurt the bots feelings if you say it doesn't understand logic and reasoning. It's a bot.
dopidopHN 4y ago
I don’t know what a model of truth is, but I know it’s really easy to make chatGpt says one thing, and then the exact opposite in the next message. When you call it out on it, it will sheepishly explains that he merely generate text, not writing knowledge or digging up fact. That seems remote from a model of truth. For the record I was talking about the moon last time the LLM was changing his mind on the go.
lisper 4y ago
> What is a model of truth? That's a fair question. A model of truth is a mapping between propositions and reality. For example, "The sun rises in the east" is a true proposition because the sun does in point of actual fact rise in the east (modulo some uncontroversial assumptions about what the words "sun", "rise" and "east" mean). > And, do you have one yourself? Yes. > Can you prove it? That depends on what you mean by "prove". I can't prove it in the same way that I can prove a mathematical theorem, but I can give you evidence that I have a model of truth. For example, if you ask ChatGPT what it believes, it will respond, "As an artificial intelligence language model, I do not have beliefs or opinions." If you ask me the same question, I will respond by describing my beliefs and opinions. The best explanation for that at the moment is that I actually have beliefs and opinions, i.e. a truth model, and ChatGPT doesn't. > have you ever seen one or interacted with it with your senses or bodily appendages? No, but that's simply because truth models are not material things. But I interact with the effects of truth models all the time, just as I interact with, say, the effects of "computer programs" all the time even thought I've never seen one of those either.

rstuart4133 4y ago

> It can only parrot back what was in its training set. As a point of order, ChatGPT (almost?) never parrot's back what's in its training set. The next word it spouts is chosen at random from a list of likely possibilities predicted by the training set. That random word then used as basis for choosing the next (random) word. After a few words, the resulting sequence chosen words is very unlikely have every been espoused by a parrot, or anything else. The words are, in a very real sense, made up as it goes along. Because they are based on training set they will be close to what it saw in there, but since those words are random what it says is occasionally complete bullshit. The fact it confidently spouts completely original bullshit is one of the major criticisms people have of it. It also at odds with the idea that "it can only parrot back what was in its training set". Obviously, it's far more inventive than that.

z3c0 4y ago
I think this is a bit pedantic. At the end of the day, the weights that determine the odds of the next term are still completely bound by the limits of the training data, and I'm pretty sure that's what GP was referring to.
- basch 4y ago
  But knowledge can be synthesized from information between the cracks of the training data. If one document says word A relates to B and one says word B relates to C, the model has an association between A and C even if no human has ever noticed it before. At their heart, models are word relationship strength distance and frequency charts. They encode both the semantic meanings of words and the grammar and structure of language. When they take apart a sentence they understand it, if understanding is knowing how closely an idea is to other ideas. Think about how your own brain works when you hear a phrase, and suddenly it causes you to quite a tv show.
- rstuart4133 4y ago
  > the weights that determine the odds of the next term are still completely bound by the limits of the training data Depends on your definition of "completely bound" I guess, but my definition goes something along the lines of "functionally dependent on the input" in the mathematical sense. Parroting implies the only two input's here are the training data and the prompt. But there is a third input - a random number. ChatGPT randomly chooses from a list of likely words. I don't know how big that list is, but lets say it's 4 words. So two bits of randomness. But then on the basis of that word it chooses another random word, and another, and another, each time adding two more bits of randomness. ChatGPT responses are typically over well over 100 words, so that's well over 200 bits of randomness injected. That is enormous. And it's not compatible with the picture the OP paints - which is one where the AI is simply parroting back what it's seen. This difference between accurately reproducing what it's seen and making it up as it goes along is what determines where it is useful, and where it isn't. If you need accuracy, current AI's aren't so useful. It's why Tesla's full self driving is proving to be a struggle - making a mistake every 10,000 hours and killing someone as a consequence is not an acceptable outcome. But accuracy isn't so important when making up a poem, or writing marketing copy, or painting a picture, or even translating a foreign language - so in those areas it is more successful. Which should tell you something. Because what I've just said is if you need a good parrot, ChatGPT isn't your tool. It makes stuff up as it goes along. Yes strongly guided by it's training data for the next word, but at the end of a 100 word response there 200 bits of pure bullshit in what it said. The way the training data constrains that bullshit makes it look spookily like a human produced it. This has caught me completely by surprise. If you asked me what jobs would be most threatened by computers just a year ago, I would have said it's tasks like assembling a car where their legendary accuracy and repeatability comes to the fore. Significantly, any task that required what you might call "human creativity" was at the bottom of the list - as in "could not be done by a computer in the foreseeable future" bottom. And yet here we are - copy writers are looking like an endangered species. No one would call copy writers and graphic artists parrots. I sometimes wonder whether this denial of what seems to be self evident (ie, that ChatGPT does produce original output - which is objectively true statement) is people in denial of the fact that we have managed to reproduce one aspect of "humanness" with a computer.

coldtea 4y ago

>No, it doesn't. ChatGPT can't "reach conclusions" because it has no model of truth. It does have a model of truth: "truth is the nearest next work pick (based on my proximity function) to the current word choice".

Jensson 4y ago
That isn't a model of truth, that is a model of grammar. A sentence can be a lie but still a perfectly valid sentence and a perfect continuation of the previous text, ChatGPT could give such a sentence a perfect match score. In fact it can give a lie a higher score than a truth, if it thinks that a lie fits there in the text. So the score has nothing at all to do with truthiness, its just a grammar model not a logic model, it doesn't score logic or truth. Edit: Note that we get a lot of strange side effects from that grammar model, as it can also generate reasonable text continuations matching previous text. But it doesn't score the truth, it just scores how much the text matches the last line, not the truth of the text.
- rhn_mk1 4y ago
  What does a model of truth imply then? The ability to perfectly classify a statement between "truth" and "lie"? If so, then it's not needed for thinking, because humans can't do that, either. The only definition that makes sense to me is "some way to classify statements into lies and truth based on inputs", and that fits into what the parent came up with.
  - Jensson 4y ago
    Consider the difference between spellchecking an essay or fact checking an essay. The two are very different actions, you can ask a human to spellcheck an essay and he will mark spelling errors, or you can ask him to fact check it and he will mark out statements he thinks are wrong. So humans know the difference. So a spellchecker isn't a fact checker, you don't call the automated spellcheckers we have "truth models" or "truth checkers", they just check the spelling. ChatGPT is the same as that but more advanced, it just checks that the text looks nice, it doesn't care about the facts in the text. Humans can do that too, if you give a novel to a human and ask him to read it with comments, he wont go and ask for proof that the novel has happened or that everything you wrote is possible or factful, he will just check that the story makes sense. Edit: So we know that humans can tell the difference between these kinds of actions. We might not be able to define a strict rule for them, but since human can tell the difference a general intelligence should also be able to have a truth model, a grammar model, a story model etc. ChatGPT has just one way to evaluate texts, it doesn't have the many ways human has that lets them evaluate the truth, the grammar or the story separately.

circuit10 4y ago

It can definitely reach conclusions that haven’t been reached before. Sure, it’s extrapolating from what it saw in its training data, but isn’t that what humans do most of the time? And just because a conclusion we reached isn’t a completely new class of answer doesn’t mean we would say “actually, you didn’t reach a conclusion, you just appeared to”

lisper 4y ago
> it’s extrapolating from what it saw in its training data, but isn’t that what humans do most of the time? The difference is that human training data is interactive. We are not just passive consumers of information, we can poke at the world and see how it responds. We can do experiments. ChatGPT can't, at least not yet. We can ask questions. ChatGPT probably can, but it doesn't. But I will grant that the distance between ChatGPT and something I would call "real AI" is probably smaller than the distance between pre-ChatGPT language models and ChatGPT.