5 ms·

it indeed understands you. A lot of people are just parroting the same thing over and over again saying it's just a probabilistic word generator. No, it's not,

it indeed understands you. A lot of people are just parroting the same thing over and over again saying it's just a probabilistic word generator. No, it's not, it's more then that. Take a look at this: https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ Read to the end. The beginning is trivial the ending is unequivocal: chatGPT understands you. I think a lot of people are just in denial. Because the last year there's been the same headlines over and over again and some people get a little too excited about the headlines and other armchair experts just try to temper the excitement with their "expert opinions" on LLMs that they read from popular articles. Then when something that's an actual game changer hits the scene (chatGPT) they completely miss it. chatGPT is different. From a technical perspective, it's simply an LLM with additional reinforcement training... BUT you can't deny the results are remarkable. If anything this much is clear to me: We are at a point where we can neither confirm or deny whether chatGPT represents some aspect of sentience. This is especially true given the fact that we don't even fully know what sentience is.

didntreadarticl 4y ago
I think about its 4000 token length. For the brief amount of time that it absorbs and processes those 4000 tokens, is there a glimmer of a hint of sentience? Like it is microscopically sentient for very short bursts and then resets back to zero.
- kilgnad 4y ago
  Does sentience need memory? I would say it's orthogonal. There are examples of people in the real world who only remember things for about 3 minutes before they lose it. They can't form any real memories. These people are still sentient despite lack of memory. See: https://www.damninteresting.com/living-in-the-moment/ https://www.damninteresting.com/living-in-the-moment/ If chatGPT was sentient, I would say it has nothing to do with the 4000 character limit. The 4000 character limit has more to do with it's ability to display evidence of "sentience".
  - didntreadarticl 4y ago
    I dont think of the 4000 tokens as its memory as such. Its more like the size of its thinking workspace
    - int_19h 4y ago
      It also functions as memory in practice, though. With some complex tasks that can be broken down into steps, it often makes a big difference if you tell GPT to summarize its "understanding" of the current state as it goes through those steps - by repeating a phrase, keeping it within the token window, it effectively "remembers" something.
LegionMammal978 4y ago
> Read to the end. The beginning is trivial the ending is unequivocal: chatGPT understands you. How does this necessarily and unequivocally follow from the blog post? All I see in it is a bunch of output formed by analogy: it has a general concept of what each command's output is kinda supposed to look like given the inputs (since it has a bajillion examples of each), and what an HTML or JSON document is kinda supposed to look like, and how free-form information tends to fit into these documents. I'll admit that this direct reasoning by analogy is impressive, simply for the fact that nothing else but humans can do it with such consistency, but it's a very long way off from the indirect reasoning I'd expect from a sentient entity.
- kilgnad 4y ago
  Honestly I seriously find it hard to believe someone can read it to the end without mentioning how it queried itself. You're just naming the trivial things that it did. In the end It fully imagined a bash shell, an imaginary internet, an imaginary chatGPT on the imaginary internet, then on the imaginary chatGPT it created a new imaginary bash shell. The level of recursive depth here indicates deep understanding and situational awareness of what it is being asked. It demonstrates awareness of what "itself" is and what "itself" is capable of doing. I'm not saying it's sentient. But it MUST understand your query in order to produce the output show in the article. That much is obvious. Also it's not clear what you mean by reasoning by analogy or indirect reasoning.
  - LegionMammal978 4y ago
    > In the end It fully imagined a bash shell, an imaginary internet, an imaginary chatGPT on the imaginary internet, then on the imaginary chatGPT it created a new imaginary bash shell. In the general case, a shell is merely a particular prompt-response format with special verbs; the internet is merely a mapping from URLs to HTML and JSON documents; those document formats are merely particular facades for presenting information; and a "large language model" is merely something that answers free-form questions. > The level of recursive depth here indicates deep understanding and situational awareness of what it is being asked. It demonstrates awareness of what "itself" is and what "itself" is capable of doing. Uh, what? Why does that output require self-awareness? First, it's requested to produce the source of a document "https://chat.openai.com/chat https://chat.openai.com/chat". What might be behind such a URL? OpenAI Chat, presumably! And OpenAI is well known to create large language models, so a Chat feature is likely a large language model the user can chat with. Thus it invents "Assistant", and puts the description into the facade of a typical HTML document. Then, it starts getting prompted with POST requests for the same URL, and it knows from the context of its previous output that the URL is associated with an OpenAI chatbot. So all that is left is to follow a regular question-answer format (since that's what large language models are supposed to do) and slap it into a JSON facade. > But it MUST understand your query in order to produce the output show in the article. That much is obvious. I'm saying that it "understands" your query only insofar as its words can be tied to the web of associations it's memorized. The impressive part (to me) is that some of its concepts can act as facades for other concepts: it can insert arbitrary information into an HTML document, a poem, a shell session, a five-paragraph essay, etc. All of that can be achieved by knowing which concepts are directly associated with which other concepts, or patterns of writing. This is the reasoning by analogy that I refer to: if it knows what a poem about animals might look like, and it can imagine what kinds of qualities space ducks might possess, then it can transfer the pattern to create a poem about space ducks. But none of this shows that it can relate ideas in ways more complex than the superficial, and follow the underlying patterns that don't immediately fall out from the syntax. For instance, it's probably been trained on millions of algebra problems, but in my experience it still tends to produce outputs that look vaguely plausible but are mathematically nonsensical. If it remembers a common method that looks kinda right, then it will always prefer that to an uncommon method. I mean, it's not utterly impossible that GPT-4 comes along and humbles all the naysayers like myself with its frightening powers of intellect, but I won't be holding my breath just yet.

didntreadarticl 4y ago

I think about its 4000 token length. For the brief amount of time that it absorbs and processes those 4000 tokens, is there a glimmer of a hint of sentience? Like it is microscopically sentient for very short bursts and then resets back to zero.

kilgnad 4y ago
Does sentience need memory? I would say it's orthogonal. There are examples of people in the real world who only remember things for about 3 minutes before they lose it. They can't form any real memories. These people are still sentient despite lack of memory. See: https://www.damninteresting.com/living-in-the-moment/ https://www.damninteresting.com/living-in-the-moment/ If chatGPT was sentient, I would say it has nothing to do with the 4000 character limit. The 4000 character limit has more to do with it's ability to display evidence of "sentience".
- didntreadarticl 4y ago
  I dont think of the 4000 tokens as its memory as such. Its more like the size of its thinking workspace
  - int_19h 4y ago
    It also functions as memory in practice, though. With some complex tasks that can be broken down into steps, it often makes a big difference if you tell GPT to summarize its "understanding" of the current state as it goes through those steps - by repeating a phrase, keeping it within the token window, it effectively "remembers" something.

LegionMammal978 4y ago

> Read to the end. The beginning is trivial the ending is unequivocal: chatGPT understands you. How does this necessarily and unequivocally follow from the blog post? All I see in it is a bunch of output formed by analogy: it has a general concept of what each command's output is kinda supposed to look like given the inputs (since it has a bajillion examples of each), and what an HTML or JSON document is kinda supposed to look like, and how free-form information tends to fit into these documents. I'll admit that this direct reasoning by analogy is impressive, simply for the fact that nothing else but humans can do it with such consistency, but it's a very long way off from the indirect reasoning I'd expect from a sentient entity.

kilgnad 4y ago
Honestly I seriously find it hard to believe someone can read it to the end without mentioning how it queried itself. You're just naming the trivial things that it did. In the end It fully imagined a bash shell, an imaginary internet, an imaginary chatGPT on the imaginary internet, then on the imaginary chatGPT it created a new imaginary bash shell. The level of recursive depth here indicates deep understanding and situational awareness of what it is being asked. It demonstrates awareness of what "itself" is and what "itself" is capable of doing. I'm not saying it's sentient. But it MUST understand your query in order to produce the output show in the article. That much is obvious. Also it's not clear what you mean by reasoning by analogy or indirect reasoning.
- LegionMammal978 4y ago
  > In the end It fully imagined a bash shell, an imaginary internet, an imaginary chatGPT on the imaginary internet, then on the imaginary chatGPT it created a new imaginary bash shell. In the general case, a shell is merely a particular prompt-response format with special verbs; the internet is merely a mapping from URLs to HTML and JSON documents; those document formats are merely particular facades for presenting information; and a "large language model" is merely something that answers free-form questions. > The level of recursive depth here indicates deep understanding and situational awareness of what it is being asked. It demonstrates awareness of what "itself" is and what "itself" is capable of doing. Uh, what? Why does that output require self-awareness? First, it's requested to produce the source of a document "https://chat.openai.com/chat https://chat.openai.com/chat". What might be behind such a URL? OpenAI Chat, presumably! And OpenAI is well known to create large language models, so a Chat feature is likely a large language model the user can chat with. Thus it invents "Assistant", and puts the description into the facade of a typical HTML document. Then, it starts getting prompted with POST requests for the same URL, and it knows from the context of its previous output that the URL is associated with an OpenAI chatbot. So all that is left is to follow a regular question-answer format (since that's what large language models are supposed to do) and slap it into a JSON facade. > But it MUST understand your query in order to produce the output show in the article. That much is obvious. I'm saying that it "understands" your query only insofar as its words can be tied to the web of associations it's memorized. The impressive part (to me) is that some of its concepts can act as facades for other concepts: it can insert arbitrary information into an HTML document, a poem, a shell session, a five-paragraph essay, etc. All of that can be achieved by knowing which concepts are directly associated with which other concepts, or patterns of writing. This is the reasoning by analogy that I refer to: if it knows what a poem about animals might look like, and it can imagine what kinds of qualities space ducks might possess, then it can transfer the pattern to create a poem about space ducks. But none of this shows that it can relate ideas in ways more complex than the superficial, and follow the underlying patterns that don't immediately fall out from the syntax. For instance, it's probably been trained on millions of algebra problems, but in my experience it still tends to produce outputs that look vaguely plausible but are mathematically nonsensical. If it remembers a common method that looks kinda right, then it will always prefer that to an uncommon method. I mean, it's not utterly impossible that GPT-4 comes along and humbles all the naysayers like myself with its frightening powers of intellect, but I won't be holding my breath just yet.