5 ms·

That's not even the important point. ChatGPT can't do anything. It exhibits behavior that was already encapsulated into the semantics of language itself. The

That's not even the important point. ChatGPT can't do anything. It exhibits behavior that was already encapsulated into the semantics of language itself. The only behavior ChatGPT has is to generate semantic continuations from its implicit language model. Every other behavior is a feature of language, not of ChatGPT. Even if ChatGPT could exhibit a passing exam, that would be the feature of carefully curated language in its dataset; not a feature of ChatGPT itself.

l33t233372 4y ago
I really don’t understand the distinction you’re getting at between what ChatGPT is and what ChatGPT does.
pharmakom 4y ago
it can output procedures for other systems that do something, e.g. code.
elcomet 4y ago
If a human passed the bar exam, it is because he trained for it and the knowledge is encoded in his brain. The distinction you're trying to make doesn't exist. ChatGPT is what it can do.
- thomastjeffery 4y ago
  The distinction may not be obvious, but it is there. The reason it is not obvious is that nearly everything you have heard about ChatGPT itself is wrong. The first thing people do to explain what ChatGPT is and does is to personify it. From then on, they are talking about ChatGPT personified, and not ChatGPT as it literally exists. The second thing people do is draw conclusions about the nature and behavior of ChatGPT itself from the narrative they are telling about ChatGPT personified. It's a case of mistaken identity. ChatGPT has a "brain", but the context that "brain" interacts with is semantics not symbolics. A human, when answering a question, interprets the symbols present in the language, then considers them logically. Finally, they formulate an answer, and express that answer with more symbols. ChatGPT does none of that. ChatGPT doesn't even know what sentences, punctuation, or even words are. The only subjects ChatGPT has in mind are short groups of characters: the tokens from the lexical analysis step. ChatGPT reads those tokens (groups of characters) in order, and generates an implicit model from them. That model is like a map: each token is a feature in the landscape. When ChatGPT gets a prompt, it tokenizes it, then checks the map for the closest match. Then it starts at that location, and steps forward, writing out what it sees along the way. That's everything that "ChatGPT as it literally exists" can do. So where does all the behavior come from? It's the content in the map. It's in language itself. ChatGPT's behavior is limited to interacting with that map, but the effect of interacting with that map is where we get all the interesting behavior. Language does not simply encode data: it also encodes instructions and logical relationships. By simply walking through text and feeling the semantic landscape, ChatGPT exhibits the behavior that was already encoded into the symbolic meaning of that text. It accomplished this implicitly without ever defining the meaning of any symbol. It doesn't even know what a symbol is in the first place! So when ChatGPT exhibits the behavior of a person writing correct answers to an exam, it is not behaving like a person at all. It's not interpreting the questions or finding the answers. Instead, it is simply filling the hole in the story with the semantic landscape it sees nearby. If the result is to place answer after question, that is because that data is already present in the training text that ChatGPT was modeled around. Because of this distinction, we can have a much better understanding of what ChatGPT is and isn't capable of. Because language itself holds the features of truth and lie, mistake and success, elegance and verbosity, love and hate, logic and fallacy, defined and abstract, ambiguous and unambiguous, etc. all equal, ChatGPT must rely on the implementation of language - what was written in the first place - to exhibit behaviors we want it to exhibit. But there is a critical flaw in that. Language allows, and even depends on, ambiguity. The context that resolves ambiguity can exist in many semantic shapes, so a model cannot be guaranteed to choose the semantic content that contains the disambiguation. We haven't solved the context dependence problem of natural language. We have only moved it. ChatGPT's success is dependent entirely on the content it is given. It cannot change its behavior to improve that system.
vidarh 4y ago
How do we prove that there is a meaningful distinction between a sufficiently advanced "feature of carefully curated language in [an entity's] dataset" and general intelligence? Is there such a meaningful distinction?
wazoox 4y ago
That's not exactly true. What you describe is a Markov chain, not LLM. The LLMs use neural networks to extract information from the language itself, and make decisions based on this stored model. The model is built from the language, not reality, but the model can integrate information that isn't present in the language, but on the meanings that were conveyed by training data.
deleted 4y ago
[deleted]

l33t233372 4y ago

I really don’t understand the distinction you’re getting at between what ChatGPT is and what ChatGPT does.

pharmakom 4y ago

it can output procedures for other systems that do something, e.g. code.

elcomet 4y ago

If a human passed the bar exam, it is because he trained for it and the knowledge is encoded in his brain. The distinction you're trying to make doesn't exist. ChatGPT is what it can do.

thomastjeffery 4y ago
The distinction may not be obvious, but it is there. The reason it is not obvious is that nearly everything you have heard about ChatGPT itself is wrong. The first thing people do to explain what ChatGPT is and does is to personify it. From then on, they are talking about ChatGPT personified, and not ChatGPT as it literally exists. The second thing people do is draw conclusions about the nature and behavior of ChatGPT itself from the narrative they are telling about ChatGPT personified. It's a case of mistaken identity. ChatGPT has a "brain", but the context that "brain" interacts with is semantics not symbolics. A human, when answering a question, interprets the symbols present in the language, then considers them logically. Finally, they formulate an answer, and express that answer with more symbols. ChatGPT does none of that. ChatGPT doesn't even know what sentences, punctuation, or even words are. The only subjects ChatGPT has in mind are short groups of characters: the tokens from the lexical analysis step. ChatGPT reads those tokens (groups of characters) in order, and generates an implicit model from them. That model is like a map: each token is a feature in the landscape. When ChatGPT gets a prompt, it tokenizes it, then checks the map for the closest match. Then it starts at that location, and steps forward, writing out what it sees along the way. That's everything that "ChatGPT as it literally exists" can do. So where does all the behavior come from? It's the content in the map. It's in language itself. ChatGPT's behavior is limited to interacting with that map, but the effect of interacting with that map is where we get all the interesting behavior. Language does not simply encode data: it also encodes instructions and logical relationships. By simply walking through text and feeling the semantic landscape, ChatGPT exhibits the behavior that was already encoded into the symbolic meaning of that text. It accomplished this implicitly without ever defining the meaning of any symbol. It doesn't even know what a symbol is in the first place! So when ChatGPT exhibits the behavior of a person writing correct answers to an exam, it is not behaving like a person at all. It's not interpreting the questions or finding the answers. Instead, it is simply filling the hole in the story with the semantic landscape it sees nearby. If the result is to place answer after question, that is because that data is already present in the training text that ChatGPT was modeled around. Because of this distinction, we can have a much better understanding of what ChatGPT is and isn't capable of. Because language itself holds the features of truth and lie, mistake and success, elegance and verbosity, love and hate, logic and fallacy, defined and abstract, ambiguous and unambiguous, etc. all equal, ChatGPT must rely on the implementation of language - what was written in the first place - to exhibit behaviors we want it to exhibit. But there is a critical flaw in that. Language allows, and even depends on, ambiguity. The context that resolves ambiguity can exist in many semantic shapes, so a model cannot be guaranteed to choose the semantic content that contains the disambiguation. We haven't solved the context dependence problem of natural language. We have only moved it. ChatGPT's success is dependent entirely on the content it is given. It cannot change its behavior to improve that system.

vidarh 4y ago

How do we prove that there is a meaningful distinction between a sufficiently advanced "feature of carefully curated language in [an entity's] dataset" and general intelligence? Is there such a meaningful distinction?

wazoox 4y ago

That's not exactly true. What you describe is a Markov chain, not LLM. The LLMs use neural networks to extract information from the language itself, and make decisions based on this stored model. The model is built from the language, not reality, but the model can integrate information that isn't present in the language, but on the meanings that were conveyed by training data.

deleted 4y ago

[deleted]