3 ms·

> convincing simulations of conscious thought entirely via predicting language tokens means something profound about the nature of language itself. > there's a

> convincing simulations of conscious thought entirely via predicting language tokens means something profound about the nature of language itself. > there's also something big about us waiting there under the surface. I don't believe so. In "The Origins of Knowledge and Imagination" by Jacob Brownoski, he argues that human language have four unique characteristics: - We can separate information (data of what being described) from emotional content (how we're supposed to react). There's no longer a bijection between communication and action. - We can extend the time reference of the communication content. We talk about the past, we plan for the future. - We can refer to ourselves. So we examine what we've done and iterate over it until we fix the errors. We can see ourselves doing the action without actually doing it. - We can rearrange units of languages to have different meanings. The same words can have different meanings based on their order. So meaning depends not only on the words, but their sequence. And that goes from words to phrases to sequence of dialogs. The fourth point is the most important. LLMs by predicting languages tokens can give use the most common order for a particular context. And because we don't have that many words, their orders can be extracted from books and other written content. But then they fail for the higher levels, mostly because that's when everything get unique. As for the third point, by observing ourselves, our communication is constantly being based on reality, which grounds it in truth. And because we can extend the reference it's based on, that leads us to observe changes and model laws. The first point allows us to separate what things are from what we should do or feel based on their existence and absence. Instead of the LLMs fooling us, it's more us fooling ourselves, because by recognizing meaning in sentences, we try to extract meanings for longer sequences of text where there aren't any. Why? Because there is no "I" that has done the job of extracting information and using language to transmit it (while still cognizant of the imperfection of natural languages). LLMs are lossy compressions of ideas. Only the smallest survives and then it generates much more false ones.

justinjlynn 2y ago
Are you certain that you're not playing with words to arrive at a predetermined conclusion? What is this "I" to which you're referring and how can you demonstrate that "I" does not or cannot exist within systems such as these? Further, if you are to find something which qualifies as an "I" elsewhere, what makes that elsewhere fundamentally different and therefore capable of supporting and being an "I" and is that elsewhere such simply by definition or in and of itself? Further, if the language usage is indistinguishable from the language usage of an "I", is the difference of source meaningful? If so, why?
- skydhash 2y ago
  The "I" is stemmed from the theory of the mind. We can only access our own mind and thus has no way to infer the thoughts of other. So we observe them and infer based on our own patterns. In a sense, we assume that others have the same mechanism that we possess, and thus we engage in interactions with them. So far, there is no demonstration of reasoning within systems such as these, it's all simulation of the communication channel themselves. > Further, if the language usage is indistinguishable from the language usage of an "I", is the difference of source meaningful? Is it indistinguishable? The first thing we look for in communication is consistency so that we can examine for intent. And this is after we determine the other party. Because we know the intent is not ours. But what I've seen of prompt engineering is that the communication intent always come from the person, not the models. Then it goes on to find the most likely continuation of this intent (based on the model training) and then it quickly become an echo chamber. It's search in lexical space and you can see the limits when it became a oscillating loop between the same set of reply. Because there's no "I don't know" damping.
brianush1 2y ago
Why does there need to be an "I" that uses language to transmit information? Language itself encodes information. I can read a piece of text and gain something from it. Where the text came from is irrelevant.
- skydhash 2y ago
  > Language itself encodes information. Which it does in a lossy manner. Information is independent from language. The more complex the information, the more language fails. Which is why there is so many mediums for communication. Language has three main components: the symbols, the grammar, and the dictionary. The first refers to the tokens of our vocabulary, the second to the rules to arrange these tokens, and the third describes the relation of the tokens to the things they represent. The relation between the three is interdependent. We name new things we encounter, creating entry in the dictionary, we figure the rules that governs these things, and the relation to other things encountered previously. And thus, we can issue statements. We can also name these statements and it continues recursively. But each of us possess its own copy of these stuff with its own variations. What you gain from what I said may be different from what I intended to transmit. And what I intended to transmit may be a poor description of the things itself. So flawed interpretation, flawed description, and flawed transmission result in flawed understanding. To correct it, you need to be in presence of the thing itself. Missing that, you strive to establish the tokens, the grammar, and the dictionary of the person that have written the text. In LLMs, the dictionary is missing. The token "snow" has no relation to the thing we call snow. But because it's often placed near other tokens like "ice", "freeze", etc,... Then a rule emerges (embedding?) that these things must be related to each other. In what way it does not know. But if we apply the data collected in a statistical manner, we can arrange these tokens and the result will probably be correct. But there's still a non-zero chance that the generated statement is meaningless as there's no foundation rule that drives it. So there's only tokens. And rules derived from analyzing texts (which lack the foundation rules that comes from being in the real world). All of these to say the act of learning is either observing the real world and figure how it works. Or read from someone that has done the observing and has written his interpretation, then go outside and confirm it. Barring that, we reconstruct the life of this person so that we can correct the imperfection of languages. With LLMs, there's no way to correct as the statement themselves are not truthful. they can just be accidentally be right.

justinjlynn 2y ago

Are you certain that you're not playing with words to arrive at a predetermined conclusion? What is this "I" to which you're referring and how can you demonstrate that "I" does not or cannot exist within systems such as these? Further, if you are to find something which qualifies as an "I" elsewhere, what makes that elsewhere fundamentally different and therefore capable of supporting and being an "I" and is that elsewhere such simply by definition or in and of itself? Further, if the language usage is indistinguishable from the language usage of an "I", is the difference of source meaningful? If so, why?

skydhash 2y ago
The "I" is stemmed from the theory of the mind. We can only access our own mind and thus has no way to infer the thoughts of other. So we observe them and infer based on our own patterns. In a sense, we assume that others have the same mechanism that we possess, and thus we engage in interactions with them. So far, there is no demonstration of reasoning within systems such as these, it's all simulation of the communication channel themselves. > Further, if the language usage is indistinguishable from the language usage of an "I", is the difference of source meaningful? Is it indistinguishable? The first thing we look for in communication is consistency so that we can examine for intent. And this is after we determine the other party. Because we know the intent is not ours. But what I've seen of prompt engineering is that the communication intent always come from the person, not the models. Then it goes on to find the most likely continuation of this intent (based on the model training) and then it quickly become an echo chamber. It's search in lexical space and you can see the limits when it became a oscillating loop between the same set of reply. Because there's no "I don't know" damping.

brianush1 2y ago

Why does there need to be an "I" that uses language to transmit information? Language itself encodes information. I can read a piece of text and gain something from it. Where the text came from is irrelevant.

skydhash 2y ago
> Language itself encodes information. Which it does in a lossy manner. Information is independent from language. The more complex the information, the more language fails. Which is why there is so many mediums for communication. Language has three main components: the symbols, the grammar, and the dictionary. The first refers to the tokens of our vocabulary, the second to the rules to arrange these tokens, and the third describes the relation of the tokens to the things they represent. The relation between the three is interdependent. We name new things we encounter, creating entry in the dictionary, we figure the rules that governs these things, and the relation to other things encountered previously. And thus, we can issue statements. We can also name these statements and it continues recursively. But each of us possess its own copy of these stuff with its own variations. What you gain from what I said may be different from what I intended to transmit. And what I intended to transmit may be a poor description of the things itself. So flawed interpretation, flawed description, and flawed transmission result in flawed understanding. To correct it, you need to be in presence of the thing itself. Missing that, you strive to establish the tokens, the grammar, and the dictionary of the person that have written the text. In LLMs, the dictionary is missing. The token "snow" has no relation to the thing we call snow. But because it's often placed near other tokens like "ice", "freeze", etc,... Then a rule emerges (embedding?) that these things must be related to each other. In what way it does not know. But if we apply the data collected in a statistical manner, we can arrange these tokens and the result will probably be correct. But there's still a non-zero chance that the generated statement is meaningless as there's no foundation rule that drives it. So there's only tokens. And rules derived from analyzing texts (which lack the foundation rules that comes from being in the real world). All of these to say the act of learning is either observing the real world and figure how it works. Or read from someone that has done the observing and has written his interpretation, then go outside and confirm it. Barring that, we reconstruct the life of this person so that we can correct the imperfection of languages. With LLMs, there's no way to correct as the statement themselves are not truthful. they can just be accidentally be right.