3 ms·

As other posters have pointed, the core of a LLM is a pure function, which computes a token probability distribution from an input context. An automaton, which

As other posters have pointed, the core of a LLM is a pure function, which computes a token probability distribution from an input context. An automaton, which can chat with you or write a program, is built externally to the LLM function, by storing the context and making it change, depending on the output of the LLM function. However, the LLM pure function is exceedingly complex so it is essentially unpredictable what will it produce for a given input context. So one may have to treat the LLM function as a black box and explore the huge space of the input contexts by varying them in various ways, inclusive by using words that express human emotions, and monitor how the output of the function changes, i.e. how the LLM "reacts" to the expressed emotions. A "reaction" similar to that of a human is to be expected, because human emotions were expressed in the training texts, followed by reactions of humans to those emotions, and the LLM function will change its output token probability function in a manner mimicking the behavior of the humans from the training texts. Even functions that are many orders of magnitude simpler than LLMs are still to complex for anyone to understand how their output changes when you move through the space of the possible input arguments. The most essential part of cryptography is the existence of a class of functions which were named by Claude Shannon "good mixing transformations". All the important cryptographic primitives, e.g. block cipher functions or one-way hash functions, are built from such "good mixing transformations". The impossibility of breaking a cryptographic system with secret keys is based on the assumption that it is impossible to predict how the output of such a "good mixing transformation" changes when its input is changed. All such "good mixing transformations" have the so-called avalanche property, which means that even if you change a single input bit, any of the output bits may change with a probability of exactly 50%, so it is unpredictable for any output bit whether it will change, or not. If such simple functions, e.g. with 128 input bits and 128 output bits, can have a completely unpredictable behavior, then it is not surprising that LLM functions that may have an input of up to a few million bits (the length of the context window) are completely unpredictable and you can just observe their behavior when given various kinds of contexts and search for empirical approximate rules describing the behavior.

threethirtytwo 7mo ago
If you read carefully my point is not about the external behavior of the LLM. It is the black box aspect of the LLM. The sheer complexity of the pure function is not something we can understand even though the high level structure is a feed forward network the core algorithm is in actuality encoded by weights. Yes there are complex functions besides LLMs that we don’t understand but those functions usually aren’t compelling because the LLM, unlike those other functions has output that implicates reasoning and emotions. The problem is we can’t understand what’s going on under the hood so we don’t know either way. This is what I mean by stupidity. You completely missed the point, and you’re also operating under the assumption that the human brain is also not following a similar deterministic pathway. You hold humanity and biological intelligence in such high regard that you cannot even imagine that all of physics implies that human intelligence is mechanical. So the emotions you feel are under a black box same as the LLM and you apply you biased assumptions in a singular direction assuming your emotions are not deterministic and that LLM emotions are fake but that reasoning has no basis.
- adrian_b 7mo ago
  I might have not explained it clearly, but my position is not what you have said. I agree with you that in principle it will be possible to design an artificial automaton that will have something equivalent with human emotions (though I do not believe that it makes sense to attempt to design such a system). However, I do not believe that an LLM is such a thing, because the training algorithm just ensures that an LLM will mimic whatever is recorded in the training inputs, with or without human emotions in them. There is nothing in the structure of an LLM that can generate emotions by itself. If you train an LLM, for example, only on programs without comments or only on mathematical formulae, it will never display any kind of emotions. Regarding human emotions, they are recorded in a static way in a book or in a movie, but we do not say that the book or the movie has human emotions itself. With an LLM, the behavior is much more complex, because it does not just play a sequential recording of human emotions, but it can combine them in various way, while responding to various stimuli that are similar to those that had elicited emotions in the training texts. But regardless of this behavioral complexity, the human emotions are not generated somehow intrinsically by the LLM, but they correspond to those previously recorded in the texts used for training, so they just mimic humans.

threethirtytwo 7mo ago

If you read carefully my point is not about the external behavior of the LLM. It is the black box aspect of the LLM. The sheer complexity of the pure function is not something we can understand even though the high level structure is a feed forward network the core algorithm is in actuality encoded by weights. Yes there are complex functions besides LLMs that we don’t understand but those functions usually aren’t compelling because the LLM, unlike those other functions has output that implicates reasoning and emotions. The problem is we can’t understand what’s going on under the hood so we don’t know either way. This is what I mean by stupidity. You completely missed the point, and you’re also operating under the assumption that the human brain is also not following a similar deterministic pathway. You hold humanity and biological intelligence in such high regard that you cannot even imagine that all of physics implies that human intelligence is mechanical. So the emotions you feel are under a black box same as the LLM and you apply you biased assumptions in a singular direction assuming your emotions are not deterministic and that LLM emotions are fake but that reasoning has no basis.

adrian_b 7mo ago
I might have not explained it clearly, but my position is not what you have said. I agree with you that in principle it will be possible to design an artificial automaton that will have something equivalent with human emotions (though I do not believe that it makes sense to attempt to design such a system). However, I do not believe that an LLM is such a thing, because the training algorithm just ensures that an LLM will mimic whatever is recorded in the training inputs, with or without human emotions in them. There is nothing in the structure of an LLM that can generate emotions by itself. If you train an LLM, for example, only on programs without comments or only on mathematical formulae, it will never display any kind of emotions. Regarding human emotions, they are recorded in a static way in a book or in a movie, but we do not say that the book or the movie has human emotions itself. With an LLM, the behavior is much more complex, because it does not just play a sequential recording of human emotions, but it can combine them in various way, while responding to various stimuli that are similar to those that had elicited emotions in the training texts. But regardless of this behavioral complexity, the human emotions are not generated somehow intrinsically by the LLM, but they correspond to those previously recorded in the texts used for training, so they just mimic humans.