4 ms·

Whenever I come to HN I see a bunch of people say LLMs are just next token predictors and they completely understand LLMs. And almost every one of these people

Whenever I come to HN I see a bunch of people say LLMs are just next token predictors and they completely understand LLMs. And almost every one of these people are so utterly self assured to the point of total confidence because they read and understand what transformers do. Then I watch videos like this straight from the source trying to understand LLMs like a black box and even considering the possibility that LLMs have emotions. How does such a person reconcile with being utterly wrong? I used to think HN was full of more intelligent people but it’s becoming more and more obvious that HNers are pretty average or even below.

big_toast 7mo ago
One day I realized I needed to make sure I'm voting on quality stories/comments. I wonder if there was a call to vote substantively and often, if that might change the SNR. The guidelines encourage substantive comments, but maybe voters are part of the solution too. Kinda like having a strong reward model for training LLMs and avoiding reward hacking or other undesirable behavior.
- threethirtytwo 7mo ago
  if voters are stupid then it doesn't really help. I think what's happening is reality is asserting itself too hard that people can't be so stupid anymore.
qaadika 7mo ago
I'm kinda one of those who believes they 'completely' understand LLMs. But I've also developed my understanding of them such that the internal mechanisms of the transformer, or really any future development in the space based on neural networks and machine learning is irrelevant. 1. A string of unicode characters is converted into an array of integers values (tokens) and input to a black box of choice. 2. The black box takes in the input, does its magic, and returns an output as an array of integer values. 3. The returned output is converted into a string of unicode characters and given to the user, or inserted in a code file, or whatever. At no point does the black box "read" the input in any way analogous to how a human reads. Where people get "The AIs have emotions!!!" from returning an array of integers values is beyond me. It's definitely more complicated than "next token predictor", but it really is as simple as "Make words look like numbers, numbers go in, numbers come out, we make the numbers look like words."
- threethirtytwo 7mo ago
  Yeah nothing personal but my claim here is you’re not smart. The next token predictor aspect is something anyone can understand… the transformer is not quantum physics. Like look at what you wrote. You called it black box magic and in the same post you claim you understand LLMs. How the heck can you understand and call it a black box at the same time? The level of mental gymnastics and stupidity is through the roof. Clearly the majority of the utilitarian nature of the LLM is within the whole section you just waved away as “black box”. > Where people get "The AIs have emotions!!!" from returning an array of integers values is beyond me Let me spell it out for you. Those integers can be translated to the exact same language humans use when they feel identical emotions. So those people claim that the “black box” feels the emotions because what they observe is identical to what they observe in a human. The LLM can claim it feels emotions just like a human can claim the same thing. We assume humans feel emotions based off of this evidence but we don’t apply that logic to LLMs? The truth of the matter is we don’t actually know and it’s equally dumb to claim that you know LLMs feel emotions to claiming that they dont feel emotions. You have to be pretty stupid to not realize this is where they are coming from so there’s an aspect of you lying to yourself here because I don’t think you’re that stupid.
  - adrian_b 7mo ago
    Of course LLMs display human emotions, if they have been trained on texts that have recorded humans displaying human emotions. With an input context that contains words that excite certain human emotions, the output of the core LLM function will generate a token probability distribution that is representative for the human emotions displayed by humans in the training texts. This is something expected and non-sensational. An LLM mimics the human behavior that was recorded in the training texts, much in the same way as a photographic image of a human face mimics the appearance of that human face. A photographic image is designed to reproduce the light field created by a face that reflects the ambient light, a LLM is created to reproduce the typical conversational behavior that was recorded in the training texts. Depending on how it was trained, one should expect a LLM to be affected by the choice of words used in the input in a similar way how a human would be affected. However, that does not mean that a LLM that shows signs of emotional distress feels some pain because of that. A LLM is designed for mimicry and it does not feel more pain or more happiness than a photograph of a wound feels pain from the wound or a photograph of a smiley face feels happiness. The fact that the current LLMs do not actually feel the human emotions that they may be able to mimic in an accurate way, does not mean that you could not build a robot which would have some built-in mechanisms for feeling pain and various emotions, which could be made to have similar functions like in an animal, serving a functional purpose and not being used for mimicry. However, for now it does not make any sense to attempt to do such a thing, because in a deterministic program there are better ways to ensure that a robot is "loyal" to its owner and acts in self-preservation when possible.
adrian_b 7mo ago
As other posters have pointed, the core of a LLM is a pure function, which computes a token probability distribution from an input context. An automaton, which can chat with you or write a program, is built externally to the LLM function, by storing the context and making it change, depending on the output of the LLM function. However, the LLM pure function is exceedingly complex so it is essentially unpredictable what will it produce for a given input context. So one may have to treat the LLM function as a black box and explore the huge space of the input contexts by varying them in various ways, inclusive by using words that express human emotions, and monitor how the output of the function changes, i.e. how the LLM "reacts" to the expressed emotions. A "reaction" similar to that of a human is to be expected, because human emotions were expressed in the training texts, followed by reactions of humans to those emotions, and the LLM function will change its output token probability function in a manner mimicking the behavior of the humans from the training texts. Even functions that are many orders of magnitude simpler than LLMs are still to complex for anyone to understand how their output changes when you move through the space of the possible input arguments. The most essential part of cryptography is the existence of a class of functions which were named by Claude Shannon "good mixing transformations". All the important cryptographic primitives, e.g. block cipher functions or one-way hash functions, are built from such "good mixing transformations". The impossibility of breaking a cryptographic system with secret keys is based on the assumption that it is impossible to predict how the output of such a "good mixing transformation" changes when its input is changed. All such "good mixing transformations" have the so-called avalanche property, which means that even if you change a single input bit, any of the output bits may change with a probability of exactly 50%, so it is unpredictable for any output bit whether it will change, or not. If such simple functions, e.g. with 128 input bits and 128 output bits, can have a completely unpredictable behavior, then it is not surprising that LLM functions that may have an input of up to a few million bits (the length of the context window) are completely unpredictable and you can just observe their behavior when given various kinds of contexts and search for empirical approximate rules describing the behavior.
- threethirtytwo 7mo ago
  If you read carefully my point is not about the external behavior of the LLM. It is the black box aspect of the LLM. The sheer complexity of the pure function is not something we can understand even though the high level structure is a feed forward network the core algorithm is in actuality encoded by weights. Yes there are complex functions besides LLMs that we don’t understand but those functions usually aren’t compelling because the LLM, unlike those other functions has output that implicates reasoning and emotions. The problem is we can’t understand what’s going on under the hood so we don’t know either way. This is what I mean by stupidity. You completely missed the point, and you’re also operating under the assumption that the human brain is also not following a similar deterministic pathway. You hold humanity and biological intelligence in such high regard that you cannot even imagine that all of physics implies that human intelligence is mechanical. So the emotions you feel are under a black box same as the LLM and you apply you biased assumptions in a singular direction assuming your emotions are not deterministic and that LLM emotions are fake but that reasoning has no basis.

big_toast 7mo ago

One day I realized I needed to make sure I'm voting on quality stories/comments. I wonder if there was a call to vote substantively and often, if that might change the SNR. The guidelines encourage substantive comments, but maybe voters are part of the solution too. Kinda like having a strong reward model for training LLMs and avoiding reward hacking or other undesirable behavior.

threethirtytwo 7mo ago
if voters are stupid then it doesn't really help. I think what's happening is reality is asserting itself too hard that people can't be so stupid anymore.

qaadika 7mo ago

I'm kinda one of those who believes they 'completely' understand LLMs. But I've also developed my understanding of them such that the internal mechanisms of the transformer, or really any future development in the space based on neural networks and machine learning is irrelevant. 1. A string of unicode characters is converted into an array of integers values (tokens) and input to a black box of choice. 2. The black box takes in the input, does its magic, and returns an output as an array of integer values. 3. The returned output is converted into a string of unicode characters and given to the user, or inserted in a code file, or whatever. At no point does the black box "read" the input in any way analogous to how a human reads. Where people get "The AIs have emotions!!!" from returning an array of integers values is beyond me. It's definitely more complicated than "next token predictor", but it really is as simple as "Make words look like numbers, numbers go in, numbers come out, we make the numbers look like words."

threethirtytwo 7mo ago
Yeah nothing personal but my claim here is you’re not smart. The next token predictor aspect is something anyone can understand… the transformer is not quantum physics. Like look at what you wrote. You called it black box magic and in the same post you claim you understand LLMs. How the heck can you understand and call it a black box at the same time? The level of mental gymnastics and stupidity is through the roof. Clearly the majority of the utilitarian nature of the LLM is within the whole section you just waved away as “black box”. > Where people get "The AIs have emotions!!!" from returning an array of integers values is beyond me Let me spell it out for you. Those integers can be translated to the exact same language humans use when they feel identical emotions. So those people claim that the “black box” feels the emotions because what they observe is identical to what they observe in a human. The LLM can claim it feels emotions just like a human can claim the same thing. We assume humans feel emotions based off of this evidence but we don’t apply that logic to LLMs? The truth of the matter is we don’t actually know and it’s equally dumb to claim that you know LLMs feel emotions to claiming that they dont feel emotions. You have to be pretty stupid to not realize this is where they are coming from so there’s an aspect of you lying to yourself here because I don’t think you’re that stupid.
- adrian_b 7mo ago
  Of course LLMs display human emotions, if they have been trained on texts that have recorded humans displaying human emotions. With an input context that contains words that excite certain human emotions, the output of the core LLM function will generate a token probability distribution that is representative for the human emotions displayed by humans in the training texts. This is something expected and non-sensational. An LLM mimics the human behavior that was recorded in the training texts, much in the same way as a photographic image of a human face mimics the appearance of that human face. A photographic image is designed to reproduce the light field created by a face that reflects the ambient light, a LLM is created to reproduce the typical conversational behavior that was recorded in the training texts. Depending on how it was trained, one should expect a LLM to be affected by the choice of words used in the input in a similar way how a human would be affected. However, that does not mean that a LLM that shows signs of emotional distress feels some pain because of that. A LLM is designed for mimicry and it does not feel more pain or more happiness than a photograph of a wound feels pain from the wound or a photograph of a smiley face feels happiness. The fact that the current LLMs do not actually feel the human emotions that they may be able to mimic in an accurate way, does not mean that you could not build a robot which would have some built-in mechanisms for feeling pain and various emotions, which could be made to have similar functions like in an animal, serving a functional purpose and not being used for mimicry. However, for now it does not make any sense to attempt to do such a thing, because in a deterministic program there are better ways to ensure that a robot is "loyal" to its owner and acts in self-preservation when possible.

adrian_b 7mo ago

As other posters have pointed, the core of a LLM is a pure function, which computes a token probability distribution from an input context. An automaton, which can chat with you or write a program, is built externally to the LLM function, by storing the context and making it change, depending on the output of the LLM function. However, the LLM pure function is exceedingly complex so it is essentially unpredictable what will it produce for a given input context. So one may have to treat the LLM function as a black box and explore the huge space of the input contexts by varying them in various ways, inclusive by using words that express human emotions, and monitor how the output of the function changes, i.e. how the LLM "reacts" to the expressed emotions. A "reaction" similar to that of a human is to be expected, because human emotions were expressed in the training texts, followed by reactions of humans to those emotions, and the LLM function will change its output token probability function in a manner mimicking the behavior of the humans from the training texts. Even functions that are many orders of magnitude simpler than LLMs are still to complex for anyone to understand how their output changes when you move through the space of the possible input arguments. The most essential part of cryptography is the existence of a class of functions which were named by Claude Shannon "good mixing transformations". All the important cryptographic primitives, e.g. block cipher functions or one-way hash functions, are built from such "good mixing transformations". The impossibility of breaking a cryptographic system with secret keys is based on the assumption that it is impossible to predict how the output of such a "good mixing transformation" changes when its input is changed. All such "good mixing transformations" have the so-called avalanche property, which means that even if you change a single input bit, any of the output bits may change with a probability of exactly 50%, so it is unpredictable for any output bit whether it will change, or not. If such simple functions, e.g. with 128 input bits and 128 output bits, can have a completely unpredictable behavior, then it is not surprising that LLM functions that may have an input of up to a few million bits (the length of the context window) are completely unpredictable and you can just observe their behavior when given various kinds of contexts and search for empirical approximate rules describing the behavior.

threethirtytwo 7mo ago
If you read carefully my point is not about the external behavior of the LLM. It is the black box aspect of the LLM. The sheer complexity of the pure function is not something we can understand even though the high level structure is a feed forward network the core algorithm is in actuality encoded by weights. Yes there are complex functions besides LLMs that we don’t understand but those functions usually aren’t compelling because the LLM, unlike those other functions has output that implicates reasoning and emotions. The problem is we can’t understand what’s going on under the hood so we don’t know either way. This is what I mean by stupidity. You completely missed the point, and you’re also operating under the assumption that the human brain is also not following a similar deterministic pathway. You hold humanity and biological intelligence in such high regard that you cannot even imagine that all of physics implies that human intelligence is mechanical. So the emotions you feel are under a black box same as the LLM and you apply you biased assumptions in a singular direction assuming your emotions are not deterministic and that LLM emotions are fake but that reasoning has no basis.