3 ms·

You should not expect intelligence from LLMs. They merely predict the next token based on your input. You should not anticipate their predictions to be anything

You should not expect intelligence from LLMs. They merely predict the next token based on your input. You should not anticipate their predictions to be anything other than a normal distribution, due to the data they have been trained on.

famouswaffles 3y ago
They "just" predict tokens ? What does that actually means ? Do you know what they do in order to predict tokens accurately?
- pk-protect-ai 3y ago
  This is a good one. The answer to this question is both very simple and very complex. Here is the simple (and very simplified) answer: Text is converted into tokens (indices of the tokenizer's vocabulary), so the model does not work with text directly but with the indices of the tokenizer's dictionary. The tokens are then organized into a tensor for processing. This tensor is typically a 2D tensor, where each row represents a sequence of tokens. The number of rows corresponds to the number of sequences being processed simultaneously (the batch size), and the number of columns corresponds to the length of the longest sequence in the batch (the sequence length). Each element in the tensor is an integer that represents a specific token. The tensor is then passed through an embedding layer, which converts each token (integer) into a dense vector of fixed size. The embedding layer is essentially a lookup table, where each row corresponds to a token in the vocabulary. It is expected that this embedding vector captures the semantic meaning of the input text. These embeddings undergo a series of arithmetic operations in each hidden layer of the model. Each element of the hidden layer matrix represents a real number, known as a weight. Weights are "learned" - approximated during training from an enormous amount of data. The set of weights learned from the data represents the probabilistic behavior of the data processed during training (please note that the weights do not represent probabilities directly). The embeddings are multiplied by these weights. This is followed by the addition of a bias term and the application of a non-linear activation function (like swish). This process transforms the embeddings, allowing the model to learn complex patterns in the data. This process continues for all the hidden layers, and this is where the "magic" happens. I can't explain this magic because I'm under-qualified for this. However, what I can tell is that probabilities of tokens appearing close to each other or far from each other are stored in the network of weights. The final layer of the model produces a tensor of logits, which represent the model’s predictions for the next token in the sequence for each input token. Depending on the architecture, a softmax function may be applied to these logits to produce a probability distribution over the vocabulary for each predicted token. The method for selecting the predicted token can vary, including choosing the token with the highest probability, using beam search, nucleus sampling, or other methods specified in the architecture. This is also part of the "magic". These predicted tokens are then converted back into human-readable text, forming the model’s generated response. This is how the model processes text and generates responses. You might find clues to the complex part in the simple answer tagged as "magic". I'm under-qualified for the magic part. You need to ask those with extensive knowledge of mathematics and probability theory to reveal the actual math behind the magic for you. But there is a part that no one can answer at the moment: How do the relations between tokens caught from the training data cause the spatial comprehension and basic arithmetic capabilities (yet very limited) in LLMs? We can't answer this question for living creatures as well. How can a goldfish distinguish "less" from "more", and how can chickens count to 10? In any case, LLMs are very spectacular predictors for what they have learned, and for what they haven't, they are even more spectacular.
  - famouswaffles 3y ago
    I know how LLMs operate in that degree. My point is that I can't for the life of me understand sentences like "You should not expect intelligence from token prediction" when you neither understand the operations of intelligence in the brain nor what the computation that preceeds any prediction means. Even if you somehow did, why does GPT have to be a brain to be intelligent any more than a plan has to be a bird to fly.
    - pk-protect-ai 3y ago
      Ha-ha, you can't assign an "intelligence" label to anything either because there is no clear definition of intelligence. Even if something resembles or even strikes you as intelligent (whatever this means), it might be a simulation with a very good prediction of your expectations. Have you ever caught yourself talking out of your ass sometimes? Don't be embarrassed, anyone does it sometimes. That's how our neural network hallucinates when we don't know the answer. I would consider this a property of approximation networks rather than a sign of intelligence. Wouldn't you agree?

famouswaffles 3y ago

They "just" predict tokens ? What does that actually means ? Do you know what they do in order to predict tokens accurately?

pk-protect-ai 3y ago
This is a good one. The answer to this question is both very simple and very complex. Here is the simple (and very simplified) answer: Text is converted into tokens (indices of the tokenizer's vocabulary), so the model does not work with text directly but with the indices of the tokenizer's dictionary. The tokens are then organized into a tensor for processing. This tensor is typically a 2D tensor, where each row represents a sequence of tokens. The number of rows corresponds to the number of sequences being processed simultaneously (the batch size), and the number of columns corresponds to the length of the longest sequence in the batch (the sequence length). Each element in the tensor is an integer that represents a specific token. The tensor is then passed through an embedding layer, which converts each token (integer) into a dense vector of fixed size. The embedding layer is essentially a lookup table, where each row corresponds to a token in the vocabulary. It is expected that this embedding vector captures the semantic meaning of the input text. These embeddings undergo a series of arithmetic operations in each hidden layer of the model. Each element of the hidden layer matrix represents a real number, known as a weight. Weights are "learned" - approximated during training from an enormous amount of data. The set of weights learned from the data represents the probabilistic behavior of the data processed during training (please note that the weights do not represent probabilities directly). The embeddings are multiplied by these weights. This is followed by the addition of a bias term and the application of a non-linear activation function (like swish). This process transforms the embeddings, allowing the model to learn complex patterns in the data. This process continues for all the hidden layers, and this is where the "magic" happens. I can't explain this magic because I'm under-qualified for this. However, what I can tell is that probabilities of tokens appearing close to each other or far from each other are stored in the network of weights. The final layer of the model produces a tensor of logits, which represent the model’s predictions for the next token in the sequence for each input token. Depending on the architecture, a softmax function may be applied to these logits to produce a probability distribution over the vocabulary for each predicted token. The method for selecting the predicted token can vary, including choosing the token with the highest probability, using beam search, nucleus sampling, or other methods specified in the architecture. This is also part of the "magic". These predicted tokens are then converted back into human-readable text, forming the model’s generated response. This is how the model processes text and generates responses. You might find clues to the complex part in the simple answer tagged as "magic". I'm under-qualified for the magic part. You need to ask those with extensive knowledge of mathematics and probability theory to reveal the actual math behind the magic for you. But there is a part that no one can answer at the moment: How do the relations between tokens caught from the training data cause the spatial comprehension and basic arithmetic capabilities (yet very limited) in LLMs? We can't answer this question for living creatures as well. How can a goldfish distinguish "less" from "more", and how can chickens count to 10? In any case, LLMs are very spectacular predictors for what they have learned, and for what they haven't, they are even more spectacular.
- famouswaffles 3y ago
  I know how LLMs operate in that degree. My point is that I can't for the life of me understand sentences like "You should not expect intelligence from token prediction" when you neither understand the operations of intelligence in the brain nor what the computation that preceeds any prediction means. Even if you somehow did, why does GPT have to be a brain to be intelligent any more than a plan has to be a bird to fly.
  - pk-protect-ai 3y ago
    Ha-ha, you can't assign an "intelligence" label to anything either because there is no clear definition of intelligence. Even if something resembles or even strikes you as intelligent (whatever this means), it might be a simulation with a very good prediction of your expectations. Have you ever caught yourself talking out of your ass sometimes? Don't be embarrassed, anyone does it sometimes. That's how our neural network hallucinates when we don't know the answer. I would consider this a property of approximation networks rather than a sign of intelligence. Wouldn't you agree?