4 ms·

I have very limited understanding of chatGPTs architecture, and neural nets in general. From what I've read over the last few days, the "attention" mechanism u

I have very limited understanding of chatGPTs architecture, and neural nets in general. From what I've read over the last few days, the "attention" mechanism used in chatGPT and similar LLMs does indeed dynamically change weights of a portion of the model.

famouswaffles 3y ago
the connection between weights are dynamic but the weights themselves aren't being changed i.e being updated in response to the text. when training a model, the forward pass would happen i.e the generation and then depending on how close to truth it was, the configuration settings (aka the weights/neurons) would be adjusted to incorporate whatever little insight was gained from the text. Weights are matrices. The values of the matrices aren't changing.
- FPGAhacker 3y ago
  Maybe it comes down to semantics but when I read things like [1] I come away with the idea that the weights are altered. But it could also just be my misunderstanding. 1: https://towardsdatascience.com/an-intuitive-explanation-of-self-attention-4f72709638e1 https://towardsdatascience.com/an-intuitive-explanation-of-s...
  - fenomas 3y ago
    An attention mechanism is one or more layers in the neural network. When someone talks about attention altering the input vectors, they're referring to what those layers are doing and how data is transformed as it passes through them. But zooming out to the big picture, a neural network is a bunch of layers full of weights, and none of the weights changes except during training (including the weights in the attention layers).
  - whiteandnerdy 3y ago
    I think it does come down to semantics. When you say "weights", people will take you to mean the pre-trained parameters of the network. I agree that in some sense the attention weights are more like meta-weights that are applied to the context of the conversation to decide how to actually weight the various words. So it's totally correct to say that previous words in the conversation affect how future words will be weighted, and I think it's reasonable to call that 'learning': for example, you can tell ChatGPT new words and it will be able to use them in context. Again though, people usually take 'learning' to mean making updates to the trained parameters of the model itself, which obviously isn't happening here.
PartiallyTyped 3y ago
It is a run-time executed reweighting of predictions. The scalar is a multiplication between the outputs of the Key and Query kernels passed through softmax, which then multiply the result of the value kernel V(x). Now, you may argue that because it's a multiplication with a linear or affine kernel, you might as well use commutative property of scalar multiplication and multiply the factor with the weights first, and then multiply with the input to the kernel. But this only holds for very few kernels.

famouswaffles 3y ago

the connection between weights are dynamic but the weights themselves aren't being changed i.e being updated in response to the text. when training a model, the forward pass would happen i.e the generation and then depending on how close to truth it was, the configuration settings (aka the weights/neurons) would be adjusted to incorporate whatever little insight was gained from the text. Weights are matrices. The values of the matrices aren't changing.

FPGAhacker 3y ago
Maybe it comes down to semantics but when I read things like [1] I come away with the idea that the weights are altered. But it could also just be my misunderstanding. 1: https://towardsdatascience.com/an-intuitive-explanation-of-self-attention-4f72709638e1 https://towardsdatascience.com/an-intuitive-explanation-of-s...
- fenomas 3y ago
  An attention mechanism is one or more layers in the neural network. When someone talks about attention altering the input vectors, they're referring to what those layers are doing and how data is transformed as it passes through them. But zooming out to the big picture, a neural network is a bunch of layers full of weights, and none of the weights changes except during training (including the weights in the attention layers).
- whiteandnerdy 3y ago
  I think it does come down to semantics. When you say "weights", people will take you to mean the pre-trained parameters of the network. I agree that in some sense the attention weights are more like meta-weights that are applied to the context of the conversation to decide how to actually weight the various words. So it's totally correct to say that previous words in the conversation affect how future words will be weighted, and I think it's reasonable to call that 'learning': for example, you can tell ChatGPT new words and it will be able to use them in context. Again though, people usually take 'learning' to mean making updates to the trained parameters of the model itself, which obviously isn't happening here.

PartiallyTyped 3y ago

It is a run-time executed reweighting of predictions. The scalar is a multiplication between the outputs of the Key and Query kernels passed through softmax, which then multiply the result of the value kernel V(x). Now, you may argue that because it's a multiplication with a linear or affine kernel, you might as well use commutative property of scalar multiplication and multiply the factor with the weights first, and then multiply with the input to the kernel. But this only holds for very few kernels.