4 ms·

I'm not into AI, but I like to watch from the sidelines. Here's my non-AI summary of the paper after glossing through (corrections appreciated): The multilayer

I'm not into AI, but I like to watch from the sidelines. Here's my non-AI summary of the paper after glossing through (corrections appreciated): The multilayered perceptron[1] layers used in modern neural networks, like LLMs, essentially partitions the input into multiple regions. They show that the number of regions a single MLP layer can partition into depends exponentially on the intrinsic dimension[2] of the input. The number of regions/partitions increases the approximation power of the MLP layer. Thus you can significantly increase the approximation power of a MLP layer without increasing the number of neurons, by essentially "distilling" the input to it. In the transformer architecture, the inputs to the MLP layers are the self-attention layers[3]. The authors then show that the graph density of the self-attention layers[3] correlates strongly with the intrinsic dimension of the self-attention layer. Thus a more dense self-attention layer means the MLP can do a better job. One way of increasing the density of the attention layers is to add more context. (edited, see comment) They show that prepending any token as context to a question which increases the intrinsic dimension of the final layer makes the LLM perform better. They also note that the transformer architecture is susceptible to compounding approximation errors, and that the much more precise partitioning provided by the MLP layers when fed with high intrinsic-dimensional input can help with this. However the impact of this on generalization remains to be explored further. If the results hold up it does seem like this paper provides nice insight into how to better optimize LLMs and similar neural networks. [1]: https://en.wikipedia.org/wiki/Multilayer_perceptron https://en.wikipedia.org/wiki/Multilayer_perceptron [2]: https://en.wikipedia.org/wiki/Intrinsic_dimension https://en.wikipedia.org/wiki/Intrinsic_dimension [3]: https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)#Multi-head_attention https://en.wikipedia.org/wiki/Transformer_(deep_learning_arc...

Fripplebubby 2y ago
Awesome summarization by someone who read and actually understood the paper. > One way of increasing the density of the attention layers is to add more context. They show that simply prepending any token as context to a question makes the LLM perform better. Adding relevant context makes it even better. Right, I think a more intuitive way to think about this is to define density: the number of _edges_ in the self-attention graph connecting tokens. Maybe a simpler explanation: the number of times a token had some connection to another token divided by the number of tokens. So, tokens which actually relate to one another and provide information are good, non sequitur tokens don't help except that you say > They show that simply prepending any token as context to a question makes the LLM perform better. I think this is not quite right. What they found was: > pre-pending the question at hand with any type of token does increase the intrinsic dimension at the first layer > however, this increase is not necessarily correlated with the reasoning capability of the model but it is only > when the pre-pended tokens lead to an increase in the intrinsic dimension at the *final layer* of the model, the reasoning capabilities of the LLM improve significantly. (emphasis mine)
- magicalhippo 2y ago
  Thanks, good catch, got distracted by the editing flaws at the end there (they rewrote a section without removing the old one).
- deleted 2y ago
  [deleted]
deleted 2y ago
[deleted]
cs702 2y ago
Great summary. Thank you for posting it here! Your comment, along with Fripplebuddy's response to it, deserve to be at the top of the page. I have little to add, except for a question: Isn't the number of distinct regions of interest a subset of (and in the extreme, equal to) the Vapnik-Chervonenkis dimension[a] of the data? As I write this, there is no mention of VC-dimension in the OP. --- [a] https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_dimension https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_di... --- EDITS: After a bit of thought, I shortened the comment and turned it into a question.
- justkk 2y ago
  Since in LLM what is important is the result, I wonder if all those dimensions are dependent of accuracy, so that the dimension can be low if you want low accuracy but you need a high dimension (a large number of parameters) in order to increase accuracy. If this intuition is right then dimension is not the key concept rather the key is how minimal dimension at required accuracy scale with accuracy. A metaphor is the way in which humans structure knowledge, we don't learn by heart, rather we learn by considering local and global relations with other areas in order to construct global knowledge. So the curve that reflects the best tradeoff of dimension versus accuracy is an important curve that merits to be studied. In general, to learn well you need to separate clearly the main parts, so the regions should be structured in such a way that they provide rich and independent information, so simply using the number of regions don't seem to me to be enough, it can contain a lot of noise or randomness. Another point about the number of regions: if the number of regions is similar to the number of clusters in a clustering algorithm then the number of cluster is not a key factor since very different number of clusters could give similar performance and looking for a minimum number could limit the generalization capabilities of the model. In support vector machines there is the concept of margin between regions. If we fix a threshold to separate regions by a fixed margin then the number of regions is less noisy since you eliminate redundant and low information regions. So fixing the minimum margin or threshold seems to be the first step prior to studying the relation between the number of parameters, number of regions and performance of the model. PS: edited several times.

Fripplebubby 2y ago

Awesome summarization by someone who read and actually understood the paper. > One way of increasing the density of the attention layers is to add more context. They show that simply prepending any token as context to a question makes the LLM perform better. Adding relevant context makes it even better. Right, I think a more intuitive way to think about this is to define density: the number of _edges_ in the self-attention graph connecting tokens. Maybe a simpler explanation: the number of times a token had some connection to another token divided by the number of tokens. So, tokens which actually relate to one another and provide information are good, non sequitur tokens don't help except that you say > They show that simply prepending any token as context to a question makes the LLM perform better. I think this is not quite right. What they found was: > pre-pending the question at hand with any type of token does increase the intrinsic dimension at the first layer > however, this increase is not necessarily correlated with the reasoning capability of the model but it is only > when the pre-pended tokens lead to an increase in the intrinsic dimension at the *final layer* of the model, the reasoning capabilities of the LLM improve significantly. (emphasis mine)

magicalhippo 2y ago
Thanks, good catch, got distracted by the editing flaws at the end there (they rewrote a section without removing the old one).
deleted 2y ago
[deleted]

deleted 2y ago

[deleted]

cs702 2y ago

Great summary. Thank you for posting it here! Your comment, along with Fripplebuddy's response to it, deserve to be at the top of the page. I have little to add, except for a question: Isn't the number of distinct regions of interest a subset of (and in the extreme, equal to) the Vapnik-Chervonenkis dimension[a] of the data? As I write this, there is no mention of VC-dimension in the OP. --- [a] https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_dimension https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_di... --- EDITS: After a bit of thought, I shortened the comment and turned it into a question.

justkk 2y ago
Since in LLM what is important is the result, I wonder if all those dimensions are dependent of accuracy, so that the dimension can be low if you want low accuracy but you need a high dimension (a large number of parameters) in order to increase accuracy. If this intuition is right then dimension is not the key concept rather the key is how minimal dimension at required accuracy scale with accuracy. A metaphor is the way in which humans structure knowledge, we don't learn by heart, rather we learn by considering local and global relations with other areas in order to construct global knowledge. So the curve that reflects the best tradeoff of dimension versus accuracy is an important curve that merits to be studied. In general, to learn well you need to separate clearly the main parts, so the regions should be structured in such a way that they provide rich and independent information, so simply using the number of regions don't seem to me to be enough, it can contain a lot of noise or randomness. Another point about the number of regions: if the number of regions is similar to the number of clusters in a clustering algorithm then the number of cluster is not a key factor since very different number of clusters could give similar performance and looking for a minimum number could limit the generalization capabilities of the model. In support vector machines there is the concept of margin between regions. If we fix a threshold to separate regions by a fixed margin then the number of regions is less noisy since you eliminate redundant and low information regions. So fixing the minimum margin or threshold seems to be the first step prior to studying the relation between the number of parameters, number of regions and performance of the model. PS: edited several times.