5 ms·

I think this is a lot of the mathematics of scaling LLM training. Which is quite important! One fundamental requirement though for any machine learning enginee

I think this is a lot of the mathematics of scaling LLM training. Which is quite important! One fundamental requirement though for any machine learning engineer working on these kinds of systems is https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf https://people.math.harvard.edu/~ctm/home/text/others/shanno.... I do not want to be entirely hypocritical as I am still ingesting this theory myself (started several years ago!), but I've found it _absolutely crucial_ in working in ML, as it implicitly informs every single decision you make when designing, deploying, and scaling neural networks. Without it, I feel the field turns into an empirical "stabby stab-around in the dark" kind of game, which very much has its dopamine highs and lows, but ala Sutton, does not scale very well in the long-run. ;P

jackblemming 3y ago
Please list a single decision you've made that was directly influenced by Shannon's paper, and no I do not mean something you post hoc rationalized.
- jdkoeck 3y ago
  Good challenge, I hope we will get a response.
  - tysam_and 3y ago
    Your wish hath been granted! <3 :))))
- tysam_and 3y ago
  Sure. Basically everything in https://github.com/tysam-code/hlb-CIFAR10 https://github.com/tysam-code/hlb-CIFAR10 was directly founded on concepts shared in the above paper, down to the coding, commenting, and layout styles (hence why I advocate so strongly for it as a requirement for ML. The empirical benefits are clear to me). Before I sat down and wrote my first line, I spent a very long time thinking about how to optimize the repo. Not just in terms of information flow during training, but how the code was laid out (minimize the expected value of deltas for changes from a superset of possible code changes), to even the explanatory comments (ratio of space vs mental effort to decode the repo for experienced vs inexperienced developers). I really want it to be a good exemplary model of a different, more scalable, and more efficient way of conducting small-scale (and potentially resource-constrained) research. To do that, you have to maximize information efficiency at every stage of the pipeline, including temporally (!!!!). It's not perfect, but I've used info theory as a strong guiding light for that repo. There's more to say here, but it's a long conversation about the expected utility of doing research a few different kinds of ways.
  - deleted 3y ago
    [deleted]
- quickthrower2 3y ago
  I am new to all this but I suspect it is good to at least know why cross entropy loss is a good measure.
  - tysam_and 3y ago
    Just a point of clarification -- are you looking to know why it is, or are you pointing out that it's important? I'm suspecting the latter but am happy to share some resources if it's the former. (Also, the poster that you're replying to has a bit of a history of...stirring the pot in comment sections if you look. But hey, it let me show some of my open source work, so that's good, at least! ;PPPP)
    - quickthrower2 3y ago
      I think it is useful to know what works well for foundational reasons so you know what to tweak and what is less likely yo work. I would love to know more about why. I have hundreds of why questions about transformers! And deep learning in general. The main one being why are we so lucky that gradient descent is stable and works! Feels like a miracle and the opposite of the trickyness thrown at us by physics, maths and computer science! I mean there are plenty of reasons I intuitively (just gut feel without mathematical intuition) think gradient descent should break down on big models. For example it could just become chaotic, nudging any parameter has a butterfly effect and the loss bounces around randomly and doesn’t converge.
      - tysam_and 3y ago
        Feel free to shoot, I can do the best that I can. W.r.t. the weight updates, ah, but it is extremely chaotic, your intuition is correct I feel! :) :D It took me a lot of years to figure that out though, so good job on that one. We've just adapted our architectures to clamp down on those extra degrees of freedom and break that chained dependency from layer to layer. For example, how residuals softly 'force' a single, defined latent space for each group of layers that adds into a residual, how batchnorm tends to re-center and scale the data to prevent a statistical explosion -- decoupling it from learned scale and bias. How label smoothing basically can be viewed as an interpolation between raw cross entropy and predicting a uniform distribution every time -- what do you think that does to the internal sensitivity of the network? It's a great, label-free way to explicitly train for stability. If you look at the expected value of the magnitudes of cross entropy for output distributions of different entropies, you'll quickly see why an entropy-regulated loss greatly slows down the popcorn-bag of changing kernels (heh) to a dull mumble. Heavy ball momentum prevents a lot of oscillation because it smooths out our gradient updates over time -- surprisingly effective! And of course LR schedules like OneCycle that do a kind of LR annealing are very effective as well. There's many, many, many more little things that go into that, and I think it took a few decades for us to find a lot of those things! We could go back 20 years with the ideological innovations that we've found and even on the constrained compute would be leagues ahead of what was available at the time, as a result. Another cool effect is how the Lottery Ticket Hypothesis shows that with rolling enough 'tickets', one can find weights suitably close to a decent representation from the start -- and much of the training is just tuning, and potentially suppressing the noise from more poor initializations. That still somewhat goes over my head but it is a cool hypothesis indeed. I'm not sure if that's a good teaser response or not, but I'm happy to talk at length about certain things. :)))) <3 <3 <3 :D
davnn 3y ago
Do you mean that information theory in general is essential in working with ML systems, or a specific point raised by Shannon?
- tysam_and 3y ago
  I'm sure there's a specific point raised by Shannon, and I've been (very recently!) learning how Shannon himself is not the sole info theory dude (even though lots of people credit him for it!). But basically, the communication of information over a noisy channel _is_ the foundation for deep learning.
  - arketyp 3y ago
    As they say, intelligence is prediction is compression is encoding...
    - tysam_and 3y ago
      Well, I agree that encoding is compression, at least, but the rest of that statement I do disagree with! It seems to be one of the more common mantras going around right now, though. Which is partially why I advocate for Shannon's theory! It's very clean and bare metal in terms of being related to intelligence, though I think one could build a decent argument for intelligence from info theory.
      - PeterisP 3y ago
        > Well, I agree that encoding is compression, at least, but the rest of that statement I do disagree with! IMHO that Shannon's paper you linked is effectively the initial work for linking prediction with compression, showing how the information transmitted (and the necessity for transmitted information) decreases as you can more accurately predict the likelihoods of the next symbol.
        tysam_and 3y ago
        Yes, but prediction != compression, and intelligence != prediction! However, prediction can inform compression, but there's necessarily some framework needed to translate the information from prediction<->compression. Perhaps that is too nitpicky, but to me it's like thinking about raw materials (like raw food or whatnot) and how it ends up as a finished product (like cornbread. Corn farming is not cornbread, but there is a semi-clear family of paths from one to the other with its own set of optimizations, rules, and etc). Again, that could be a bit nitpicky on my end depending on how one is viewing it.
godelski 3y ago
There's something I like to tell people: "You don't need math to make a good model, but you need math to know why your model is wrong." All models are wrong, and actually one of the crucial points we're at is not enough concentration on how wrong our models are (or how wrong our evaluations are, which is also a type of model). I suggest also investing time in statistics, understanding higher dimensional spaces (topologically and statistically), and metric theory. It is always a "stab-around in the dark" unfortunately, but the deeper your mathematical understanding is the brighter your candle burns to help you poke around. I think a lack of mathematical understanding has also made people misread Sutton as endorsing "scale is all you need" rather than "flexibility combined with scale has historically resulted in the largest gains." These things are very different.
- tysam_and 3y ago
  Ha! I think we may be on the same page with Sutton, that and the misuse of the NFL theorem are the two disclaimers I put out the most. My most recent Sutton one was ~3 hours ago! (https://news.ycombinator.com/item?id=37129921#37151192 https://news.ycombinator.com/item?id=37129921#37151192). That's a really good point, and I stand corrected. I guess it is still very much stabbing around in the dark, just with some more information about the probability density function of good answers. Heck, even Terry Tao's post about how to solve problems shows very much a (refined) guess-and-check method, so I have quite little ground to stand on there. Metric theory is fun, and one I'd like to learn a lot more about. I certainly have a lot to learn there.
  - godelski 3y ago
    Yeah I think the nature of research is de facto searching in the dark. I mean if it weren't, would it really be research? I think of the analogy as the knowledge space is a dark void and our collective knowledge is permanent torches lighting up the area. Researchers are exploring into the darkness and finding suitable places to place the next torch. Maybe we can say that the torch's brightness is defined by how well known your work is, since people tend to the flames more (not to be confused with how important the work is). The big question to me is about the geometry of that knowledge space. Is it bounded? If bounded (most likely imo), how do we approach the bound? Do we plateau once we hit it (unlikely imo)? Do we approach {,sub,super}-linearly? Do we approach {,sub,super}-linearly and then transition into an asymptotic convergence? (e.g. S-curve) This question actually has a profound impact on the question of the risk of a super-intelligence. IMO knowledge is most likely bound ( and most likely looks like an S-curve. If we're more than half way through that S-curve than a super intelligence that is 100x smarter than us may only know <100x as much as us, which we have good reason to believe that this is the case since knowledge appears to compound and accelerate (linear and sublinear models don't fit our historic trend, but that doesn't tell us the shape of the curve ahead). That reduces the risk of a super intelligence absolutely outsmarting us since it would be easier to close the gap, especially since learning through observation is easier than learning through exploration (can use its torches, even if they are more difficult to see). I'm not sure I see this really discussed much in the super intelligence conversations and that we're often working with different priors so that creates fundamental disagreements that become impassible. Maybe we need to set these priors first before we even discuss the next priors about motivation of a super intelligence. At least for math, I don't think I have the capacity to learn it all in the short human life, even if we're just discussing what math would be helpful to ML. But it sure is fun to learn it, so I guess the inability to learn it all is not really a downside :) Now if only I can convince others that math is fun __and__ useful to ML.

jackblemming 3y ago

Please list a single decision you've made that was directly influenced by Shannon's paper, and no I do not mean something you post hoc rationalized.

jdkoeck 3y ago
Good challenge, I hope we will get a response.
- tysam_and 3y ago
  Your wish hath been granted! <3 :))))
tysam_and 3y ago
Sure. Basically everything in https://github.com/tysam-code/hlb-CIFAR10 https://github.com/tysam-code/hlb-CIFAR10 was directly founded on concepts shared in the above paper, down to the coding, commenting, and layout styles (hence why I advocate so strongly for it as a requirement for ML. The empirical benefits are clear to me). Before I sat down and wrote my first line, I spent a very long time thinking about how to optimize the repo. Not just in terms of information flow during training, but how the code was laid out (minimize the expected value of deltas for changes from a superset of possible code changes), to even the explanatory comments (ratio of space vs mental effort to decode the repo for experienced vs inexperienced developers). I really want it to be a good exemplary model of a different, more scalable, and more efficient way of conducting small-scale (and potentially resource-constrained) research. To do that, you have to maximize information efficiency at every stage of the pipeline, including temporally (!!!!). It's not perfect, but I've used info theory as a strong guiding light for that repo. There's more to say here, but it's a long conversation about the expected utility of doing research a few different kinds of ways.
- deleted 3y ago
  [deleted]
quickthrower2 3y ago
I am new to all this but I suspect it is good to at least know why cross entropy loss is a good measure.
- tysam_and 3y ago
  Just a point of clarification -- are you looking to know why it is, or are you pointing out that it's important? I'm suspecting the latter but am happy to share some resources if it's the former. (Also, the poster that you're replying to has a bit of a history of...stirring the pot in comment sections if you look. But hey, it let me show some of my open source work, so that's good, at least! ;PPPP)
  - quickthrower2 3y ago
    I think it is useful to know what works well for foundational reasons so you know what to tweak and what is less likely yo work. I would love to know more about why. I have hundreds of why questions about transformers! And deep learning in general. The main one being why are we so lucky that gradient descent is stable and works! Feels like a miracle and the opposite of the trickyness thrown at us by physics, maths and computer science! I mean there are plenty of reasons I intuitively (just gut feel without mathematical intuition) think gradient descent should break down on big models. For example it could just become chaotic, nudging any parameter has a butterfly effect and the loss bounces around randomly and doesn’t converge.
    - tysam_and 3y ago
      Feel free to shoot, I can do the best that I can. W.r.t. the weight updates, ah, but it is extremely chaotic, your intuition is correct I feel! :) :D It took me a lot of years to figure that out though, so good job on that one. We've just adapted our architectures to clamp down on those extra degrees of freedom and break that chained dependency from layer to layer. For example, how residuals softly 'force' a single, defined latent space for each group of layers that adds into a residual, how batchnorm tends to re-center and scale the data to prevent a statistical explosion -- decoupling it from learned scale and bias. How label smoothing basically can be viewed as an interpolation between raw cross entropy and predicting a uniform distribution every time -- what do you think that does to the internal sensitivity of the network? It's a great, label-free way to explicitly train for stability. If you look at the expected value of the magnitudes of cross entropy for output distributions of different entropies, you'll quickly see why an entropy-regulated loss greatly slows down the popcorn-bag of changing kernels (heh) to a dull mumble. Heavy ball momentum prevents a lot of oscillation because it smooths out our gradient updates over time -- surprisingly effective! And of course LR schedules like OneCycle that do a kind of LR annealing are very effective as well. There's many, many, many more little things that go into that, and I think it took a few decades for us to find a lot of those things! We could go back 20 years with the ideological innovations that we've found and even on the constrained compute would be leagues ahead of what was available at the time, as a result. Another cool effect is how the Lottery Ticket Hypothesis shows that with rolling enough 'tickets', one can find weights suitably close to a decent representation from the start -- and much of the training is just tuning, and potentially suppressing the noise from more poor initializations. That still somewhat goes over my head but it is a cool hypothesis indeed. I'm not sure if that's a good teaser response or not, but I'm happy to talk at length about certain things. :)))) <3 <3 <3 :D

davnn 3y ago

Do you mean that information theory in general is essential in working with ML systems, or a specific point raised by Shannon?

tysam_and 3y ago
I'm sure there's a specific point raised by Shannon, and I've been (very recently!) learning how Shannon himself is not the sole info theory dude (even though lots of people credit him for it!). But basically, the communication of information over a noisy channel _is_ the foundation for deep learning.
- arketyp 3y ago
  As they say, intelligence is prediction is compression is encoding...
  - tysam_and 3y ago
    Well, I agree that encoding is compression, at least, but the rest of that statement I do disagree with! It seems to be one of the more common mantras going around right now, though. Which is partially why I advocate for Shannon's theory! It's very clean and bare metal in terms of being related to intelligence, though I think one could build a decent argument for intelligence from info theory.
    - PeterisP 3y ago
      > Well, I agree that encoding is compression, at least, but the rest of that statement I do disagree with! IMHO that Shannon's paper you linked is effectively the initial work for linking prediction with compression, showing how the information transmitted (and the necessity for transmitted information) decreases as you can more accurately predict the likelihoods of the next symbol.
      - tysam_and 3y ago
        Yes, but prediction != compression, and intelligence != prediction! However, prediction can inform compression, but there's necessarily some framework needed to translate the information from prediction<->compression. Perhaps that is too nitpicky, but to me it's like thinking about raw materials (like raw food or whatnot) and how it ends up as a finished product (like cornbread. Corn farming is not cornbread, but there is a semi-clear family of paths from one to the other with its own set of optimizations, rules, and etc). Again, that could be a bit nitpicky on my end depending on how one is viewing it.

godelski 3y ago

There's something I like to tell people: "You don't need math to make a good model, but you need math to know why your model is wrong." All models are wrong, and actually one of the crucial points we're at is not enough concentration on how wrong our models are (or how wrong our evaluations are, which is also a type of model). I suggest also investing time in statistics, understanding higher dimensional spaces (topologically and statistically), and metric theory. It is always a "stab-around in the dark" unfortunately, but the deeper your mathematical understanding is the brighter your candle burns to help you poke around. I think a lack of mathematical understanding has also made people misread Sutton as endorsing "scale is all you need" rather than "flexibility combined with scale has historically resulted in the largest gains." These things are very different.

tysam_and 3y ago
Ha! I think we may be on the same page with Sutton, that and the misuse of the NFL theorem are the two disclaimers I put out the most. My most recent Sutton one was ~3 hours ago! (https://news.ycombinator.com/item?id=37129921#37151192 https://news.ycombinator.com/item?id=37129921#37151192). That's a really good point, and I stand corrected. I guess it is still very much stabbing around in the dark, just with some more information about the probability density function of good answers. Heck, even Terry Tao's post about how to solve problems shows very much a (refined) guess-and-check method, so I have quite little ground to stand on there. Metric theory is fun, and one I'd like to learn a lot more about. I certainly have a lot to learn there.
- godelski 3y ago
  Yeah I think the nature of research is de facto searching in the dark. I mean if it weren't, would it really be research? I think of the analogy as the knowledge space is a dark void and our collective knowledge is permanent torches lighting up the area. Researchers are exploring into the darkness and finding suitable places to place the next torch. Maybe we can say that the torch's brightness is defined by how well known your work is, since people tend to the flames more (not to be confused with how important the work is). The big question to me is about the geometry of that knowledge space. Is it bounded? If bounded (most likely imo), how do we approach the bound? Do we plateau once we hit it (unlikely imo)? Do we approach {,sub,super}-linearly? Do we approach {,sub,super}-linearly and then transition into an asymptotic convergence? (e.g. S-curve) This question actually has a profound impact on the question of the risk of a super-intelligence. IMO knowledge is most likely bound ( and most likely looks like an S-curve. If we're more than half way through that S-curve than a super intelligence that is 100x smarter than us may only know <100x as much as us, which we have good reason to believe that this is the case since knowledge appears to compound and accelerate (linear and sublinear models don't fit our historic trend, but that doesn't tell us the shape of the curve ahead). That reduces the risk of a super intelligence absolutely outsmarting us since it would be easier to close the gap, especially since learning through observation is easier than learning through exploration (can use its torches, even if they are more difficult to see). I'm not sure I see this really discussed much in the super intelligence conversations and that we're often working with different priors so that creates fundamental disagreements that become impassible. Maybe we need to set these priors first before we even discuss the next priors about motivation of a super intelligence. At least for math, I don't think I have the capacity to learn it all in the short human life, even if we're just discussing what math would be helpful to ML. But it sure is fun to learn it, so I guess the inability to learn it all is not really a downside :) Now if only I can convince others that math is fun __and__ useful to ML.