3 ms·

ML has gone through a significant enough transformation in the last few years that it's velocity essentially makes it very difficult to form consensus about any

by jpgvm 5y ago

ML has gone through a significant enough transformation in the last few years that it's velocity essentially makes it very difficult to form consensus about anything right now as everything feels up for replacement in short order.

sillysaurusx 5y ago
That's probably true, which is why I love it so much. :) You're right that it might not be the best example. Still... in particle physics, for example, I've heard that there was consensus in favor of certain particles existing, even to the extent of "multiple experiments have achieved an N-sigma result showing that this particle exists," for some fairly high N. And then a later experiment showed that the particle simply didn't exist, and that the previous results were experimental error. (Specifically, experimental bias – they were expecting to see certain results, and so they were probably subconsciously combing the data for such results.) My knowledge on that subject comes from Veritasium, not from reading physics papers or following along with experimental breakthroughs. But I've seen similar things happen in ML, where people are pretty convinced a certain technique is good (hello, style mixing in StyleGAN) and no one ever bothers to check whether it's true in reality (style mixing is what prevents StyleGAN from being as cohesive as BigGAN). I guess I broadly agree with your point. There is consensus that black holes exist, and that we've even observed one, so it's pretty likely that if you were to travel there, you'd find a black hole. But given how much science itself has changed throughout history, alarm bells start going off when hearing things like "Consensus is extremely common, and it's almost universally a good thing."
- jpgvm 5y ago
  "Consensus is extremely common, and it's almost universally a good thing." isn't about saying things are assumed when they are not, just that the community is able to quickly adapt to changes in fundamental theory in order to explore the new ground opened up as a result. Take your example about StyleGAN vs BigGan, I assume once it became clear that the latter was superior to the former that likely resulted in changes to existing architectures that then found additional improvements. This change in consensus is what enables that and is a good thing. I think the problem here is conflating consensus with faith, that is assuming truth in absence of data. Consensus is easily changed with the introduction of new data, faith hangs on no matter how much evidence is put forward that it's horseshit.
  - sillysaurusx 5y ago
    > Take your example about StyleGAN vs BigGan, I assume once it became clear that the latter was superior to the former that likely resulted in changes to existing architectures that then found additional improvements. This change in consensus is what enables that and is a good thing. Well, no. :) But I think the "no" is because of the uniqueness of ML rather than a "no" to your point in general. You might be right about other fields; I don't have experience there. In ML, there an enormous number of techniques. Style mixing was presented as a core feature of StyleGAN (https://arxiv.org/abs/1812.04948 https://arxiv.org/abs/1812.04948) and was enabled by default in the codebase (https://github.com/NVlabs/stylegan/blob/03563d18a0cf8d67d897cc61e44479267968716b/training/networks_stylegan.py#L310 https://github.com/NVlabs/stylegan/blob/03563d18a0cf8d67d897...). So there's a lot of "inertia" -- for example, when StyleGAN 2 came out, style mixing was still the default (https://github.com/NVlabs/stylegan2/blob/f2f751cdc7f996e3138b8b1c08b6cf7ed36a1fc6/training/networks_stylegan2.py#L159 https://github.com/NVlabs/stylegan2/blob/f2f751cdc7f996e3138...). I haven't had time to dig into StyleGAN 3, but I suspect that style mixing might still be enabled by default. It wasn't until we did a detailed, methodical analysis side-by-side with BigGAN, specifically to answer the question "Why is BigGAN so much better for diverse datasets?" that, on a whim, I turned off style mixing and was astonished to see BigGAN type quality pop out of a StyleGAN type arch. Discoveries like that usually go unnoticed, frankly because it's a lot of effort to write a paper specifically to say "Hey, if you're training StyleGAN, definitely turn off style mixing. It only seems to work well on faces." However, if such a paper were to be written, and accepted into a peer-reviewed journal, then your original point would probably be valid. So I don't even know if it's worth writing all of this -- I just thought it'd be interesting to point out the "Well, not really" in this case. The knowledge ends up floating around on Twitter and Discord rather than being transmitted via scientific papers... But, this all does tie in to your final point: > Consensus is easily changed with the introduction of new data, faith hangs on no matter how much evidence is put forward that it's horseshit. It's remarkably easy for old, accepted ideas to hang around. You'd think it'd just be a matter of "Run the experiment; experiment proves thing; thing becomes accepted." But in practice it's felt quite different... The thing is, everything you're saying is true in general. As t approaches infinity, there tends to be more and more consensus about older ideas, like the existence of black holes, or the validity of laws like F=ma. So we should probably pay attention when there is 99% consensus on a particular topic. But, for example, one reason I wouldn't want to publish a paper claiming style mixing was bad, is because it would contradict the results of Karras, who is famous. I'd better be very certain about my claim! So there's sometimes a reluctance to contradict the consensus, too, which ends up equivalent to "faith" in your example -- we have faith that famous scientists are correct. (They usually are.) As a cherry on top, I'll just leave a link to Feynman's messenger lectures: https://www.youtube.com/watch?v=-kFOXP026eE&ab_channel=TalkReelTV https://www.youtube.com/watch?v=-kFOXP026eE&ab_channel=TalkR... ... the history of science is fascinating. I'd dreamt for years of becoming a scientist, but the actual experience turned out to be surprisingly different than what I thought it'd be. I love it though -- all these weird corner cases are the spice of life.
- cycomanic 5y ago
  I'm not sure what particle physics experiment you are talking about, generally it is very difficult (or even impossible) to show that something does not exist, so I think the story likely went somewhat different (although the gist of it might still be correct). I'm not a particle physicist either though. Regarding your comment on ML and consensus, I think there is a bit of a misconception of what scientific consensus is. I would say it is much broader than the examples you give. ML is essentially terrible example to argue about consensus, I would say there isn't even consensus about what ML even is, let alone any of the details of how you train a network. Regarding your earlier example about loss, I'm a bit confused. Isn't it that in optimisation theory the goal that you want to minimise your loss (so lower loss is always better)? Now that might not mean that you should travel along the steepest gradient (this might lead you to a local minimum) and your loss might not represent what you actually want to optimise for (i.e. your loss function is faulty). But from a fundamental theoretical point of view, you always want the lowest loss?
  - sillysaurusx 5y ago
    Unfortunately, it's 2:45am, and I have a meeting tomorrow. But I'll try to set aside some time for a detailed reply. The cliff's notes are: the first time I was able to train GPT 1.5B on gwern's poetry, I was super proud of myself, because it was extremely hard to get 1.5B to train back in those days. So we let it train as long as possible, and the loss kept dropping lower and lower. So we'd periodically dump samples -- surely we'd get better poetry, right? No. Eventually the model degenerated into broken-sounding English. Because our training data was 50MB, so it quickly forgot how to speak English at all -- the prior knowledge got wiped out, and was replaced by that 50MB dataset. The term for this is "overfitting," and it's remarkably easy to run into it in practice. The cure is to have more data. But how do you gather more than 50MB of poetry? That's a lot of poetry. So what do you do when you want to train a poetry model? The answer is to train a general model on a massive dataset (https://arxiv.org/abs/2101.00027 https://arxiv.org/abs/2101.00027), and then to fine-tune it on a smaller dataset (like 50MB of poetry). But during that fine tuning step, you'd better watch out, because everything you thought you knew about ML is wrong: You need to have a gentle downwards-sloping loss curve -- steep curve is terrible, since it means you're destroying the old knowledge without giving it a broad view of the new knowledge. And you need to stop before hitting loss < 2.1, otherwise you'll be overfitting and the model will suck. I suspect that hundreds of great ML models were overlooked for these reasons. There are hundreds of thousands of people messing around with ML models for fun. Lots of them try to train those models on small datasets, because small datasets are all you have when you start out. So they google how to do it, hear "Low loss is good," train overnight, dump samples, and then end up going "Oh well, the model seems to suck, but the loss is low, so I guess I didn't do it properly." They probably did train it properly, but they weren't continuously dumping samples, so they didn't notice they needed to stop before the model turned into word salad. (Or art salad if it's a GAN.) I would say that from a fundamental theoretical point of view, "you always want the lowest loss" is only true when you have an infinite amount of data in a target domain. But in practice, you probably have a few gigabytes if you're lucky. Even imagenet is only 1,000 different image classes, which means it's possible for a model to memorize those 1,000 classes and overfit, rather than achieving a general "understanding" of different kinds of objects. I got all of these details wrong when I first started out, so it was really eye-opening to see that it's a terrible idea to rely on common knowledge in ML. Common knowledge is an excellent starting point, but you quickly learn to think for yourself, or you just end up imitating what the big players are doing while wondering why none of your models seem to be working. (... I guess I wrote a detailed reply after all. Ah well, sleep is optional as long as you choose to ignore the effects of sleep deprivation. That's how that works.)
- Dylan16807 5y ago
  I think you really need to name a couple particles like that if you're going to bring it up. The most biased super-precise physics I can think of offhand is when someone measures a physical constant slightly wrong and then subsequent results all lean in the same direction. This happens more than once, most notably with fundamental electric charge, but it's absolutely nothing compared to a nonexistent value! https://en.m.wikipedia.org/wiki/List_of_experimental_errors_and_frauds_in_physics https://en.m.wikipedia.org/wiki/List_of_experimental_errors_... has https://en.m.wikipedia.org/wiki/Oops-Leon https://en.m.wikipedia.org/wiki/Oops-Leon? But that is a single experiment which very specifically did not have a high sigma result.

by jpgvm 5y ago

sillysaurusx 5y ago

That's probably true, which is why I love it so much. :) You're right that it might not be the best example. Still... in particle physics, for example, I've heard that there was consensus in favor of certain particles existing, even to the extent of "multiple experiments have achieved an N-sigma result showing that this particle exists," for some fairly high N. And then a later experiment showed that the particle simply didn't exist, and that the previous results were experimental error. (Specifically, experimental bias – they were expecting to see certain results, and so they were probably subconsciously combing the data for such results.) My knowledge on that subject comes from Veritasium, not from reading physics papers or following along with experimental breakthroughs. But I've seen similar things happen in ML, where people are pretty convinced a certain technique is good (hello, style mixing in StyleGAN) and no one ever bothers to check whether it's true in reality (style mixing is what prevents StyleGAN from being as cohesive as BigGAN). I guess I broadly agree with your point. There is consensus that black holes exist, and that we've even observed one, so it's pretty likely that if you were to travel there, you'd find a black hole. But given how much science itself has changed throughout history, alarm bells start going off when hearing things like "Consensus is extremely common, and it's almost universally a good thing."

jpgvm 5y ago
"Consensus is extremely common, and it's almost universally a good thing." isn't about saying things are assumed when they are not, just that the community is able to quickly adapt to changes in fundamental theory in order to explore the new ground opened up as a result. Take your example about StyleGAN vs BigGan, I assume once it became clear that the latter was superior to the former that likely resulted in changes to existing architectures that then found additional improvements. This change in consensus is what enables that and is a good thing. I think the problem here is conflating consensus with faith, that is assuming truth in absence of data. Consensus is easily changed with the introduction of new data, faith hangs on no matter how much evidence is put forward that it's horseshit.
- sillysaurusx 5y ago
  > Take your example about StyleGAN vs BigGan, I assume once it became clear that the latter was superior to the former that likely resulted in changes to existing architectures that then found additional improvements. This change in consensus is what enables that and is a good thing. Well, no. :) But I think the "no" is because of the uniqueness of ML rather than a "no" to your point in general. You might be right about other fields; I don't have experience there. In ML, there an enormous number of techniques. Style mixing was presented as a core feature of StyleGAN (https://arxiv.org/abs/1812.04948 https://arxiv.org/abs/1812.04948) and was enabled by default in the codebase (https://github.com/NVlabs/stylegan/blob/03563d18a0cf8d67d897cc61e44479267968716b/training/networks_stylegan.py#L310 https://github.com/NVlabs/stylegan/blob/03563d18a0cf8d67d897...). So there's a lot of "inertia" -- for example, when StyleGAN 2 came out, style mixing was still the default (https://github.com/NVlabs/stylegan2/blob/f2f751cdc7f996e3138b8b1c08b6cf7ed36a1fc6/training/networks_stylegan2.py#L159 https://github.com/NVlabs/stylegan2/blob/f2f751cdc7f996e3138...). I haven't had time to dig into StyleGAN 3, but I suspect that style mixing might still be enabled by default. It wasn't until we did a detailed, methodical analysis side-by-side with BigGAN, specifically to answer the question "Why is BigGAN so much better for diverse datasets?" that, on a whim, I turned off style mixing and was astonished to see BigGAN type quality pop out of a StyleGAN type arch. Discoveries like that usually go unnoticed, frankly because it's a lot of effort to write a paper specifically to say "Hey, if you're training StyleGAN, definitely turn off style mixing. It only seems to work well on faces." However, if such a paper were to be written, and accepted into a peer-reviewed journal, then your original point would probably be valid. So I don't even know if it's worth writing all of this -- I just thought it'd be interesting to point out the "Well, not really" in this case. The knowledge ends up floating around on Twitter and Discord rather than being transmitted via scientific papers... But, this all does tie in to your final point: > Consensus is easily changed with the introduction of new data, faith hangs on no matter how much evidence is put forward that it's horseshit. It's remarkably easy for old, accepted ideas to hang around. You'd think it'd just be a matter of "Run the experiment; experiment proves thing; thing becomes accepted." But in practice it's felt quite different... The thing is, everything you're saying is true in general. As t approaches infinity, there tends to be more and more consensus about older ideas, like the existence of black holes, or the validity of laws like F=ma. So we should probably pay attention when there is 99% consensus on a particular topic. But, for example, one reason I wouldn't want to publish a paper claiming style mixing was bad, is because it would contradict the results of Karras, who is famous. I'd better be very certain about my claim! So there's sometimes a reluctance to contradict the consensus, too, which ends up equivalent to "faith" in your example -- we have faith that famous scientists are correct. (They usually are.) As a cherry on top, I'll just leave a link to Feynman's messenger lectures: https://www.youtube.com/watch?v=-kFOXP026eE&ab_channel=TalkReelTV https://www.youtube.com/watch?v=-kFOXP026eE&ab_channel=TalkR... ... the history of science is fascinating. I'd dreamt for years of becoming a scientist, but the actual experience turned out to be surprisingly different than what I thought it'd be. I love it though -- all these weird corner cases are the spice of life.
cycomanic 5y ago
I'm not sure what particle physics experiment you are talking about, generally it is very difficult (or even impossible) to show that something does not exist, so I think the story likely went somewhat different (although the gist of it might still be correct). I'm not a particle physicist either though. Regarding your comment on ML and consensus, I think there is a bit of a misconception of what scientific consensus is. I would say it is much broader than the examples you give. ML is essentially terrible example to argue about consensus, I would say there isn't even consensus about what ML even is, let alone any of the details of how you train a network. Regarding your earlier example about loss, I'm a bit confused. Isn't it that in optimisation theory the goal that you want to minimise your loss (so lower loss is always better)? Now that might not mean that you should travel along the steepest gradient (this might lead you to a local minimum) and your loss might not represent what you actually want to optimise for (i.e. your loss function is faulty). But from a fundamental theoretical point of view, you always want the lowest loss?
- sillysaurusx 5y ago
  Unfortunately, it's 2:45am, and I have a meeting tomorrow. But I'll try to set aside some time for a detailed reply. The cliff's notes are: the first time I was able to train GPT 1.5B on gwern's poetry, I was super proud of myself, because it was extremely hard to get 1.5B to train back in those days. So we let it train as long as possible, and the loss kept dropping lower and lower. So we'd periodically dump samples -- surely we'd get better poetry, right? No. Eventually the model degenerated into broken-sounding English. Because our training data was 50MB, so it quickly forgot how to speak English at all -- the prior knowledge got wiped out, and was replaced by that 50MB dataset. The term for this is "overfitting," and it's remarkably easy to run into it in practice. The cure is to have more data. But how do you gather more than 50MB of poetry? That's a lot of poetry. So what do you do when you want to train a poetry model? The answer is to train a general model on a massive dataset (https://arxiv.org/abs/2101.00027 https://arxiv.org/abs/2101.00027), and then to fine-tune it on a smaller dataset (like 50MB of poetry). But during that fine tuning step, you'd better watch out, because everything you thought you knew about ML is wrong: You need to have a gentle downwards-sloping loss curve -- steep curve is terrible, since it means you're destroying the old knowledge without giving it a broad view of the new knowledge. And you need to stop before hitting loss < 2.1, otherwise you'll be overfitting and the model will suck. I suspect that hundreds of great ML models were overlooked for these reasons. There are hundreds of thousands of people messing around with ML models for fun. Lots of them try to train those models on small datasets, because small datasets are all you have when you start out. So they google how to do it, hear "Low loss is good," train overnight, dump samples, and then end up going "Oh well, the model seems to suck, but the loss is low, so I guess I didn't do it properly." They probably did train it properly, but they weren't continuously dumping samples, so they didn't notice they needed to stop before the model turned into word salad. (Or art salad if it's a GAN.) I would say that from a fundamental theoretical point of view, "you always want the lowest loss" is only true when you have an infinite amount of data in a target domain. But in practice, you probably have a few gigabytes if you're lucky. Even imagenet is only 1,000 different image classes, which means it's possible for a model to memorize those 1,000 classes and overfit, rather than achieving a general "understanding" of different kinds of objects. I got all of these details wrong when I first started out, so it was really eye-opening to see that it's a terrible idea to rely on common knowledge in ML. Common knowledge is an excellent starting point, but you quickly learn to think for yourself, or you just end up imitating what the big players are doing while wondering why none of your models seem to be working. (... I guess I wrote a detailed reply after all. Ah well, sleep is optional as long as you choose to ignore the effects of sleep deprivation. That's how that works.)
Dylan16807 5y ago
I think you really need to name a couple particles like that if you're going to bring it up. The most biased super-precise physics I can think of offhand is when someone measures a physical constant slightly wrong and then subsequent results all lean in the same direction. This happens more than once, most notably with fundamental electric charge, but it's absolutely nothing compared to a nonexistent value! https://en.m.wikipedia.org/wiki/List_of_experimental_errors_and_frauds_in_physics https://en.m.wikipedia.org/wiki/List_of_experimental_errors_... has https://en.m.wikipedia.org/wiki/Oops-Leon https://en.m.wikipedia.org/wiki/Oops-Leon? But that is a single experiment which very specifically did not have a high sigma result.