4 ms·

I'm not sure what particle physics experiment you are talking about, generally it is very difficult (or even impossible) to show that something does not exist,

I'm not sure what particle physics experiment you are talking about, generally it is very difficult (or even impossible) to show that something does not exist, so I think the story likely went somewhat different (although the gist of it might still be correct). I'm not a particle physicist either though. Regarding your comment on ML and consensus, I think there is a bit of a misconception of what scientific consensus is. I would say it is much broader than the examples you give. ML is essentially terrible example to argue about consensus, I would say there isn't even consensus about what ML even is, let alone any of the details of how you train a network. Regarding your earlier example about loss, I'm a bit confused. Isn't it that in optimisation theory the goal that you want to minimise your loss (so lower loss is always better)? Now that might not mean that you should travel along the steepest gradient (this might lead you to a local minimum) and your loss might not represent what you actually want to optimise for (i.e. your loss function is faulty). But from a fundamental theoretical point of view, you always want the lowest loss?

sillysaurusx 5y ago
Unfortunately, it's 2:45am, and I have a meeting tomorrow. But I'll try to set aside some time for a detailed reply. The cliff's notes are: the first time I was able to train GPT 1.5B on gwern's poetry, I was super proud of myself, because it was extremely hard to get 1.5B to train back in those days. So we let it train as long as possible, and the loss kept dropping lower and lower. So we'd periodically dump samples -- surely we'd get better poetry, right? No. Eventually the model degenerated into broken-sounding English. Because our training data was 50MB, so it quickly forgot how to speak English at all -- the prior knowledge got wiped out, and was replaced by that 50MB dataset. The term for this is "overfitting," and it's remarkably easy to run into it in practice. The cure is to have more data. But how do you gather more than 50MB of poetry? That's a lot of poetry. So what do you do when you want to train a poetry model? The answer is to train a general model on a massive dataset (https://arxiv.org/abs/2101.00027 https://arxiv.org/abs/2101.00027), and then to fine-tune it on a smaller dataset (like 50MB of poetry). But during that fine tuning step, you'd better watch out, because everything you thought you knew about ML is wrong: You need to have a gentle downwards-sloping loss curve -- steep curve is terrible, since it means you're destroying the old knowledge without giving it a broad view of the new knowledge. And you need to stop before hitting loss < 2.1, otherwise you'll be overfitting and the model will suck. I suspect that hundreds of great ML models were overlooked for these reasons. There are hundreds of thousands of people messing around with ML models for fun. Lots of them try to train those models on small datasets, because small datasets are all you have when you start out. So they google how to do it, hear "Low loss is good," train overnight, dump samples, and then end up going "Oh well, the model seems to suck, but the loss is low, so I guess I didn't do it properly." They probably did train it properly, but they weren't continuously dumping samples, so they didn't notice they needed to stop before the model turned into word salad. (Or art salad if it's a GAN.) I would say that from a fundamental theoretical point of view, "you always want the lowest loss" is only true when you have an infinite amount of data in a target domain. But in practice, you probably have a few gigabytes if you're lucky. Even imagenet is only 1,000 different image classes, which means it's possible for a model to memorize those 1,000 classes and overfit, rather than achieving a general "understanding" of different kinds of objects. I got all of these details wrong when I first started out, so it was really eye-opening to see that it's a terrible idea to rely on common knowledge in ML. Common knowledge is an excellent starting point, but you quickly learn to think for yourself, or you just end up imitating what the big players are doing while wondering why none of your models seem to be working. (... I guess I wrote a detailed reply after all. Ah well, sleep is optional as long as you choose to ignore the effects of sleep deprivation. That's how that works.)
- cycomanic 5y ago
  Ok, thanks for the clarification. I agree if you are overfitting lowest loss is not always best, and I did not think about that aspect when I wrote my reply. I think with everything you describe is one of the main "gripes" I as a more traditional engineer/scientist have with ML. It often seems like a giant black box that I know is generally vastly overfitted and it is often extremely difficult/impossible why we get the solutions that we get, i.e. we don't really get scientific insight. Nevertheless I realise it is an extremely powerful tool. Hope you can catch up with some sleep today, sleep deprivation effects can only be ignored for so long (especially as you get older as I have noticed).

sillysaurusx 5y ago

Unfortunately, it's 2:45am, and I have a meeting tomorrow. But I'll try to set aside some time for a detailed reply. The cliff's notes are: the first time I was able to train GPT 1.5B on gwern's poetry, I was super proud of myself, because it was extremely hard to get 1.5B to train back in those days. So we let it train as long as possible, and the loss kept dropping lower and lower. So we'd periodically dump samples -- surely we'd get better poetry, right? No. Eventually the model degenerated into broken-sounding English. Because our training data was 50MB, so it quickly forgot how to speak English at all -- the prior knowledge got wiped out, and was replaced by that 50MB dataset. The term for this is "overfitting," and it's remarkably easy to run into it in practice. The cure is to have more data. But how do you gather more than 50MB of poetry? That's a lot of poetry. So what do you do when you want to train a poetry model? The answer is to train a general model on a massive dataset (https://arxiv.org/abs/2101.00027 https://arxiv.org/abs/2101.00027), and then to fine-tune it on a smaller dataset (like 50MB of poetry). But during that fine tuning step, you'd better watch out, because everything you thought you knew about ML is wrong: You need to have a gentle downwards-sloping loss curve -- steep curve is terrible, since it means you're destroying the old knowledge without giving it a broad view of the new knowledge. And you need to stop before hitting loss < 2.1, otherwise you'll be overfitting and the model will suck. I suspect that hundreds of great ML models were overlooked for these reasons. There are hundreds of thousands of people messing around with ML models for fun. Lots of them try to train those models on small datasets, because small datasets are all you have when you start out. So they google how to do it, hear "Low loss is good," train overnight, dump samples, and then end up going "Oh well, the model seems to suck, but the loss is low, so I guess I didn't do it properly." They probably did train it properly, but they weren't continuously dumping samples, so they didn't notice they needed to stop before the model turned into word salad. (Or art salad if it's a GAN.) I would say that from a fundamental theoretical point of view, "you always want the lowest loss" is only true when you have an infinite amount of data in a target domain. But in practice, you probably have a few gigabytes if you're lucky. Even imagenet is only 1,000 different image classes, which means it's possible for a model to memorize those 1,000 classes and overfit, rather than achieving a general "understanding" of different kinds of objects. I got all of these details wrong when I first started out, so it was really eye-opening to see that it's a terrible idea to rely on common knowledge in ML. Common knowledge is an excellent starting point, but you quickly learn to think for yourself, or you just end up imitating what the big players are doing while wondering why none of your models seem to be working. (... I guess I wrote a detailed reply after all. Ah well, sleep is optional as long as you choose to ignore the effects of sleep deprivation. That's how that works.)

cycomanic 5y ago
Ok, thanks for the clarification. I agree if you are overfitting lowest loss is not always best, and I did not think about that aspect when I wrote my reply. I think with everything you describe is one of the main "gripes" I as a more traditional engineer/scientist have with ML. It often seems like a giant black box that I know is generally vastly overfitted and it is often extremely difficult/impossible why we get the solutions that we get, i.e. we don't really get scientific insight. Nevertheless I realise it is an extremely powerful tool. Hope you can catch up with some sleep today, sleep deprivation effects can only be ignored for so long (especially as you get older as I have noticed).