16 ms·
The lottery ticket hypothesis: why neural networks work
- derbOac 1y agoIn some sense, isn't this overfitting, but "hidden" by the typical feature sets that are observed? Time and time again, some kind of process will identify some simple but absurd adversarial "trick stimulus" that throws off the deep network solution. These seem like blatant cases of over fitting that go unrecognized or unchallenged in typical life because the sampling space of stimuli doesn't usually include the adversarial trick stimuli. I guess I've not really thought of the bias-variance tradeoff necessarily as being about number of parameters, but rather, the flexibility of the model relative to the learnable information in the sample space. There's some formulations (e.g., Shtarkov-Rissanen normalized maximum likelihood) that treat overfitting in terms of the ability to reproduce data that is wildly outside a typical training set. This is related to, but not the same as, the number of parameters per se.
- deleted 1y ago[deleted]
- api 1y agoThis sounds like it's proposing that what's happening during large model training is a little bit akin to genetic algorithms: many small networks emerge and there is a selection process, some get fixed, and the rest fade and are then repurposed/drifted into other roles, repeat.
- xg15 1y agoWouldn't this imply that most of the inference time storage and compute might be unnecessary? If the hypothesis is true, it makes sense to scale up models as much as possible during training - but once the model is sufficiently trained for the task, wouldn't 99% of the weights be literal "dead weight" - because they represent the "failed lottery tickets", i.e. the subnetworks that did not have the right starting values to learn anything useful? So why do we keep them around and waste enormous amounts of storage and compute on them?
- tough 1y agosomeone on twitter was exploring and linked to some related papers where you can for example trim experts on a MoE model if you're 100% sure they're never active for your specific task what the bigger wide net bigs you is generalization
- markeroon 1y agoLook into pruning
- paulsutter 1y agoThat’s exactly how it works, read up on pruning. You can ignore most of the weights and still get great results. One issue is that sparse matrices are vastly less efficient to multiply. But yes you’ve got it
- FuckButtons 1y agoFor any particular single pattern learned 99% of the weights are dead weight. But it’s not the same 99% for each lesson learned.
- janalsncm 1y agoQuick example, Kimi K2 is a recent large mixture of experts model. Each “expert” is really just a path within it. At each token, 32B out of 1T are active. This means only 3.2% are active for any one token.
- Sophira 1y agoThat sounds surprisingly like "Humans only use 10% of their brain at any given time."
- solbob 1y ago[dead]
- highfrequency 1y agoEnjoyed the article. To play devil’s advocate, an entirely different explanation for why huge models work: the primary insight was framing the problem as next-word prediction. This immediately creates an internet-scale dataset with trillions of labeled examples, which also has rich enough structure to make huge expressiveness useful. LLMs don’t disprove bias-variance tradeoff; we just found a lot more data and the GPUs to learn from it. It’s not like people didn’t try bigger models in the past, but either the data was too small or the structure too simple to show improvements with more model complexity. (Or they simply trained the biggest model they could fit on the GPUs of the time.)
- pixl97 1y agoI think a lot of it is the massive amount of compute we've got in the last decade. While inference may have been possible on the hardware the training would have taken lifetimes.
- graemep 1y agoI have a textbook somewhere in the house from about 2000 that says that there is no point having more than three layers in a neural network. Compute was just too expensive to have neural networks big enough for this not to be true.
- leumassuehtam 1y agoPeople believe that more parameters would lead to overfit instead generalization. The various regularization methods we use today to avoid overfit hadn't been discovered yet. Your statement is mostly likely about this.
- graemep 1y agoPossibly, I would have to dig up the book to check. IIRC it did not mention overfitting but it was a long time ago.
- Silphendio 1y ago
- belter 1y agoThis article is like a quick street rap. Lots of rhythm, not much thesis. Big on tone, light on analysis...Or no actual thesis other than a feelgood factor. I want these 5 min back.
- fgfarben 1y agoDo you think a GPT that already trained on something "feels" the same way when reading it a second time?
- JasonSage 1y agoOn the other hand, as somebody not well-read in AI I found it to be a rather intuitive explanation for why pruning helps avoid the overfitting scenario I learned when I first touched neural networks in the ‘10s. Sure, this could’ve been a paragraph, but it wasn’t. I don’t think it’s particularly offensive for that.
- mcphage 1y ago> I want these 5 min back. Tell me, what is it you plan to do with your five wild and precious minutes?
- abhinuvpitale 1y agoInteresting article, is it concluding that different small networks are formed for different types of problems that we are trying to solve with the larger network? How is this different from overfitting though? (PS: Overfitting isn't that bad if you think about it, as long as the test dataset or inference time model is trying to solve problems in the supposedly large enough training dataset)
- deepfriedchokes 1y agoRather than reframing intelligence itself, wouldn’t Occam’s Razor suggest instead that this isn’t intelligence at all?
- pixl97 1y agoI don't really think that is what Occam’s Razor is about. The Razor says the simplest answer is most likely the best, but we already know that intelligence is very complex so the simplest answer to intelligence is still going to be a massively complex solution. In some ways this answer does fit Occam's Razor by saying the simplicity is simply scale, not complex algorithms.
- gavmor 1y ago> Intelligence isn't about memorising information—it's about finding elegant patterns that explain complex phenomena. Scale provides the computational space needed for this search, not storage for complicated solutions. I think the word finding is overloaded, here. Are we "discovering," "deriving," "deducing," or simple "looking up" these patterns? If "finding" can be implemented via a multi-page tour—ie deterministic choose-your-own-adventure—of a three-ring-binder (which is, essentially, how inference operates) then we're back at Searle's Chinese Room, and no intelligence is operative at runtime. On the other hand, if the satisfaction of "finding" necessitates the creative synthesis of novel records pertaining to—if not outright modeling—external phenomena, ie "finding" a proof, then arguably it's not happening at training time, either. How many novel proofs have LLMs found?
- akomtu 1y agoEven simpler: intelligence is the art of simplifying. LLMs can fool us if they reduce a book into one wise-looking statement, but remove the deceptive medium - our language - and tell it to reduce a vast dataset of points into one formula, and LLMs will show how much intelligence they truly have.
- Eisenstein 1y agoUnless you can provide a definition for intelligence which is internally consistent and does not exclude things are obviously intelligent or include things which are obviously not intelligent, the only thing occam's razor suggests is that the basis for solving novel problems is the ability to pattern match combined with a lot of background knowledge.
- gotoeleven 1y agoThis article gives a really bad/wrong explanation of the lottery ticket hypothesis. Here's the original paper https://arxiv.org/abs/1803.03635 https://arxiv.org/abs/1803.03635
- brulard 1y agoThanks for the 42 page long document. Can you explain in few words why you evaluated it as "really bad/wrong explanation"?
- frrlpp 1y agoWhat are LLMs for?
- jeremyscanvic 1y agoExactly what I was looking for while reading the post. Thanks!
- ghssds 1y agoCan someone explain how AI research can have a 300 years history?
- anthonj 1y ago"For over 300 years, one principle governed every learning system: the bias-variance tradeoff." The bias-variance tradeoff is a very old concept in statistics (but not sure how old, might very well be 300) Anyway note the first algorithms realted to neural networks are older then the digital computer by a decade at least.
- woadwarrior01 1y ago300 years is a stretch. But Legendre described linear regression ~220 years ago (1805). And from a very high level perspective, modern neural networks are mostly just stacks of linear regression layers with non-linearities sandwiched between them. I'm obviously oversimplifying it a lot, but that't the gist of it.
- littlestymaar 1y agoMaybe it wasn't there originally, but now there's a footnote: > 1. The 300-year timeframe refers to the foundational mathematical principles underlying modern bias-variance analysis, not the contemporary terminology. Bayes' theorem (1763) established the mathematical framework for updating beliefs with evidence, whilst Laplace's early work on statistical inference (1780s-1810s) formalised the principle that models must balance fit with simplicity to avoid spurious conclusions. These early statistical insights—that overly complex explanations tend to capture noise rather than signal—form the mathematical bedrock of what we now call the bias-variance tradeoff. The specific modern formulation emerged over several decades in the latter 20th century, but the core principle has governed statistical reasoning for centuries.↩
- nitwit005 1y ago> For over 300 years, one principle governed every learning system This seems strangely worded. I assume that date is when some statistics paper was published, but there's no way to know with no definition or citations.
- littlestymaar 1y agoThere is in fact a footnote about the date: > 1. The 300-year timeframe refers to the foundational mathematical principles underlying modern bias-variance analysis, not the contemporary terminology. Bayes' theorem (1763) established the mathematical framework for updating beliefs with evidence, whilst Laplace's early work on statistical inference (1780s-1810s) formalised the principle that models must balance fit with simplicity to avoid spurious conclusions. These early statistical insights—that overly complex explanations tend to capture noise rather than signal—form the mathematical bedrock of what we now call the bias-variance tradeoff. The specific modern formulation emerged over several decades in the latter 20th century, but the core principle has governed statistical reasoning for centuries.
- nitwit005 1y agoThey added it after my post.
- doctoboggan 1y agoThis article definitely feels like chatgptese. Also, I don't necessarily feel like the size of LLMs even comes close to overfitting the data. From a very unscientific standpoint it seems like the size of weights on disk would have to meet or exceed the size of the training data (modulo lossless encryption techniques) for overfitting to occur. Since the training data is multiple orders of magnitude larger than the resulting weights, isn't that proof that the weights are some sort of generalization of the input data rather than a memorization?
- porridgeraisin 1y ago1) yes it's definitely chatgpt 2) The weights are definitely a generalization. The compression-based argument is sound. 3) There is definitely no overfitting. The article however used the word over-parameterization, which is a different thing. And LLMs are certainly over-parameterized. They have more parameters than strictly required to represent the dataset in a degrees-of-freedom statistical sense. This is not a bad thing though. Just like having an over-parameterized database schema: quiz(id, title, num_qns) question(id, text, answer, quiz_id FK) can be good for performance sometimes, The lottery ticket hypothesis as chatgpt explained in TFA means that over-parameterization can also be good for neural networks sometimes. Note that this hypothesis is strictly tied to the fact that we use SGD (or adam or ...) as the optimisation algorithm. SGD is known to be biased towards generalized compressions [the lottery ticket hypothesis hypothesises why this is so]. That is to say, it's not an inherent property of the neural network architecture or transformers or such.
- quantgenius 1y agoThe idea that simply having a lot of parameters leads to overfitting was shown to not be the case over 30 years ago by Vapnik et al. He proved that a large number of parameters is fine so long as you regularize enough. This is why Support Vector Machines work and I believe has a lot to do with why deep NNs work. The issue with Vapnik's work is that it's pretty dense and actually figuring out the Vapnik-Chervonekis (VC) dimension etc is pretty complicated, and one can develop pretty good intuition once you understand the stuff without having to actually calculate, so most people don't take the time to do the calculation. And frankly, a lot of the time, you don't need to. There may be something I'm missing completely, but to me the fact that models continue to generalize with a huge number of parameters is not all that surprising given how much we regularize when we fit NNs. A lot of the surprise comes from the fact that people in mathematical statistics and people who do neural networks (computer scientists) don't talk to each other as much as they should. Strongly recommend the book Statistical Learning Theory by Vapnik for more on this.
- math_dandy 1y agoI don't buy the narrative that the article is promoting. I think the machine learning community was largely over overfitophobia by 2019 and people were routinely using overparametrized models capable of interpolating their training data while still generalizing well. The Belkin et al. paper wasn't heresy. The authors were making a technical point - that certain theories of generalization are incompatible with this interpolation phenomenon. The lottery ticket hypothesis paper's demonstration of the ubiquity of "winning tickets" - sparse parameter configurations that generalize - is striking, but these "winning tickets" aren't the solutions found by stochastic gradient descent (SGD) algorithms in practice. In the interpolating regime, the minima found by SGD are simple in a different sense perhaps more closely related to generalization. In the case of logistic regression, they are maximum margin classifiers; see https://arxiv.org/pdf/1710.10345 https://arxiv.org/pdf/1710.10345. The article points out some cool papers, but the narrative of plucky researchers bucking orthodoxy in 2019 doesn't track for me.
- ActorNightly 1y agoYeah this article gets a whole bunch of history wrong. Back in 2000s, the reason why nobody was pursuing neural nets was simply due to compute power, and the fact that you couldn't iterate fast enough to make smaller neural networks work. People were doing genetic algorithms and PSO for quite some time. Everyone knew that multi dimentionality was the solution to overfitting - the more directions you can use to climb out of valleys the better the system performed.
- jfrankle 1y agowhyyy
- deleted 1y ago[deleted]
- moi2388 1y agoIsn’t small or large in relation to the amount of data, and the current large models a result of there being so incredibly much data available?