11 ms·
How do neural networks learn?
- deleted 3y ago[deleted]
- malux85 3y agoDoes anyone else find reading sites like this on mobile unbearable? The number and size of ads is crazy, in terms of pixel space I think there’s more ads than content
- zipping1549 3y agouBO + Firefox reader mode man.. I just don't read whatever they show me if I don't have adblock handy
- SCUSKU 3y agoTIL about Firefox reader mode! Thank you! Option + Command + R on MacOS, will commit that to memory and use that from now on!
- fransje26 3y agoIf you're fast, you can sometimes even bypass some paywalled articles in reader mode, as some website pre-load their articles before they load a login/paywall screen on top of it.
- tomduncalf 3y agoFirefox Focus seems to block them all, I just use that as a content blocker in Safari and am rarely bothered by ads. I really am not opposed to sites using ads to monetise and resisted ad blocking for many years, but advertisers took it way too far with both the number and intrusiveness of ads so I ended up relenting and installing an ad blocker.
- xcv123 3y agohttps://brave.com https://brave.com No ads
- bongodongobob 3y agoI block that at the DNS level via pihole. Nothing on my network sees that stuff.
- spurgu 3y agoSucks when you leave your comfy dungeon. Hence I use NextDNS to achieve the same thing, anywhere.
- salesynerd 3y agoBlokada works great for me.
- UberFly 3y agoBest not to use the default browser on any device.
- jamboca 3y agoThe article is not so specific on how the research actually works. Then, "We hope this will help democratize AI," but the linked paper is behind a paywall. Curious to see how they would go about explaining how a network selects its important features
- psyklic 3y agoArxiv (2023 paper w/ similar title&authors): https://arxiv.org/abs/2212.13881 https://arxiv.org/abs/2212.13881 Summary by co-author (of the above): https://twitter.com/dbeagleholeCS/status/1627819164906975232 https://twitter.com/dbeagleholeCS/status/1627819164906975232
- zwaps 3y agoBefore I read this, is this yet another paper where physicists believe that modern NN are trained by gradient descent and consist of only linear FNN (or another simplification) and can therefore be explained by <insert random grad level physics method here>? Because we already had hundreds of those. Or is this more substantial?
- pcw7321 3y agoIt's yet another.. just more adds..
- 3abiton 3y agoWhy is spinning a physics lens to NN a bad thing?
- toxik 3y agoIt isn't, but the kinds of paper referred to make so many assumptions that the conclusions lack external validity for actual deep learning. In other words, they typically don't actually say anything about state-of-the-art deep networks.
- gammalost 3y agoWhat do you mean with "modern NN" that differ from the basic FNN? How do you train a modern NN if not through backpropagation?
- zwaps 3y agoFnn are easy for sequential data because there is no relationship between the elements. Hence, these NN are a frequent target of simplified analyses. However, they also never led to exciting models which we now call AI. Instead, real models are an eclectic mix of attention or other sequential mixers, gates, ffn, norms and positional tomfoolery. In other words, everything that makes AI models great is what these analyses usually skip. Of course, while wildly claiming generalized insights about how AI really works. There’s a dozen papers like that every few months.
- 3y ago
- Xcelerate 3y agoWhat would be the requirements in order for most researchers to agree that a “conclusive” answer has been established to the question “How do neural networks work?” I ask not because this paper isn’t insightful research, but rather because if you search Google Scholar or arXiv for papers purporting to describe how neural networks “actually work”, you get thousands upon thousands of results all claiming to answer the question, and yet you never really come away with the sense that the question has truly been resolved in a satisfactory way (see also: the measurement problem). I’ve noticed that each paper uses a totally different approach to addressing the matter that just happens to correspond to the researchers’ existing line of work (It’s topology! No, group theory has the answer! Actually, it’s compressed sensing... computational complexity theory... rebranded old-school quantum chemistry techniques... and so on.) I suppose my question is more about human psychology than neural networks, since neural networks seem to be working just fine regardless of how well we understand them. But I think it could be useful to organize a multi-disciplinary conference where some open questions regarding machine learning are drafted (similar to Hilbert’s problems), that upon their successful resolution would mean neural networks are no longer widely considered “black boxes”.
- sabas123 3y agoWhat do you mean when you say "how they work"? A technique that is applied to widely different fields obviously yield a large set of interpretations each with the lense of their own field. But that doesn't invalidate any of those interpretations no?
- smokel 3y agoThe problem lies in our understanding of the concept "understanding". It is still pretty unclear at a fundamental level what it means to understand, learn, or conceptualize things. This quickly leads to thought about consciousness and other metaphysical issues that have not been resolved, and probably never will be.
- logicchains 3y agoIt was resolved just over 100 years ago by Wittgenstein. Either you fully define "understanding", in which case you've answered your question, or you don't clearly define it, it in which case you can't have a meaningful discussion about it because you don't even have an agreement on exactly what the word means.
- garyiskidding 3y agoThe 2013 Zeiler and Fergus paper also explains this based on activations during training of the network resulting in feature detection across layers. Paper : https://arxiv.org/pdf/1311.2901.pdf https://arxiv.org/pdf/1311.2901.pdf
- richrichie 3y agoIt is funny how they left out the first author of the paper :) Way too complicated name!
- FrustratedMonky 3y agoFrom article "But these networks remain a black box whose inner workings engineers and scientists struggle to understand." "We are trying to understand neural networks from first principles,"" There is a large contingent of CS people in HN that think that since we built AI, and can examine the code, the models, the weights, that this means we understand it. Hope this article helps explain the problem.
- biophysboy 3y agoHere’s a question I’ve been asking myself with the latest ML advancements: what is the difference between understanding and pretending to understand really, really well?
- est 3y agomy 2 cents: understanding is getting know what it is. understanding really well is getting know what it isn't. A "stochastic parrot" can emulate a topic really well, until you push the limits and it fails at corner cases. If a model "understand really well" then it knows exactly where the boundaries are.
- Solvency 3y agoI don't understand.
- s_dev 3y ago>what is the difference between understanding and pretending to understand really, really well? Predicting the outcome of an experiment. A person/thing who understands something, can predict outcomes to a degree. A person pretending to understand cannot predict with any degree of reliability.
- jdthedisciple 3y agoI don't think that is correct. LLMs are literally predicting the outcome of an experiment really well constantly, yet they are best described as pretending to understand really, really well ...
- s_dev 3y agoDoes Newton's Theory of Gravity actually understand gravity or is it just a really good tool for 'predicting outcomes'. The understanding is baked in to the formula and is a reflection of what Newton experienced in reality. Newton's theory is ultimately and slightly wrong but still super useful and many LLMs are basically like this. I can see why all this becomes confusing but I think part of that is when we anthropomorphic words to describe these things that are just math models.
- canjobear 3y agoDoes it explain double descent?
- bilsbie 3y agoAm I wrong or is this a breakthrough?
- cfgauss2718 3y agoBy minimizing a loss functional with respect to a bunch of numbers that amount to entries in matrices (or tensors, whatever) using an approximate hill climbing approach. I’m not sure what insights there are to be gained here, it doesn’t seem more exotic or interesting to me then asking “how does the pseudo inverse of A ‘learn’ to approximate the formula Ax=b?”. Maybe this seems reductive, but once you nail down what the loss functional is (often MSE loss for regression or diffusion models, cross entropy for classification, and many others) and perhaps the particulars of the model architecture (feed-forward vs recurrent, fully connected bits vs convolutions, encoder/decoders) then it’s unclear to me what is left for us to discover about how “learning” works beyond understanding old fundamental algorithms like Newton-Krylov for minimizing nonlinear functions (which subsumes basically all deep learning and goes well beyond). My gut tells me that the curious among you should spend more time learning about fundamentals of optimization than puzzling over some special (and probably non-existent) alchemy inherent in deep networks.
- deleted 3y ago[deleted]
- HarHarVeryFunny 3y agoSure, but their title seems poorly chosen and doesn't match what they are claiming in the article itself, which includes understanding how GPT-2 makes it's predictions. How does GPT-2 learn, for example, that copying a word from way back in the context helps it to minimize the prediction error? How does it even manage to copy a word from the context to the output? We know that it is minimizing prediction errors, and learned to do so via gradient descent, but HOW is it doing it? (we've discovered a few answers, but it's still a research area)
- cfgauss2718 3y agoI haven’t read the manuscript yet, and am not sure that I will. However I don’t agree with the question. Gradient descent, the properties of the loss function are the “how”. It seems like you want to know how some properties of the data are manifested in the network itself during/after training (what these properties are doesn’t seem to be something that people know they are looking for). Maybe that’s what the authors are interested in as well. If I could bet money in Vegas on the answer to that question, my bet would be in most cases that structures we may probe in the network and see in them correlations to aspects of the problem or task that we (as humans) can recognize, well very likely this will boil down to approximations of fundamental and eminently useful quantities like, say, approximate singular value decompositions of regions in the data manifold, or approximate eigenfunctions etc. I could see how these kind of empirical investigations are interesting, but what would their impact be? Another guess, that these investigations may lead to insights that help engineers design better architectures or incrementally improve training methods. But I think that’s about it - this type of research strikes me as engineering and application.
- cs702 3y agoInteresting: Given an input x to a layer f(x)=Wσ(x), where σ is an activation function and W is a weight matrix, the authors define the layer's "neural feature matrix" (NFM) as WᵀW, and show that throughout training, it remains proportional to the average outer product of the layer's gradients, i.e., WᵀW ∝ mean(∇f(x)∇f(x)ᵀ), with the mean computed over all samples in the training data. The authors posit that layers learn to up-weight features strongly related to model output via the NFM. The authors do interesting things with the NFM, including explaining why pruning should even be possible and why we see grokking during learning. They also train a kernel machine iteratively, at each step alternating between (1) fitting the model's kernel matrix to the data and (2) computing the average gradient outer product of the model and replacing the kernel matrix with it. The motivation is to induce the kernel machine to "learn to identify features." The approach seems to work well. The authors' kernel machine outperforms all previous approaches on a common tabular data benchmark.
- cfgauss2718 3y agoThis NFM is a curious quantity . It has the flavor of a metric on the space of inputs to that layer. However, the fact that W’W remains proportional to DfDf’ seems to be an obvious consequence of the very form of f… since Df is itself Ds’WW’Ds, then this should be expected under some assumptions (perhaps mild) on the statistics of Ds, no?
- cs702 3y agoYes, that sounds right, but it doesn't make the work less worthwhile or less interesting. The authors do interesting things with the NFM, including explaining why pruning should even be possible and why we see grokking during learning. They also train a kernel machine iteratively, at each step alternating between (1) fitting the model's kernel matrix to the data and (2) computing the average gradient outer product of the model and replacing the kernel matrix with it. The motivation is to induce the kernel machine to "learn to identify features." The approach seems to work well, outperforming all previous approaches on tabular data. PS. I've updated my comment to add these additional points.
- deleted 3y ago[deleted]
- swalsh 3y agoI've been trying to figure out how LLMs learn logic/reasoning. It's just not intuitive to me how that works.
- mikewarot 3y agoIt's the same way we do it, form a number of possible variants and use the ones that work best. They have the advantage of millions or even billions of times more compute to throw at the learning process. Something that might be a one in a million insight happens consistently at that scale.
- tech_ken 3y agoFull pub is up on arXiv here: https://arxiv.org/pdf/2212.13881.pdf https://arxiv.org/pdf/2212.13881.pdf My 2c: The phys.org summary isn't great. The authors are focused on a much narrower topic than simply "how NNs learn", they're trying to characterize the mechanism by which deep NNs develop 'features'. They identify a quantity which is definable for each layer of the NN (outer product of the input weight matrix), and posit that this quantity is proportional to the average derivative of the layer with respect to its inputs, where the average is taken over all training data. They (claim to, I haven't evaluated) prove this formally for the case of deep FNNs trained on gradient descent. They argue that, by treating this quantity as a measure of 'feature importance', it can be used to explain certain behaviors of NNs that are otherwise difficult to understand. Specifically they address: * Simple and spurious features (they argue that their proposal can identify when this has occurred) * "Lottery ticket" NNs (they argue that their proposal can help explain why pruning the connections of a fully connected NN improves its performance) * Grokking (they argue that their proposal can help explain why NNs can exhibit sudden performance improvements, even after training performance is 100%) Finally they propose a heuristic ML algorithm which updates their proposal directly during training, rather than the underlying weights, and shows that this achieves superior performance to some existing alternatives. Overall I would say that they have defined a nice tool for measuring NN feature importance, in terms of features that the NN itself is defining (not in terms of the original space). I can definitely see why this has a lot of value, and I'm especially intrigued by their comparisons of the NFM to testing performance in their 'grokking' case study. With that said, I'm not really active in the NN space, so it seems a little surprising that their result is really that novel. The quantity they define (outer product of the weight matrix) seems fairly intuitive as a way to rank the importance of inputs at any given layer of the NN, so I'm wondering if truly nobody else has ever done this before? Possibly the novelty is in their derivation of the proportionality, or in the analysis of this quantity over training iterations. I'd guess that their model proposal is totally new, and I'm curious to try it out on some test cases, it seems promising for cases where light-weight models are required. It also seems interesting to point out how both training performance AND the development of feature importance both jointly influence testing accuracy, but again I'm surprised that this is really novel. I also have to wonder how this extends to more complicated architectures with, eg. recursive elements; it's not discussed anywhere, but seems like it would be an important extension of this framework given where genAI is currently at (although first draft was pub'd in '22 so it's possible that this just wasn't as pressing when it was being written).
- mikewarot 3y agoLearning how deep networks learn features isn't obvious, and teasing out the details is valuable research. Superhuman levels of feature recognition are at play during the training of LLMs, and those insights are compiled into the resulting weights in ways we have little visibility into.