8 ms·
Kolmogorov-Arnold networks may make neural networks more understandable
- itsthecourier 2y agoTL;DR: they are talking about KAN (Kolmogorov-Arnold networks)
- weberer 2y agoYeah. Thankfully, HN updated the title to be more descriptive. (Old title was "Novel Architecture Makes Neural Networks More Understandable")
- RustySpottedCat 2y agoCan someone explain exactly what is the "unknown" of neural networks? We built them, we know what they comprise of and how they work. Yes, we can't map out every single connection between nodes in this "multilayer perceptron" but don't we know how these connections are formed?
- taneq 2y agoThere’s a ton of research going into analysing and reverse engineering NNs, this “they’re mysterious black boxes and forever inscrutable” narrative is outdated.
- lupire 2y agoWe don't know what each connection means, what information is encoded in each weight. We don't know how it would behave differently if each of the million or trillion weights was changed. Compare this to dictionaey, where it's obvious what information is on each page and each line.
- wslh 2y agoThe brain serves as a useful analogy, even though LLMs are not brains. Just as we can’t fully understand how we think by merely examining all of our neurons, understanding LLMs requires more than analyzing their individual components, though decoding LLMs is most likely easier, which doesn't mean easy.
- famouswaffles 2y agoSota LLMs like GPT-4o can natively understand b64 encoded text. Now we have algorithms that can decode and encode b64 text. Is that what GPT-4o is doing ? Did training learn that algorithm ? Clearly not or at least not completely because typos in b64 that would destroy any chance of extracting meaning in the original text for our algorithms are barely an inconvenience for 4o. So how is it decoding b64 then ? We have no idea. We don't built Neural Networks. Not really. We build architectures and then train them. Whatever they learn is outside the scope of human action beyond supplying the training data. What they learn is largely unknown beyond trivial toy examples. We know connections form, we can see the weights, we can even see the matrices multiplying. We don't know what any of those calculations are doing. We don't know what they mean. Would an alien understand C Code just because he could see it executing ?
- HarHarVeryFunny 2y agoBase64 encoding is very simple - it's just taking each 6-bits of the input and encoding (replacing) it as one of the 64 (2^6) characters A-Za-z0-9+/. If the input is 8-bit ASCII text, then each 3 input characters will be encoded as 4 Base64 characters (3 * 8 = 24 bits = 4 * 6-bit Base64 chunks). So, this is very similar to an LLM having to deal with tokenized input, but instead of sequences of tokens representing words you've got sequences of Base64 characters representing words.
- famouswaffles 2y agoIt's not about how simple B64 is or isn't. In fact i chose a simple problem we've already solved algorithmically on purpose. It's that all you've just said, reasonable as it may sound is entirely speculation. Maybe "no idea" was a bit much for this example but any idea certainly didn't come from seeing the matrices themselves fly.
- HarHarVeryFunny 2y agoHuh? I just pointed out what Base64 encoding actually is - not some complex algorithm, but effectively just a tokenization scheme. This isn't speculation - I've implemented Base64 decode/encode myself, and you can google for the definition if you don't believe I've accurately described it!
- spencerchubb 2y agoWe know the process to train a model, but when a model makes a prediction we don't know exactly "how" it predicts the way it does. We can use the economy as an analogy. No single person really understands the whole supply chain. But we know that each person in the supply chain is trying to maximize their own profit, and that ultimately delivers goods and services to a consumer.
- Lerc 2y agoWe know how they are formed(and how to form them), we don't know why forming in that particular way solves the problem at hand. Even this characterization is not strictly valid anymore, there is a great deal of research into what's going on inside the black box. The problem was never that it was a black box(we can look inside at any time), but that it was hard to understand. KANs help some of that be placed into mathematical formulation. Generating mappings of activations over data similarly grants insight.
- mjburgess 2y ago* Given the training data, and the architecture of the network, why does SGD with backprop find the given f? vs. any other of an infinite set. * Why are there are a set of f each with 0-loss that work? * Given the weight space, and an f within it, why/when is a task/skill defined as a subset of that space covered by f? I think a major reasons why these are hard to answer is that it's assumed that NNs are operating within an inferential statistical context (ie., reversing some latent structure in the data). But they're really bad at that. In my view, they are just representation-builders that find proxy representations in a proxy "task" space (def, aprox, proxy = "shadow of some real structure, as captured in an unrelated space").
- _navierstokes 2y agoSkipping some detail: the model applies many high-dimensional functions to the input, and we don't know the reasoning for why these functions solve the problem. Reducing the dimension of the weights to human-readable values is non-trivial, and multiple neurons interact in unpredictable ways. Interpretability research has resulted in many useful results and pretty visualizations[1][2], and there are many efforts to understand Transformers[3][4] but we're far from being able to completely explain the large models currently in use. [1] - https://distill.pub/2018/building-blocks/ https://distill.pub/2018/building-blocks/ [2] - https://distill.pub/2019/activation-atlas/ https://distill.pub/2019/activation-atlas/ [3] - https://transformer-circuits.pub/ https://transformer-circuits.pub/ [4] - https://arxiv.org/pdf/2407.02646 https://arxiv.org/pdf/2407.02646
- xiaodai 2y agoIt doesn't that's the problem
- mansoor_ 2y agoNot really. For a trivial function fitting problem, a KAN will allow you to visualise the contribution of each base function into the next layer of your network. Still, these trivial shallow networks are the ones nobody needs to introspect. A deep NN will not be explainable using this approach.
- Taikonerd 2y agoYeah. I'm not sure if anything with millions or billions of parameters will ever be "explainable" in the way we want. I mean, imagine a regular multivariable function with billions of terms, written out on a (very big) whiteboard. Are we ever really going to understand why it produces the numbers it does? KANs may have an order of magnitude fewer parameters, but the basic problem is still the same.
- afiori 2y agoI found these articles very interesting in the context of future ways to understand LLM/AIs https://www.astralcodexten.com/p/the-road-to-honest-ai https://www.astralcodexten.com/p/the-road-to-honest-ai https://www.astralcodexten.com/p/god-help-us-lets-try-to-understand https://www.astralcodexten.com/p/god-help-us-lets-try-to-und...
- etiam 2y agoGood points. Personally I'm still basically with Geoff Hinton's early conjecture that people will have to choose whether they want a model that's easy to explain or one that actually works as well as it could. I'd imagine the really big whiteboard would often be understandable in principle, but most people wouldn't be very satisfied at having the model go "Jolly good. Set aside the next 25 years in your calendar then, and tell me when you're ready to start on practicing the prerequisites!". On the other hand, one might question how often we really understand something complex ostensibly "explained" to us, rather than just gloss over real understanding. A lot of the time people seem to act as if they don't care about really knowing it, and just (hopefully!) want to get an inkling what's involved and make sure that the process could be demonstrated not to be seriously flawed. The models are being held to standards that are typically not applied to people nor to most traditional software. But sure, there are also some real issues about reliability, trust and bureaucratic certifications.
- stefanpie 2y agoThe main author of KANs did a tutorial session yesterday at MLCAD, an academic conference focused on the intersection of hardware / semiconductor design and ML / deep learning. It was super fascinating and seems really good for what they advertise it for, gaining insight and interpret for physical systems (symbolic expressions, conserved quantities , symmetries). For science and mathematics this can be useful but for engineering this might not be the main priority of an ML / deep learning (to some extent). There are still unknowns for leaning hard tasks and learning capacity over harder problems. Even choices in for things like the chosen basis function used for the KAN “activations” and what other architectures these layers can be plugged into with some gain is still unexplored. I think as people mess around with KANs we’ll get better answers to these questions.
- light_hue_1 2y agoThey cannot. Just because one internal operation is understandable, doesn't imply that the whole network is understandable. Take even something much simpler: decision trees. Textbooks give these as an example of understandable systems. A tree where you make one decision based on one feature at a time then at the leaves you output something. Like a bunch of if statements. And in the 90s when computers were slow and trees were small this was true. Today massive decision trees and approaches like random forests can create trees with millions of nodes. Nothing is interpretable about them. We have a basic math gap when it comes to understanding complex systems. Yet another network type solves nothing.
- ImHereToVote 2y agoA formula or equation that enables you to reason about complex systems might simply not exists. It could very well be that to reason about complexity forces you to actually do the complexity.
- empath75 2y agoEven extremely complicated decision trees are interpretable to some extent because you can just walk through the tree and answer questions like: "If this had not been true, would the result have been different?". It may not be possible to hold the entire tree in your head at once, but it's certainly possible to investigate the tree as needed to understand the path that was taken through it.
- svboese 2y agoBut couldn‘t the same be said about standard MLPs or NNs in general?
- empath75 2y ago_Sometimes_, and people do find features in neural networks by tweaking stuff and seeing how the neurons activate, but in general, no. Any given weight or layer or perceptron or whatever can be reused for multiple purposes and it's extremely difficult to say "this is responsible for that", and if you do find parts of the network responsible for a particular task, you don't know if it's _also_ responsible for something else. Whereas with a decision tree it's pretty simple to trace causality and tweak things without changing unrelated parts of the tree. Changing weights in a neural network leads to unpredictable results.
- empath75 2y agoI have a question, which might not even be related to this -- one of the keys to the power of neural networks is exploiting the massive parallelism enabled by GPUs, but are we leaving some compute on the table by using just scalar weights? What if instead of a matrices of weights, what if they were matrices of functions?
- mglz 2y agoGPUs are optimized for matrices of floating point values, so current neural networks use this as a basis (with matrices containing the scalar weights).
- immibis 2y agoEach row/column (I always forget which way around matrices go) of weights followed by a nonlinearity is a learnable function.
- dahart 2y agoThey way to think about NNs is that they are already made of functions; groups of layered nodes become complex nonlinear functions. For example a small 3-layer network can learn to model a cubic spline function. The internals of the function are learned at every step of the way; every addition and multiplication. You can assume the number of functions in a network is a fraction of the number of weights. This makes the NN theoretically more flexible and powerful than modeling it using more complex functions, because it learns and adapts each and every function during training. I would assume its possible using certain functions to, say, model a small fixed-function MLP could perhaps result in more efficient training, if we know the right functions to use. But you could end up losing perf too if not careful. I’d guess the main problems are we don’t know what functions to use, and adding nonlinear functions might come with added difficultly wrt performance and precision and new modes of initialization and normalization. Linear math is easy and powerful and already capable of modeling complex functions, but nonlinear math might be useful I’d guess… needs more study! ;)
- ocular-rockular 2y agoWhat you're describing is very similar to deep Gaussian processes.
- esafak 2y agoRecently discussed in https://news.ycombinator.com/item?id=40219205 https://news.ycombinator.com/item?id=40219205
- 752963e64 2y ago[dead]
- IWeldMelons 2y agoFad.
- CamperBob2 2y agoWhat evidence would change your mind?
- IWeldMelons 2y agoEvidence? I am a simple person. At least a working and more efficient reimplementation of AlexNet.
- CamperBob2 2y agoAlexNet was really what kicked off the latest round of advances in ML. When was that, 2011? 2012? It has obviously been blown into the weeds by later work, at least from a raw performance standpoint, but how would you measure or compare its efficiency with current models? Let's see some numbers.
- throwaway2562 2y agoThe point on interpretability is scientific applications is in symbolic regression - MLPs cannot always spit out an equation for some data set: KANs can.
- buildbot 2y agoI thought that MLPs are universal function approximators? https://en.wikipedia.org/wiki/Universal_approximation_theorem https://en.wikipedia.org/wiki/Universal_approximation_theore...
- tomrod 2y agoOne of the beautiful cosmic injustices about mathematics is that proofs about objects need not be constructive to prove existence.
- js8 2y agoI don't what KANs are, but from the informal description in the article "turn function on many variables into many functions of single variable", it sounds reminiscent of lambda calculus.
- samus 2y agoNope, that's just currying and/or partial application.
- triclops200 2y agoThe (semi) automatic simplification algorithm provided in the paper for KANs seem, to me, like they're solving a similar problem to https://arxiv.org/pdf/2112.04035 https://arxiv.org/pdf/2112.04035, but with the additional constraint of forward functional interpretability as the goal instead of just a generalized abstraction compressor.
- akasya_lisse 2y ago[dead]
- the_precipitate 2y ago[dead]