3 ms·
> Then they could summarize the entire KAN in an intuitive one-line function (including all the component activation functions), in some cases perfectly reconst
by xg15 2y ago
> Then they could summarize the entire KAN in an intuitive one-line function (including all the component activation functions), in some cases perfectly reconstructing the physics function that created the dataset.
The idea of KANs sounds really exciting, but just to nitpick, you could also write any traditional NN as a closed-form "one line" expression - the line will just become very very long. I don't see how the expression itself would become less complex if you used splines instead of weights (even if this resulted in less neurons for the same decision boundary).
- rsfern 2y agoIn the original KAN paper, they do two things to address this: first they have some sparsity-inducing regularization, and second they have a symbolification step so that you can ideally find a compact symbolic model after learning a sparse computation graph of splines. I guess in principle you could do something similar with MLPs but since MLP representations are sort of delocalized they might be harder to sparsify and symbolify