3 ms·

KANs seem like a great tool for the right job. However, based on my understanding of how they work, my intuitions tell me that they would be awful at image proc

KANs seem like a great tool for the right job. However, based on my understanding of how they work, my intuitions tell me that they would be awful at image processing, which I think was one of the author's test beds.

ein0p 2y ago
On what do you base your intuitions? The way I see it is either they are good universal approximators (like ANNs), and then they’re usable for everything, or they aren’t, and then they’re at best very narrow in application. I’ve yet to see any evidence for the latter.
- eximius 2y ago
  Uninformed response: if two different universal approximators can have different modes/rates of learning, then even if they're usable in theory, they could be less good in practice.
- davesque 2y ago
  Then why would you choose to use a CNN over a fully connected neural network for visual data? For the same reason you'd choose a KAN over a traditional neural network if you were trying to fit a continuous function that can easily be modeled as a combination of b-splines. Machine learning models aren't magic and their internal function very much determines the kinds of problems to which they are well applied. My intuitions about KANs and visual data comes from an impression that it would be hard for a decision boundary on visual data to behave nicely if it could only be built from b-splines. Judging the usefulness of a machine learning architecture is not a matter of determining which architecture will perform the best in all scenarios.
- nyrikki 2y ago
  Universal approximation just indicates existence of a suitable sequence, not that it is findable and doesn't apply with finite numbers of neurons. But MLPs are not good for everything. Where Simulated annealing works better than auto-diff is the classic example that is easier to visualize, at least for me. Even if the sequence 'exists', finding it is the problem, it doesn't matter if a method can represent an unfindable sequence. That said, IMHO, MLP vs KAN is probably safer to think of as horses for courses, they are better at different things. At least with your definition of 'usable' being undefined.

ein0p 2y ago

On what do you base your intuitions? The way I see it is either they are good universal approximators (like ANNs), and then they’re usable for everything, or they aren’t, and then they’re at best very narrow in application. I’ve yet to see any evidence for the latter.

eximius 2y ago
Uninformed response: if two different universal approximators can have different modes/rates of learning, then even if they're usable in theory, they could be less good in practice.
davesque 2y ago
Then why would you choose to use a CNN over a fully connected neural network for visual data? For the same reason you'd choose a KAN over a traditional neural network if you were trying to fit a continuous function that can easily be modeled as a combination of b-splines. Machine learning models aren't magic and their internal function very much determines the kinds of problems to which they are well applied. My intuitions about KANs and visual data comes from an impression that it would be hard for a decision boundary on visual data to behave nicely if it could only be built from b-splines. Judging the usefulness of a machine learning architecture is not a matter of determining which architecture will perform the best in all scenarios.
nyrikki 2y ago
Universal approximation just indicates existence of a suitable sequence, not that it is findable and doesn't apply with finite numbers of neurons. But MLPs are not good for everything. Where Simulated annealing works better than auto-diff is the classic example that is easier to visualize, at least for me. Even if the sequence 'exists', finding it is the problem, it doesn't matter if a method can represent an unfindable sequence. That said, IMHO, MLP vs KAN is probably safer to think of as horses for courses, they are better at different things. At least with your definition of 'usable' being undefined.