3 ms·
'hacky feeling techniques' - as opposed to the rest of DNN research??! More seriously, I wonder if some kind of hybrid approach could be possible/beneficial (e.
by anonymousDan 2y ago
'hacky feeling techniques' - as opposed to the rest of DNN research??! More seriously, I wonder if some kind of hybrid approach could be possible/beneficial (e.g. KANs for some layers?)
- janalsncm 2y agoThat’s a fair criticism but it should count for something that we’ve spent the last 20 years ironing out standard “hacks” to get NNs working. Someday we will hopefully have some theoretical grounding for basic things like network shape and size, optimizer, learning rate schedule, etc. Until that time we will be stuck using empirical methods (read: trial and error) and KANs are at best just another thing to try.