4 ms·
If I read the paper correctly, it seems to support the old quip about all AI being decision trees, at least for smaller model sizes. It also raises an interest
by pushfoo 3y ago
If I read the paper correctly, it seems to support the old quip about all AI being decision trees, at least for smaller model sizes.
It also raises an interesting UX question: is there an implicit tradeoff between legibility and power for notations as for different ways of expressing rotation, or is that a consequence of using graphics hardware to implement AI?
- PaulHoule 3y agoA RelU network looks a bit like a network of decision trees, that’s for sure.
- jszymborski 3y agoWhen I read "Dead Neuron" I definitely associated it with ReLUs. It's claimed that this tendency to create dead neurons has a regularizing effect, but frankly I don't buy that. Anecdotally, I've always had better results with Leaky ReLUs or Mish.