4 ms·
Nice! I have to admit I didn't follow all the math, but the layers get really wide, right? You still need to describe all the partitions between the pieces of t
by sp332 3y ago
Nice! I have to admit I didn't follow all the math, but the layers get really wide, right? You still need to describe all the partitions between the pieces of the piecewise linear functions. Does this save any computation at inference time?
- polygamous_bat 3y agoMy hunch is that you can really easily find (adversarial) counterexamples that may show that in the worst case you will always use more compute by shallow-ing. In my head it should be related to no free lunch theorem.