2 ms·
I wish I were so optimistic - if you use statistics, then you might remove most of interesting outliers and study just those more stable prevailing characterist
by bitL 10y ago
I wish I were so optimistic - if you use statistics, then you might remove most of interesting outliers and study just those more stable prevailing characteristics and not the problematic ones, like when using Hooke's law for designing buildings while ignoring its limited scope. And physics hits hard math limits while studying dynamic systems (i.e. fractals) anyway.
We can always use Ramsey Theory stating there is no chaos possible and you can always find some regularity in anything, yet if you need to scan 10^132 items to find the order it becomes quickly uninteresting.
- curuinor 10y agoRNN is natively and obviously dynamical but FFNN can also be seen as transient dynamical system. Little Ramsey theory ever used in neural network land, but theory of dynamical systems used from quite a long time ago, from Werbos-BPTT which was explicitly envisioned as a dynamical algorithm to the description of the dynamical instability of gradients in RNN. The statistics of statistical mechanics don't necessarily need to be CLTish sorts of things, you know?
- bitL 10y agoPossibly ;-) I'll study these things in detail soon (hopefully), so far just practical experience with all funny things from DL like self-driving cars, composing music with RNNs etc. but having extensive operations research exp in the past I tend to be careful.