RNN is natively and obviously dynamical but FFNN can also be seen as transient dynamical system. Little Ramsey theory ever used in neural network land, but the

RNN is natively and obviously dynamical but FFNN can also be seen as transient dynamical system. Little Ramsey theory ever used in neural network land, but theory of dynamical systems used from quite a long time ago, from Werbos-BPTT which was explicitly envisioned as a dynamical algorithm to the description of the dynamical instability of gradients in RNN. The statistics of statistical mechanics don't necessarily need to be CLTish sorts of things, you know?