Since we can't find general closed form solutions for simpler equations such as 5th order polynomials, general closed form NN solvers (vs optimizers) do not loo

Since we can't find general closed form solutions for simpler equations such as 5th order polynomials, general closed form NN solvers (vs optimizers) do not look likely. However, constraints on the general class of polynomials of 5th order and higher can result in subclasses that are solvable in closed form. So there may be a long tail of closed form solvable classes of NN architectures waiting to be discovered. Whether non-trivial closed-form subclasses of neural networks exist, and whether any of those classes is actually useful ... seems an area worth exploring.