Apologies for the vagueness there. Also apologies for the lack of structure following. I don't have time to write an essay, so I hope you can settle for some me

Apologies for the vagueness there. Also apologies for the lack of structure following. I don't have time to write an essay, so I hope you can settle for some meandering commentary. While the more complex topologies, activation functions, training algorithms are interesting, the current workhorse of ANNs are still plain MLPs (to my knowledge). So, yeah, I was referring mostly to MLPs. The argument I'm doing a pale job voicing is the one that exists between the statistics and ML communities. For those unfamiliar, stat people generally demand comprehensible, general statements about the performance and meaning of the various techniques. This has historically largely restricted the power of statistical techniques. Advances in computing expanded in the ML field because people aimed to solve simpler goals (maximize training and testing predictive accuracy, ignoring why or how it works). This was a major force in the current renaissance in learning technologies, but leaves a big divided camp. One side demands theory, story, and proof. The other just demands results. Which is great. They complement each other. --- ANNs (MLPs in particular) bother me because they're essentially general non-linear discrimination boundary learners. Once you have that decision boundary you can (with the usual enormous concerns toward overfitting) predict new data with arbitrary accuracy given enough compute power, large enough layers, and enough training data. But even with all of those things you won't learn much from your parameter space. Compare this to, for instance, LDA. It's hard to be certain that the usual high-level interpretation of LDA in data mining (that it finds mixtures of topics on documents) is terribly meaningful, but you still are able to learn a lot about the data space by examining the topic space. It's the kind of thing that allows for many interaction points between the algorithm and the users. MLPs of course will also induce a latent representation on the hidden layer. There are really fascinating implementations of non-linear PCA that take this approach, but it's not clear to me what the properties of this latent representation are or how to influence them. --- In some sense, the heart of my argument comes from how flexibility can hurt you in a learning algorithm: it makes your parameter space less meaningful (though I imagine mutable topologies can help this a lot — I have a little experience doing grid search over recurrent nodes and attempting to rank signals by their temporal information using the resultant best-performing topology and I'm sure that rabbit hole goes far deeper). I'm not looking to be awed by a rule based system or GAI (whatever the hell that even means). I want hypothesis spaces which allow you to flexibly describe a question about a large class of data and then learn things from the optimum. I also want these algorithms to solve real world problems. Bayesian modeling gives you a lot of the first one. MLPs give you a lot of the second.