You can't just say that things are either adhoc or not adhoc - it's a spectrum. On one end we will have engineers coming up with architectures by consulting a R

You can't just say that things are either adhoc or not adhoc - it's a spectrum. On one end we will have engineers coming up with architectures by consulting a RNG and then testing out those architectures. On the other end we will have a theory, the theory will make 100% good predictions about which designs will perform well, and we can just send send those designs to a factory to fabricate into asics, no testing needed. The adhoc-ness of how we design neural net architectures, CPUs, GPUs, high-temperature suerconductors, etc all fall on this spectrum, just at different places. pron's original comment made it sound like designing NNs were on the extreme ad-hoc end of the spectrum, I was just refuting that.