If we consider "theory" for "sciences" that can actually measure outcomes, one could argue that they should be possible to formulate as a mathematical model, so

If we consider "theory" for "sciences" that can actually measure outcomes, one could argue that they should be possible to formulate as a mathematical model, somehow. For instance, kinetic energy can be given by: E_k = (gamma - 1) * mc^2 (technically, this is just one part of SR, and the fact that is consistent with the rest of SR is a benefit beyond simply the accuracy) Another "theory" predicting cancer (or some other outcome) could be on the form: P=logistic(sum(beta_i * x_i)), where x_i are identifiable risk factors. The latter would be the theory presumed in logistic regression, and is usually a gross oversimplification, but still very usable. Arguably, this is the same situation that we have with Newtonian kinetic energy (E_k=mv^2/2), which is also known to be a simplification. I think the above direction extends naturally to Neural Networks, where the core assumption (linearity of logit(P) in X) is replaced by some given NN topology. In fact, logistic regression is just an example of (a very simple) such network. In other words, the topology of the network (possibly even abstracted to a set of networks) becomes the new "Theory". If we define "Theory" that way, I would argue that when comparing two such "Theories", and assuming equal explanatory power, the one that requires less computational power (or similarly, fewer parameters) is the better one. As the technology matures, I expect Machine Learning / Neural Networks to be able to re-descover most of the theories where Science seems to be 100% accurate (like General Relativity), and to be able to "learn" the specific solution that minimizes the number of parameters. That solution may indeed by identical to Einstein's formulation. In cases where our best theories are quite poor approximations (ie predicing cancer), machine learning models would be more complex/have many more parameters. I don't think that is a problem, as if we think we have a better understanding with current theories, we are simply fooling ourselves.