The fact that different algorithms are best for different tasks has more to do with computational complexity. If we didn't care about resource usage then we co

The fact that different algorithms are best for different tasks has more to do with computational complexity. If we didn't care about resource usage then we could just pick a sufficiently expressive model class (e.g. all algorithms) to mix over, and perform Bayesian inference. You can look up AIXI and Solomonoff induction for more thoughts along these lines.