NNs come with many caveats and lack the theoretical guarantees and support of SVMs and RFs. They should never be the first approach to solving a problem unless

NNs come with many caveats and lack the theoretical guarantees and support of SVMs and RFs. They should never be the first approach to solving a problem unless a) you have good reason to support the hypothesis that other models don't cut it such as a non stationary problem e.g. RL, or b) other methods can't scale to the difficulty of the problem. NNs are also very expensive to train compared to everything else.