An overlooked advantage of the MLP is that it is differentiable. Essentially, one trades the extra CPU for a classifier that one can pipe gradients through. Tha

An overlooked advantage of the MLP is that it is differentiable. Essentially, one trades the extra CPU for a classifier that one can pipe gradients through. That can be extremely useful in larger NN architectures.