You can bolt all kinds of funny structures into some DNN system, but if the system doesn't have well behaved gradients (or if it isn't even differentiable) it w

You can bolt all kinds of funny structures into some DNN system, but if the system doesn't have well behaved gradients (or if it isn't even differentiable) it won't train.