The model needs to be differentiable. The function needs to be continuous. Discontinuous functions are not differentiable. You can approximate discontinuous

The model needs to be differentiable. The function needs to be continuous. Discontinuous functions are not differentiable. You can approximate discontinuous functions with continuous ones (e.g. with logistic functions). In the end, we are still talking about curve fitting and optimization, not artificial general intelligence.