the gradient doesn't exist but subgradients do exist (at points of non-differentiability) and that is still useful (case in point as someone else mentions ReLU)

the gradient doesn't exist but subgradients do exist (at points of non-differentiability) and that is still useful (case in point as someone else mentions ReLU) https://see.stanford.edu/materials/lsocoee364b/01-subgradients_notes.pdf https://see.stanford.edu/materials/lsocoee364b/01-subgradien...