3 ms·

See https://en.m.wikipedia.org/wiki/Subderivative https://en.m.wikipedia.org/wiki/Subderivative

• throwawayiionqz 7y ago
  The derivative of x->x (identity) should be 1 regardless of your definition of derivative. Subgradients are only applicable when summing convex functions. Here relu(x) - relu(-x) is a sum of a convex function and a concave function.