Add: ah I see what you mean. In uni differential calculus we learned about differentiable from above(1), from below(0), so I assumed RELU is considered "differe

Add: ah I see what you mean. In uni differential calculus we learned about differentiable from above(1), from below(0), so I assumed RELU is considered "differentiable", as opposed to Cantor set or Dirichlet / Thomae's function...