For many kinks, it doesn't in practice seem to matter that much. Most applications are computing large stochastic derivatives which smooth out kinks through ave

For many kinks, it doesn't in practice seem to matter that much. Most applications are computing large stochastic derivatives which smooth out kinks through averaging. Critical kinks are those that affect the geometry of the gradient in systematic ways. For instance, a model with a mixture of discrete and continuous parameters. These are serious blockers and require more complex methods to solve such as Rao-Blackwellization (marginalizing out the discrete parameters). Generally this appears as model bias or substantially increased, often fatal variance in loss curves.