3 ms·
Can you define a hairy version of a function that misleads at every point? Ie Take a well known function: sin(x). Call it G Now make a function H that makes h
by sagebird 3y ago
Can you define a hairy version of a function that misleads at every point? Ie
Take a well known function: sin(x). Call it G
Now make a function H that makes hairy version of functions. H takes a function f, a slope m, and a probability p.
H returns a version of the function where every value is the close to f(x), and p percent of the points have a derivative of m. The remaining points have the job of undoing the derivative of m and jumping back to f(x).
If you are dealing with real numbers then things can get very hairy without ever deviating from f(x). You could turn p up to 99.999999% and m to a crazy slope.
(I am not a mathematician, I don’t know if these musings are misguided)
My point is that if you had a hairy function, you wouldn’t be able to use gradient decent on it, because a derivative takes the slope at a point, and these slopes are misleading. However, you could use a numerical derivative, that plots the slope between a point and another point separated by a non zero distance D. The hairiness would not be detected.
- CaptainNegative 3y agoTaking g(x) = f(x) + epsilon*cos(bx) adds to the derivatives a function that bounces between -b*epsilon and b*epsilon while g remains within an epsilon envelope of f. Take b large enough to fit your needs, e.g. 100/epsilon times an upper bound on |f'|. Add a scaled version of the Weierstrass function instead if you don't want a derivative at all.
- currymj 3y agoI think for the application here (analogous to autograd on neural networks) you know each individual building block of the function (a well-defined neural network layer) so can be sure such things will not happen.
- deleted 3y ago[deleted]