2 ms·
The idea of "find in which direction function decreases most quickly and go that direction" is really deep, and its implementation via this cutting-edge mathema
by Joker_vD 4y ago
The idea of "find in which direction function decreases most quickly and go that direction" is really deep, and its implementation via this cutting-edge mathematical concept of "gradient" also deserves a whole section as well.
- sillysaurusx 4y agoLook no further: https://jax.readthedocs.io/en/latest/autodidax.html https://jax.readthedocs.io/en/latest/autodidax.html "Autodidax: JAX core from scratch" walks you through it in detail.
- blagie 4y agoIt's both really shallow and really deep. On one hand, you can explain it to a 5-year-old: Go in the direction which improves things. On the other hand, we have more than a half-century of research on sophisticated mathematical methods for doing it well. The latter isn't really helpful for beginners, and the former is easy to explain. You can't use sophisticated algorithms in either case, for beginners, so you can go with something as dumb as tweak in all directions, and go where it improves most. It will work fine for dummy examples.