7 ms·
Would applying https://en.wikipedia.org/wiki/Banach_fixed-point_theorem https://en.wikipedia.org/wiki/Banach_fixed-point_theorem yield interesting convergence (
by GistNoesis 2y ago
Would applying https://en.wikipedia.org/wiki/Banach_fixed-point_theorem https://en.wikipedia.org/wiki/Banach_fixed-point_theorem yield interesting convergence (and uniqueness) guarantees ?
- sudosysgen 2y agoThe Banach fixed point theorem is extensively used for convergence proofs in reinforcement learning, but when you operate at the level of gradient descent for deep neutral networks it's difficult to do so because most commonly used optimizers are not guaranteed to converge to a unique fixed point.
- GistNoesis 2y agoThe article seems to do the work to define a Fisher Information metric space, and contractions with the Stein score. Which seems to be the hypothesis for the Banach fixed point theorem, but I am not quite sure what conclusion we would get in this instance.
- nyrikki 2y agoThis will be a good introduction, but won't cover the cost and benefits of a statistical space IID etc.. https://youtu.be/q8gng_2gn70 https://youtu.be/q8gng_2gn70