3 ms·
Many comment here are about the first section (which is indeed rediscovering the basis of optimisation) but I find the second section much more interesting. He
by nestorD 6y ago
Many comment here are about the first section (which is indeed rediscovering the basis of optimisation) but I find the second section much more interesting.
He notes that computing the gradient is much more expensive than doing a single evaluation and that the gradient tend to stay in the same overall direction. Thus he proposes to compute the gradient only a fraction of the time and to use normal evaluations in the meantime.
Following an old gradient and only updating it when it becomes clearly obsolete seems like an interesting idea to me.
- cl3misch 6y ago> Following an old gradient and only updating when it becomes clearly obsolete This sounds exactly like a linesearch https://en.m.wikipedia.org/wiki/Line_search https://en.m.wikipedia.org/wiki/Line_search
- angel_j 6y agoThat is the basic premise of Policy Gradients for multi-agent models. Each agent is assigned one gradient, which they train on until end. This replaces the updating of gradients in distributed systems; and policies seem to work well for Reinforcement Learning.