3 ms·
It does look like the gradient descent is paired with a type of genetic algorithm: > For each local optimization, Urania chooses a target from the pool accordi
by pavon 1y ago
It does look like the gradient descent is paired with a type of genetic algorithm:
> For each local optimization, Urania chooses a target from the pool according to a Boltzmann distribution, which weights better-performing setups in the pool higher and adds a small noise to escape local minima. These choices add a global character to the exploration. When one of the local optimizations of Urania finds a better parameter setting for a setup in the pool, it replaces the old solution with the superior one. Upon convergence, Urania repeats and chooses a new target from the pool. In parallel, Urania simplifies solutions from the pool by probabilistically removing elements whose removal does not impact the overall sensitivity.