5 ms·
How's that different to e.g. Optuna?
by jmakov 3y ago
How's that different to e.g. Optuna?
- dsign 3y agoNevergrad seems more similar to Nelder Mead optimization, except that Nevergrad is a framework and not a single algorithm, and it has an impressive list of optimizers. Back in the day before autograd optimizers became a commodity, I used Nelder-Mead and the likes heavily. To this day, I still use gradient-free methods for cases where the first derivative is discontinuous (but I suspect that e.g. Pytorch has some black-magic for those cases).
- haltist 3y agoOptimization problems in PyTorch are always parameterized with a smooth manifold structure so they're always differentiable, otherwise gradient descent wouldn't work.