oteytaud

6 ms·

1.
by oteytaud 8y ago
... incidentally Hyperopt has the advantage of considering conditional domains; we might either do the same or combine Nevergrad with Hyperopt...
2.
by oteytaud 8y ago
For property-based testing I would say yes, with an objective function equal to the margin by which the properties are satisfied. Program synthesis only in some particular cases, like the parametrization of programs for speed or another cri
3.
by oteytaud 8y ago
For small numbers of hyperparameters, sometimes just random search is enough. This is not an absolute rule, sometimes with just 4 parameters random search miserably fails... just my rule of thumb, empirically, is that for hyperparameters
4.
by oteytaud 8y ago
Sure GA can be great for weights as well - but mainly when gradient is unreliable. I would not use Nevergrad for training the weights of a convolutional network for image classification for example; whereas I use Nevergrad for WorldModels.
5.
by oteytaud 8y ago
GA stands for genetic algorithms.
6.
by oteytaud 8y ago
We have not yet released examples of interfaces with Pytorch. Maybe with moderate number of hyperparameters the benefit compared to random search will be moderate, whereas it will be very significant with high number of hyperparameters. It
7.
by oteytaud 8y ago
We have a wide range of experiments on plenty of objective functions in games, reinforcement learning, in real world design and machine learning hyperparameter tuning - these reports will come soon.
8.
by oteytaud 8y ago
To the best of my knowledge, Hyperopt is limited to random search and Parzen variants. We have more algorithms, and include test functions, deal with noise. On the other hand, in Hyperopt conditional variables are naturally handled, whereas
9.
by oteytaud 8y ago
To the best of my knowledge, Hyperopt is limited to random search and Parzen variants. We have more algorithms, and include test functions, deal with noise. On the other hand, in Hyperopt conditional variables are naturally handled, whereas
10.
by oteytaud 8y ago
It's black-box optimization. This means that we just have an objective function, without access to derivatives or whatever other information. This is not relevant for training weights in deep learning for image classification, or other