5 ms·
Topology generation requires the evaluation of many models - For non-trivial datasets/model sizes the evaluation of a model can be hours, days or weeks, even on
by malux85 10y ago
Topology generation requires the evaluation of many models - For non-trivial datasets/model sizes the evaluation of a model can be hours, days or weeks, even on current GPUs.
There's much better tooling in place, and hardware in place( modern GPUs) now to distribute the evaluation over a cluster of GPU machines, so it's becoming practical to push the boundaries further
edit: I also forgot to mention, older methods often required the manual specification of heuristics for mutation and crossover rules, this is no longer the case, because the machines can be trained with objective functions to prevent overfitting, optimize execution speed or memory usage and can use techniques like dropout, skip connections, layer / node hyperparameters and so on to direct the topology search
- hellofunk 10y agoIs NEAT among the methods you refer to?
- malux85 10y agoYes, but take it way further. i.e. How can you use a Deep Network to direct NEAT? Can you model the topology generation as a latent space, and what would some of the properties of that space be? Can we integrate the best parts of transfer learning to speed up the generate -> evaluate cycle. That sort of thing
- sjg007 10y agoSure sounds reasonable. This feels like Markov chain Monte Carlo.