3 ms·

These are great articles, but I'm hesitant to lump Genetic Algorithms into the rest of machine learning. It's essentially non-convex optimization and one of the

These are great articles, but I'm hesitant to lump Genetic Algorithms into the rest of machine learning. It's essentially non-convex optimization and one of the more popular metaheuristics, but I think of ML as classification / tracking / reinforcement learning. That being said, GA's are a great starting point to get into the world of non-convex optimization. I actually wrote a version of GA's that run on the GPU using WebGL -- the demo runs in the browser: http://petercottle.com/GeneticGPU/index.html http://petercottle.com/GeneticGPU/index.html It's also distributed, so you can send the link to a friend and both browsers will divide up the search space and optimize over their half (and compare minimums). Very hacky but fun project for a grad class at Berkeley.

bkanber 14y ago
In the first article I briefly mention my motivation for covering GAs first when teaching ML: it does a great job of introducing ML core concepts, and it's interesting. It's hard to teach ML to many people if you start with the comparatively boring kNN or k-means; starting with GAs gets people excited early and provides a foundation off of which you can jump into the more traditional ML topics. It shows students that there really are interesting things to learn, and they're more likely to stick it through the "boring" topics. I've been teaching this stuff for a few years, and this approach hasn't failed me yet.
Dn_Ab 14y ago
My definition of Machine learning is more inclusive than yours. What about regression? I suppose you would put that with tracking. Density estimation, Structure learning, Matrix Factorization, Ensemble Methods, Boosting, Clustering or merely estimating a joint distribution? None of those cleanly fit your categories yet are machine learning techniques. You limit to gradient based techniques but why emphasize convex optimization? That leaves out neural nets and places where SGD can do very well. Machine learning is basically the combination of statistics and optimization where you can work with a lot of data and the output of your computation is more important than the model. Here GA and other heuristics such as tabu and annealing are just search/optimization techniques for gradient free or NP-Complete problems.
orensol 14y ago
You can see some real interesting cases of genetic algorithms being used to evolve neural networks on Evolino: http://www.idsia.ch/~juergen/evolino.html http://www.idsia.ch/~juergen/evolino.html

bkanber 14y ago

In the first article I briefly mention my motivation for covering GAs first when teaching ML: it does a great job of introducing ML core concepts, and it's interesting. It's hard to teach ML to many people if you start with the comparatively boring kNN or k-means; starting with GAs gets people excited early and provides a foundation off of which you can jump into the more traditional ML topics. It shows students that there really are interesting things to learn, and they're more likely to stick it through the "boring" topics. I've been teaching this stuff for a few years, and this approach hasn't failed me yet.

Dn_Ab 14y ago

My definition of Machine learning is more inclusive than yours. What about regression? I suppose you would put that with tracking. Density estimation, Structure learning, Matrix Factorization, Ensemble Methods, Boosting, Clustering or merely estimating a joint distribution? None of those cleanly fit your categories yet are machine learning techniques. You limit to gradient based techniques but why emphasize convex optimization? That leaves out neural nets and places where SGD can do very well. Machine learning is basically the combination of statistics and optimization where you can work with a lot of data and the output of your computation is more important than the model. Here GA and other heuristics such as tabu and annealing are just search/optimization techniques for gradient free or NP-Complete problems.

orensol 14y ago

You can see some real interesting cases of genetic algorithms being used to evolve neural networks on Evolino: http://www.idsia.ch/~juergen/evolino.html http://www.idsia.ch/~juergen/evolino.html