4 ms·

My surreal moment was reading the Genetic Programming series by John Koza. He discussed a 1000 socket Pentium II cluster which was used for a genetic search. I

My surreal moment was reading the Genetic Programming series by John Koza. He discussed a 1000 socket Pentium II cluster which was used for a genetic search. I did some back of the napkin math and realized that was roughly equivalent to my Zen 2 box. So of course my failure to replicate the author's success was not the fault of my hardware.

qz2 6y ago
Similar situation here. Circa 2000 I had a few separate machines in my lab making lots of noise, taking up lots of space and drinking up power. Now I run a much larger cluster of nodes on my zen 2 desktop in virtual machines. If I need some more I rent them from amazon for a few hours rather than having to go and buy something expensive.
raverbashing 6y ago
Hah interesting. I remember a John Koza book but that was before the PII era I think. Also he liked Lisp (so maybe that's why he needed all that computing) In the end it seems GA/GP was an "overoptimizing" for very non-convex/non-linear search spaces (much more than real problems) and that we're better off with NN/SGD representations of problems
- reitzensteinm 6y ago
  There was a whole series of them, GP I-IV. I believe it was GP II that was discussing the Pentium II cluster. I have had a lot of fun this year building a neural network from scratch as well as genetic programming. For real tasks, it does seem that the former has won out. I have a layman's suspicion that GP will find itself useful in some way and have a resurgence in the future, but it might just be naïve wishful thinking.
walrus01 6y ago
This makes me think of Slashdot jokes from 20 years ago about 'imagine a beowulf cluster of these'. I once worked for a server manufacturer that did a setup with hundreds of 1U size, dual socket Pentium 3, 1.0 GHz CPUs. In the era when the only way to have more than one core was a dual socket motherboard.
chris_st 6y ago
If you like GP, you might want to check out Gene Expression Programming [1, 2]. A lot simpler to implement, with some nicer features. This excellent paper [3] has a lot of great ideas to get inspiration from (it uses GP). [1] https://www.gene-expression-programming.com https://www.gene-expression-programming.com [2] https://en.wikipedia.org/wiki/Gene_expression_programming https://en.wikipedia.org/wiki/Gene_expression_programming [3] https://ccc.inaoep.mx/archivos/CCC-17-009.pdf https://ccc.inaoep.mx/archivos/CCC-17-009.pdf
- reitzensteinm 6y ago
  I will check it out, thank you!

qz2 6y ago

Similar situation here. Circa 2000 I had a few separate machines in my lab making lots of noise, taking up lots of space and drinking up power. Now I run a much larger cluster of nodes on my zen 2 desktop in virtual machines. If I need some more I rent them from amazon for a few hours rather than having to go and buy something expensive.

raverbashing 6y ago

Hah interesting. I remember a John Koza book but that was before the PII era I think. Also he liked Lisp (so maybe that's why he needed all that computing) In the end it seems GA/GP was an "overoptimizing" for very non-convex/non-linear search spaces (much more than real problems) and that we're better off with NN/SGD representations of problems

reitzensteinm 6y ago
There was a whole series of them, GP I-IV. I believe it was GP II that was discussing the Pentium II cluster. I have had a lot of fun this year building a neural network from scratch as well as genetic programming. For real tasks, it does seem that the former has won out. I have a layman's suspicion that GP will find itself useful in some way and have a resurgence in the future, but it might just be naïve wishful thinking.

walrus01 6y ago

This makes me think of Slashdot jokes from 20 years ago about 'imagine a beowulf cluster of these'. I once worked for a server manufacturer that did a setup with hundreds of 1U size, dual socket Pentium 3, 1.0 GHz CPUs. In the era when the only way to have more than one core was a dual socket motherboard.

chris_st 6y ago

If you like GP, you might want to check out Gene Expression Programming [1, 2]. A lot simpler to implement, with some nicer features. This excellent paper [3] has a lot of great ideas to get inspiration from (it uses GP). [1] https://www.gene-expression-programming.com https://www.gene-expression-programming.com [2] https://en.wikipedia.org/wiki/Gene_expression_programming https://en.wikipedia.org/wiki/Gene_expression_programming [3] https://ccc.inaoep.mx/archivos/CCC-17-009.pdf https://ccc.inaoep.mx/archivos/CCC-17-009.pdf

reitzensteinm 6y ago
I will check it out, thank you!