3 ms·

Respectfully, We've had "learning-to-learn" algorithms for a few years now. Including LSTMs that can be learn the gradients for other LSTMs, and deep-RL algori

Respectfully, We've had "learning-to-learn" algorithms for a few years now. Including LSTMs that can be learn the gradients for other LSTMs, and deep-RL algorithms that can optimize neural networks. Its hard to say its inventing new things... there is still a clear goal - a loss - and we are optimizing it; poorly in the case of genetic algorithms. I would say genetic methods are simply poorly described reinforcement learning problems, which means that there is a 1-to-1 mapping between Deep Learning and Genetic Algorithms. PS. Making a distinction between "Genetic Algorithms" and "Genetic Programming" is like calling Deep Learning "Differential Programming" -- changing the name of a thing does not change the thing itself.

antirez 9y ago
I don't want to mean that this is "novel" at all, I played a lot with genetic programming like 20 years ago. However in my opinion genetic algorithms and genetic programming are not the same thing, while closely related: genetic programming is using genetic algorithms as a search strategy to find a way to write a program in order to solve a problem. It is more closely related to reinforcement learning in theory, you just have a problem, a fitness function, and you provide no hints about how to solve it. Note how this is fundamentally different than having already an algorithm in lack of good parameters, which is what GA does. When such parameters are computer instructions, things start to be semantically interesting. However there is a big difference between reinforcement learning and genetic programming: the second will output a program that can be simplified and understood. NNs are much more opaque so even when they outperform known techniques, what they do is not clear. A more practical example: 20 years ago I wrote a GP framework based on a simple stack language (so that programs are always valid, it's an alternative to use S-expressions). Then I used it in order to generate a new hashing function to minimize the collision I had in my hash table. The output of the GP was a code snippet that I could understand, translate into a specification, re-write in C. It effectively invented a good hashing function. You can see everything as an optimization problem, but at the end of the day if the output of a GP is similar to the output of a mathematician that you hired to write a better hash function, well, this is kinda questionable if it was just optimizing or if while doing it the program invented something. Also note that in the 90s things produced by genetic programming went patented, because they were novel algorithms.
- pmalynin 9y ago
  Right, but what would be the difference if this program was produced by Deep RL algorithm, where the state is the current program (an encoding of the program) and the actions are valid instructions in your stack language? An episode in this case is an iterative call to the Deep RL system until it outputs <STOP> at which step you give it a reward (negative number of collisions say), and before that you can send the actions to your stack machine. Once you're satisfied with the final number, just concatenate all the produced actions to get your final program. I just don't see a fundamental difference, especially once you start doing things like Asynchronous Actor-Critic et al. And then you can start doing Monte Carlo Tree Search on top of that since you have a simulator available...

antirez 9y ago

I don't want to mean that this is "novel" at all, I played a lot with genetic programming like 20 years ago. However in my opinion genetic algorithms and genetic programming are not the same thing, while closely related: genetic programming is using genetic algorithms as a search strategy to find a way to write a program in order to solve a problem. It is more closely related to reinforcement learning in theory, you just have a problem, a fitness function, and you provide no hints about how to solve it. Note how this is fundamentally different than having already an algorithm in lack of good parameters, which is what GA does. When such parameters are computer instructions, things start to be semantically interesting. However there is a big difference between reinforcement learning and genetic programming: the second will output a program that can be simplified and understood. NNs are much more opaque so even when they outperform known techniques, what they do is not clear. A more practical example: 20 years ago I wrote a GP framework based on a simple stack language (so that programs are always valid, it's an alternative to use S-expressions). Then I used it in order to generate a new hashing function to minimize the collision I had in my hash table. The output of the GP was a code snippet that I could understand, translate into a specification, re-write in C. It effectively invented a good hashing function. You can see everything as an optimization problem, but at the end of the day if the output of a GP is similar to the output of a mathematician that you hired to write a better hash function, well, this is kinda questionable if it was just optimizing or if while doing it the program invented something. Also note that in the 90s things produced by genetic programming went patented, because they were novel algorithms.

pmalynin 9y ago
Right, but what would be the difference if this program was produced by Deep RL algorithm, where the state is the current program (an encoding of the program) and the actions are valid instructions in your stack language? An episode in this case is an iterative call to the Deep RL system until it outputs <STOP> at which step you give it a reward (negative number of collisions say), and before that you can send the actions to your stack machine. Once you're satisfied with the final number, just concatenate all the produced actions to get your final program. I just don't see a fundamental difference, especially once you start doing things like Asynchronous Actor-Critic et al. And then you can start doing Monte Carlo Tree Search on top of that since you have a simulator available...