6 ms·

I'm not well-versed in machine learning, so I just want to make sure I understand the general point here: these researchers trained a competitive deep neural ne

I'm not well-versed in machine learning, so I just want to make sure I understand the general point here: these researchers trained a competitive deep neural network without using gradient descent? Does this mean neural networks might become more useful for discrete optimization problems without gradient information, like program synthesis?

deleted 9y ago
[deleted]
Yajirobe 9y ago
Neuroevolution is nothing new. I don't know why people think this is something big/innovative.
- tree_of_item 9y ago
  Because as far as I know, genetic algorithms perform poorly compared to basically every other optimization technique. I'm under the impression that they're not actually used for anything interesting these days, now that the initial hype about implementing natural selection on a computer has died down. Stochastic hill climbing as a baseline method for evaluating genetic algorithms: http://papers.nips.cc/paper/1172-stochastic-hillclimbing-as-a-baseline-method-for-evaluating-genetic-algorithms.pdf http://papers.nips.cc/paper/1172-stochastic-hillclimbing-as-... When will a genetic algorithm outperform hill climbing? http://web.cecs.pdx.edu/~mm/nips93.pdf http://web.cecs.pdx.edu/~mm/nips93.pdf A GA being competitive with modern gradient based methods is very surprising to me.
  - Yajirobe 9y ago
    Try optimizing for the shape of an antenna using hill climbing. Thing about evolutionary networks is that they can evolve topology, which usually is just a thing decided by people implementing NNs (and not guaranteed to be anywhere near most optimal). I don't see why given enough time GAs (or EAs) could not outperform simple backpropagation based on differentiating simple cost functions.
    - crimsonalucard 9y ago
      > I don't see why given enough time GAs (or EAs) could not outperform simple backpropagation based on differentiating simple cost functions. It's already happened. You are it.
      - tree_of_item 9y ago
        This was the line of reasoning that led to the initial hype about GAs, but I don't think it's sound. We still don't understand exactly how natural selection works, so there's no reason to believe the GAs we can implement now have the "special sauce" that makes real world natural selection effective. We don't know how to implement genotypes well, we don't know how to implement recombination or mutation well, etc.
        crimsonalucard 9y ago
        I disagree. The biological brain is a known upper bound for GAs thus we are fully aware of a gap in understanding. This makes it a perfect target for research. Looking at current methods of optimizing neural networks we cannot know if we are far from the upper bound or if we've already reached it... R&D effort in this area could be a dead end.
    - tree_of_item 9y ago
      Why wouldn't you be able to do that with hill climbing?
      - ramgorur 9y ago
        Because you can't establish any gradient information from the design variables (antenna topology) to the objective function (i.e. antenna gain). So most of the time you might end up climbing towards some wrong direction (i.e. local optima).
        tree_of_item 9y ago
        Hill climbing doesn't use a gradient, that's gradient descent. Hill climbing is pretty much the same as GA with a population of 1. There's no reason you can't design an antenna with it, and before this paper I'd be convinced hill climbing would actually do it better. The person I was replying to was also advocating the use of GAs to solve the antenna problem, which don't use gradient information either.
  - ramgorur 9y ago
    What do you do if you don't have any computable "gradient"? Many real world optimization problems don't come with any gradient information.
    - tree_of_item 9y ago
      Yes, I'm aware many problems don't have a gradient. Before this paper I'd have said you don't do anything, you just throw compute at it but still don't really solve it well.
dbecker 9y ago
Yes and maybe.
dragontamer 9y ago
The Blondie24 book was published in 2002. https://www.amazon.com/product-reviews/1558607838 https://www.amazon.com/product-reviews/1558607838 People need to recognize that this is not a new idea. It may have been groundbreaking in the early 2000s, but Genetic Algorithms + Neural Nets are a relatively old strategy at this point. Definitely read the book if you want a bit of a throwback. When cutting-edge AI research was about beating players on Yahoo-games Checkers with new techniques.

deleted 9y ago

[deleted]

Yajirobe 9y ago

Neuroevolution is nothing new. I don't know why people think this is something big/innovative.

tree_of_item 9y ago
Because as far as I know, genetic algorithms perform poorly compared to basically every other optimization technique. I'm under the impression that they're not actually used for anything interesting these days, now that the initial hype about implementing natural selection on a computer has died down. Stochastic hill climbing as a baseline method for evaluating genetic algorithms: http://papers.nips.cc/paper/1172-stochastic-hillclimbing-as-a-baseline-method-for-evaluating-genetic-algorithms.pdf http://papers.nips.cc/paper/1172-stochastic-hillclimbing-as-... When will a genetic algorithm outperform hill climbing? http://web.cecs.pdx.edu/~mm/nips93.pdf http://web.cecs.pdx.edu/~mm/nips93.pdf A GA being competitive with modern gradient based methods is very surprising to me.
- Yajirobe 9y ago
  Try optimizing for the shape of an antenna using hill climbing. Thing about evolutionary networks is that they can evolve topology, which usually is just a thing decided by people implementing NNs (and not guaranteed to be anywhere near most optimal). I don't see why given enough time GAs (or EAs) could not outperform simple backpropagation based on differentiating simple cost functions.
  - crimsonalucard 9y ago
    > I don't see why given enough time GAs (or EAs) could not outperform simple backpropagation based on differentiating simple cost functions. It's already happened. You are it.
    - tree_of_item 9y ago
      This was the line of reasoning that led to the initial hype about GAs, but I don't think it's sound. We still don't understand exactly how natural selection works, so there's no reason to believe the GAs we can implement now have the "special sauce" that makes real world natural selection effective. We don't know how to implement genotypes well, we don't know how to implement recombination or mutation well, etc.
      - crimsonalucard 9y ago
        I disagree. The biological brain is a known upper bound for GAs thus we are fully aware of a gap in understanding. This makes it a perfect target for research. Looking at current methods of optimizing neural networks we cannot know if we are far from the upper bound or if we've already reached it... R&D effort in this area could be a dead end.
  - tree_of_item 9y ago
    Why wouldn't you be able to do that with hill climbing?
    - ramgorur 9y ago
      Because you can't establish any gradient information from the design variables (antenna topology) to the objective function (i.e. antenna gain). So most of the time you might end up climbing towards some wrong direction (i.e. local optima).
      - tree_of_item 9y ago
        Hill climbing doesn't use a gradient, that's gradient descent. Hill climbing is pretty much the same as GA with a population of 1. There's no reason you can't design an antenna with it, and before this paper I'd be convinced hill climbing would actually do it better. The person I was replying to was also advocating the use of GAs to solve the antenna problem, which don't use gradient information either.
- ramgorur 9y ago
  What do you do if you don't have any computable "gradient"? Many real world optimization problems don't come with any gradient information.
  - tree_of_item 9y ago
    Yes, I'm aware many problems don't have a gradient. Before this paper I'd have said you don't do anything, you just throw compute at it but still don't really solve it well.

dbecker 9y ago

Yes and maybe.

dragontamer 9y ago

The Blondie24 book was published in 2002. https://www.amazon.com/product-reviews/1558607838 https://www.amazon.com/product-reviews/1558607838 People need to recognize that this is not a new idea. It may have been groundbreaking in the early 2000s, but Genetic Algorithms + Neural Nets are a relatively old strategy at this point. Definitely read the book if you want a bit of a throwback. When cutting-edge AI research was about beating players on Yahoo-games Checkers with new techniques.