3 ms·

> ... Designing neural net architectures seems like exactly the sort of problem that neural nets ought to beat humans at handily. I'd be surprised if some kind

> ... Designing neural net architectures seems like exactly the sort of problem that neural nets ought to beat humans at handily. I'd be surprised if some kind of self-optimizing architecture wasn't standard in a decade or two. wouldn't evolutionary algorithms be better suited for this ? evolution is how we got here after all ? perhaps different constraints on the 'state space' can 'guide' the whole process...

buffalobuffalo 3y ago
There are several evolutionary algorithms that are used to design network topography as well as weights. NEAT is a popular one: https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_t... There are a few problems with this approach though. For one, there are often many local maxima that a genetic algorithm can easily converge on. Secondly, the state space is often enormous, and testing/discarding millions of generations of poorly constructed networks is a very inefficient way of sampling it. These problems are often enough to prohibit GA use.
- mnky9800n 3y ago
  > There are a few problems with this approach though. For one, there are often many local maxima that a genetic algorithm can easily converge on. > Secondly, the state space is often enormous, and testing/discarding millions of generations of poorly constructed networks is a very inefficient way of sampling it. This is true for the living as well.
- signa11 3y ago
  i went over these resources, and what strikes me as kind of odd, is the fact that the folks involved in this research / exploration tried to evolve both the architecture and the weights. i was naively thinking that architecture should be evolved, while weights should be computed just as it is being done now i.e. via back-propogation etc.
- falcor84 3y ago
  I suppose that eventually, we'll have the metric being actual "money" that the neural nets try to earn (by mining crpyto, or solving captchas, etc.), such they'd have to pay for their own infrastructure costs. It would then be up to each of them to decide if they'd like to spawn "children", and then decide on an appropriate revenue-share model with them (maybe even pyramid schemes).
onesphere 3y ago
If evolution is a response to the environment, then you might say the environment also evolves with the organism, and that a neutral net is merely trying to respond to its logical environment, in a logical way, to maybe repeat (or regenerate) the environment that created it?

buffalobuffalo 3y ago

There are several evolutionary algorithms that are used to design network topography as well as weights. NEAT is a popular one: https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_t... There are a few problems with this approach though. For one, there are often many local maxima that a genetic algorithm can easily converge on. Secondly, the state space is often enormous, and testing/discarding millions of generations of poorly constructed networks is a very inefficient way of sampling it. These problems are often enough to prohibit GA use.

mnky9800n 3y ago
> There are a few problems with this approach though. For one, there are often many local maxima that a genetic algorithm can easily converge on. > Secondly, the state space is often enormous, and testing/discarding millions of generations of poorly constructed networks is a very inefficient way of sampling it. This is true for the living as well.
signa11 3y ago
i went over these resources, and what strikes me as kind of odd, is the fact that the folks involved in this research / exploration tried to evolve both the architecture and the weights. i was naively thinking that architecture should be evolved, while weights should be computed just as it is being done now i.e. via back-propogation etc.
falcor84 3y ago
I suppose that eventually, we'll have the metric being actual "money" that the neural nets try to earn (by mining crpyto, or solving captchas, etc.), such they'd have to pay for their own infrastructure costs. It would then be up to each of them to decide if they'd like to spawn "children", and then decide on an appropriate revenue-share model with them (maybe even pyramid schemes).

onesphere 3y ago

If evolution is a response to the environment, then you might say the environment also evolves with the organism, and that a neutral net is merely trying to respond to its logical environment, in a logical way, to maybe repeat (or regenerate) the environment that created it?