3 ms·
NEAT just doesn't have good, modern GPU powered implementations. NEAT would totally be competitive if someone actually gets a version running in PyTorch/Tensor
by Der_Einzige 6y ago
NEAT just doesn't have good, modern GPU powered implementations.
NEAT would totally be competitive if someone actually gets a version running in PyTorch/Tensorflow
- jawarner 6y agoYou may be interested in this implementation [1] which builds the networks using PyTorch. [1] https://github.com/uber-research/PyTorch-NEAT https://github.com/uber-research/PyTorch-NEAT
- blamestross 6y agoIt uses pytorch (and I'm probably going to use it), but doesn't effectively leverage a GPU for training.
- jawarner 6y agoWhat do you think is the best way to accomplish this?
- blamestross 6y agoYou don't. You need a different parallelism model than a GPU provides. It could work well on machines with very high CPU count, but the speedup on GPUs is the main reason bipartite graph algorithms have seen such investment.
- blamestross 6y agoIt's not that simple. Backpropagating a bipartite graph of nodes works out to a series of matrix operations that parallelize efficiently on a GPU as long as the matrices fit into the GPU's working memory. Running a GA (part of neat) doesn't normally work well on a GPU. The good NEAT algorithms even allow different neurons to have different firing response curves. This inherently defies the "same operation multiple values" style of parallelization in GPUs. The way GPUs work just fundamentally isn't well suited to speeding up NEAT.