4 ms·

I hadn't heard of PSO. Looks pretty interesting. What's your use case that you cannot use stochastic gradient descent?

• dnautics 12y ago
  I prefer using PSO versus feed-forward gradient descent because emprically for my system, a particle size of ~100 gives comparable results to standard gradient descent, on a 400-or-so neuron neural net, which is ~4x slower. I can't use backpropagation because some of the output nodes are used to apply a transformation on the input followed by reapplication of the (now transformed) input through the neural net.
  • ihodes 12y ago
    That sounds similar to a recurrent neural net? Backprop definitely works on those with some modifications. I just asked some guys at CMU/Google, they suggested looking more at SGD or some other MCMC methods that may have included the word "expert" or maybe were similar to simulated annealing or Gibb's sampling… which I can't recall… I know nothing about it, and I'm totally unqualified to say anything about them, but maybe (hopefully!) some of those words will lead to something useful! The guys I chatted with don't study PSO, so maybe don't know enough to be helpful, but were a bit down on it in general.