4 ms·

random matrices + svd: http://www.ntu.edu.sg/home/egbhuang/ http://www.ntu.edu.sg/home/egbhuang/

• gcr 12y ago
  I would be very interested to see if this approach outperforms more conventional deep networks trained with backpropagation. It's important to note that (from what I understand) this paper seems to focus on building single-layer NNs, which are quite different from the behemoth AlexNet that this article focuses on.
  • deleted 12y ago
    [deleted]
  • Fede_V 12y ago
    Very early on, people experimented heavily with non-gradient based optimization (genetic algorithms and the like) to optimize neural networks, but the results were rather poor. For very large parameter spaces, gradient based optimization is really the only game in town. There's also lots of clever tricks (gradient clipping, momentum, adagrad, etc) that are involved in optimizing the convergence.