3 ms·
http://www.denizyuret.com/2015/02/beginning-deep-learning-with-500-lines.html http://www.denizyuret.com/2015/02/beginning-deep-learning-wi... This guy did neur
by dnautics 10y ago
http://www.denizyuret.com/2015/02/beginning-deep-learning-with-500-lines.html http://www.denizyuret.com/2015/02/beginning-deep-learning-wi...
This guy did neural nets with dual numbers in julia and found it to take ~1.2 times as long as standard backpropagation, which suggests that if you had dedicated hardware for it it would be very nice.
- imurray 10y agoI think you're assuming that "automatic differentiation" (AD) is the same thing as "using dual numbers", but AD is a family of methods including dual numbers. The blog post you link to says they're using https://github.com/denizyuret/AutoGrad.jl https://github.com/denizyuret/AutoGrad.jl which uses reverse-mode differentiation not Dual numbers. That means they're performing the same operations (or nearly the same) as standard backpropagation. Dual numbers are just the wrong approach for neural nets, which have many parameters. The amazing thing about backprop / reverse-mode differentiation is that you get the derivatives wrt all the weights in one reverse sweep through the network.
- dnautics 10y agoHuh. I stand corrected. I thought for sure the package used automatic differentiation using dual numbers.