4 ms·

Thanks for the response. I always thought that the point of "making everything symbolic" is that you can differentiate a symbolic expression and thus implement

Thanks for the response. I always thought that the point of "making everything symbolic" is that you can differentiate a symbolic expression and thus implement backpropagation automatically. No? How do you handle this in Spiral?

abstractcontrol 8y ago
Yes, it is, but there exist an alternative approach called automatic differentiation which is more imperative and flexible in nature than symbolic differentiation. The idea is to keep a tape (you can think of it as a list) and then record all the operations on it as you step forward through the program. Then at the end you execute the operations backwards from the tape. I take this approach in Spiral's ML library.