4 ms·
Agreed. Besides Geometric Algebra, dual numbers also play a huge role in automatic differentiation -- the core building block of modern machine learning framewo
by alexkcd 7y ago
Agreed. Besides Geometric Algebra, dual numbers also play a huge role in automatic differentiation -- the core building block of modern machine learning frameworks.
I really like the analogy in this talk about how Al-Khwarizmi's six quadratic equations simplify to just one, once we learn about negative numbers and zero.
In a lot of ways, geometric algebra (and dual numbers) are our discovery of "negative numbers and zero", but for the 21st century.
- moralestapia 7y ago>also play a huge role in automatic differentiation Could you extend more on how GA plays a role in this? I wish to learn more but don't know where to start. Any leads would be appreciated greatly.
- spacedome 7y agoIn the video they present it as an algebraic framework where you can "add" to a number system elements x such that x^2 = -1, x^2 = 0, or x^2 = 1. Adding the element x^2 = -1 to the real numbers gives you the complex numbers, with x = i. Adding the element x^2 = 0 to the real numbers gives you the dual numbers with x = epsilon, which is what can be used for automatic differentiation. The case of x^2 = 1 is more complicated.
- alexkcd 7y agoDon't know of a good single resource that is comprehensive. A high level overview of autodiff approaches can be found here: https://www.robots.ox.ac.uk/~tvg/publications/talks/autodiff.pdf https://www.robots.ox.ac.uk/~tvg/publications/talks/autodiff... Some example implementations based on dual numbers: 1. http://www.ceres-solver.org/automatic_derivatives.html http://www.ceres-solver.org/automatic_derivatives.html 2. http://www.imm.dtu.dk/~kajm/FADBAD/ http://www.imm.dtu.dk/~kajm/FADBAD/ Newer ML frameworks do source to source transformations, which allows calculating the derivative without changing the function signature, but the concepts used remain the same.