4 ms·

This video is the best thing I watched in the past few years. Last time I loved something that much was Geoffrey Hintons course about deep learning back in 2015

This video is the best thing I watched in the past few years. Last time I loved something that much was Geoffrey Hintons course about deep learning back in 2015. It is eye opening, just like in the beginning of mainstream deep learning, it made me wonder “why didn’t we discover this before?” It elegantly solves so many annoyances about geometry, maths and physics. I’ve been evangelizing Geometric Algebra like crazy since I found this video, including in hacker news comments.

alexkcd 7y ago
Agreed. Besides Geometric Algebra, dual numbers also play a huge role in automatic differentiation -- the core building block of modern machine learning frameworks. I really like the analogy in this talk about how Al-Khwarizmi's six quadratic equations simplify to just one, once we learn about negative numbers and zero. In a lot of ways, geometric algebra (and dual numbers) are our discovery of "negative numbers and zero", but for the 21st century.
- moralestapia 7y ago
  >also play a huge role in automatic differentiation Could you extend more on how GA plays a role in this? I wish to learn more but don't know where to start. Any leads would be appreciated greatly.
  - spacedome 7y ago
    In the video they present it as an algebraic framework where you can "add" to a number system elements x such that x^2 = -1, x^2 = 0, or x^2 = 1. Adding the element x^2 = -1 to the real numbers gives you the complex numbers, with x = i. Adding the element x^2 = 0 to the real numbers gives you the dual numbers with x = epsilon, which is what can be used for automatic differentiation. The case of x^2 = 1 is more complicated.
  - alexkcd 7y ago
    Don't know of a good single resource that is comprehensive. A high level overview of autodiff approaches can be found here: https://www.robots.ox.ac.uk/~tvg/publications/talks/autodiff.pdf https://www.robots.ox.ac.uk/~tvg/publications/talks/autodiff... Some example implementations based on dual numbers: 1. http://www.ceres-solver.org/automatic_derivatives.html http://www.ceres-solver.org/automatic_derivatives.html 2. http://www.imm.dtu.dk/~kajm/FADBAD/ http://www.imm.dtu.dk/~kajm/FADBAD/ Newer ML frameworks do source to source transformations, which allows calculating the derivative without changing the function signature, but the concepts used remain the same.

alexkcd 7y ago

Agreed. Besides Geometric Algebra, dual numbers also play a huge role in automatic differentiation -- the core building block of modern machine learning frameworks. I really like the analogy in this talk about how Al-Khwarizmi's six quadratic equations simplify to just one, once we learn about negative numbers and zero. In a lot of ways, geometric algebra (and dual numbers) are our discovery of "negative numbers and zero", but for the 21st century.

moralestapia 7y ago
>also play a huge role in automatic differentiation Could you extend more on how GA plays a role in this? I wish to learn more but don't know where to start. Any leads would be appreciated greatly.
- spacedome 7y ago
  In the video they present it as an algebraic framework where you can "add" to a number system elements x such that x^2 = -1, x^2 = 0, or x^2 = 1. Adding the element x^2 = -1 to the real numbers gives you the complex numbers, with x = i. Adding the element x^2 = 0 to the real numbers gives you the dual numbers with x = epsilon, which is what can be used for automatic differentiation. The case of x^2 = 1 is more complicated.
- alexkcd 7y ago
  Don't know of a good single resource that is comprehensive. A high level overview of autodiff approaches can be found here: https://www.robots.ox.ac.uk/~tvg/publications/talks/autodiff.pdf https://www.robots.ox.ac.uk/~tvg/publications/talks/autodiff... Some example implementations based on dual numbers: 1. http://www.ceres-solver.org/automatic_derivatives.html http://www.ceres-solver.org/automatic_derivatives.html 2. http://www.imm.dtu.dk/~kajm/FADBAD/ http://www.imm.dtu.dk/~kajm/FADBAD/ Newer ML frameworks do source to source transformations, which allows calculating the derivative without changing the function signature, but the concepts used remain the same.