Automatic differentiation computes with so-called dual numbers, which store an infinitesimal (e) alongside the result of the computation. The infinitesimal is t

Automatic differentiation computes with so-called dual numbers, which store an infinitesimal (e) alongside the result of the computation. The infinitesimal is the derivative. Working with infinitesimals is how calculus started out but was abandoned when calculus was formalized using epsilon-delta limits (http://en.wikipedia.org/wiki/%28%CE%B5,_%CE%B4%29-definition_of_limit http://en.wikipedia.org/wiki/%28%CE%B5,_%CE%B4%29-definition...) The is a newer formalization of calculus, called non-standard analysis, that works with infinitesimals, called the hyperreals. As you can imagine, there is a strong relationship between dual numbers and hyperreals. I found this exchange interesting: http://math.stackexchange.com/questions/341535/is-the-theory-of-dual-numbers-strong-enough-to-develop-real-analysis-and-does-i http://math.stackexchange.com/questions/341535/is-the-theory... It's kinda cool that non-standard analysis slips in through the back door when using dual numbers. I always found working (informally!) with infinitesimals quite simple.