+1. Looking quickly at the backing paper, it's all about higher-order derivatives. As I see it, the grindy-axe is about where and how one makes the hand-off fro

+1. Looking quickly at the backing paper, it's all about higher-order derivatives. As I see it, the grindy-axe is about where and how one makes the hand-off from algebraic notation to actual computation: keeping the calculation in algebraic form allows efficient algebraic manipulations, which can then be translated into low-level computations. The question, then, is: a) whether the space of problems where you have good algebraic notation lines up well with the total scope of TF problems, and b) whether the extra complexity of supporting the full computer algebra system is 'worth it.' For the latter, keep in mind that algebraic derivatives can get cumbersome/expensive when you have an exponentially complex piecewise linear space (eg: https://arxiv.org/pdf/1711.02114.pdf); https://arxiv.org/pdf/1711.02114.pdf); the linked paper makes no mention of ReLUs... things might be fine with sigmoid activations, but they're the exception, these days...