4 ms·

homie it is literally the first sentence in the quoted paragraph >PyTorch (and Chainer) *eschew* this tape; *instead*, every intermediate result records only t

• PartiallyTyped 3y ago
  Did I ever claim that it created a tape? If my comments implied that, then I am sorry for miscommunicating, but by calling it "dynamic" I meant that there is no compiling of a tape or tracking it as done with TF and the likes. > I nternally, a Variable is simply a wrapper around a Tensor that also holds a reference to a graph of Function objects. This graph is an immutable, purely functional representation of the derivative of computed function; Variables are simply mutable pointers to this graph (they are mutated when an in-place operation occurs; see Section 3.1). A Function can be thought of as a closure that has all context necessary to compute vector-Jacobian products. They accept the gradients of the outputs, and return the gradients of the inputs (formally, the left product including the term for their respective operation.) A graph of Functions is a single argument closure that takes in a left product and multiplies it by the derivatives of all operations it contains. The left products passed around are themselves Variables, making the evaluation of the graph differentiable. Please explain how it doesn't hold a graph when it literally says it is holding one, dynamically created and kept alive via ref counting. I am losing my mind here.