The pytorch graph model is not as dynamic as you think it is, because batches. All batch elements must be processed by the exact same code trace, limiting per e

The pytorch graph model is not as dynamic as you think it is, because batches. All batch elements must be processed by the exact same code trace, limiting per element dynamism. The dynamic aspect of pytorch is an implementation detail for delivering autograd over a trace of tensor operations. What pytorch does very well is to play nice python/numpy semantics, especially via broadcasting[0]. There is very little cognitive overhead between a pytorch program and its equivalent python/numpy representation. What tensorflow does very well is to execute computational graphs on a wide variety of backends. One gpu, many gpus, distributed gpus, servers, mobile, browsers. The recently announced autograph[1] merges the clarity of python/numpy coding style with tensorflow execution engine, offering autograd via compile-time abstract interpretation over all possible traces. [0] https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html https://docs.scipy.org/doc/numpy/user/basics.broadcasting.ht... [1] https://www.tensorflow.org/guide/autograph https://www.tensorflow.org/guide/autograph