4 ms·

Cool idea, but is this of any benefit? Isn't this essentially what TensorFlow does internally, except it inserts CUDA primitives at the right positions...

• perone 10y ago
  Yes it is, and in my opinion is very important. TensorFlow team is actually actively working on a JIT (https://github.com/tensorflow/tensorflow/issues/164 https://github.com/tensorflow/tensorflow/issues/164). I'll paste here a relevant part of the TensorFlow paper regarding the Future Work: "We also have a number of concrete directions to improve the performance of TensorFlow. One such direction is our initial work on a just-in-time compiler that can take a subgraph of a TensorFlow execution, perhaps with some runtime profiling information about the typical sizes and shapes of tensors, and can generate an optimized routine for this subgraph. This compiler will understand the semantics of perform a number of optimizations such as loop fusion, blocking and tiling for locality, specialization for particular shapes and sizes, etc." http://download.tensorflow.org/paper/whitepaper2015.pdf http://download.tensorflow.org/paper/whitepaper2015.pdf
• ragebol 10y ago
  Indeed, some benchmarks would be interesting.
• vrv 10y ago
  TensorFlow doesn't yet do loop fusion (though I believe the specific example shown in that article may already be done via constant folding). But if you have a bunch of elementwise operations, JIT-techniques can reduce the number of memory passes over a buffer. If your model is already very computationally dense (time dominated by matmul or convolution), then this won't help as much, but otherwise, JIT techniques can help.