3 ms·

Could someone please explain how this compares to the TensorFlow approach? I can only assume that it's omitted from the article due to marketing reasons.

smhx 9y ago
Tensor Comprehensions are not a deep learning framework, but a way to write new optimized operators for deep learning. Hence, Tensor Comprehensions can be integrated with TensorFlow.
- liuliu 9y ago
  There is XLA though: https://www.tensorflow.org/performance/xla/ https://www.tensorflow.org/performance/xla/ But I guess unlike TC / Halide / TVM, that is less generic and much less customization.
davesque 9y ago
My understanding is that Tensor Comprehensions provides a way to automatically generate optimized CUDA code for algorithms written in a high-level language that more closely mirrors the notation used in mathematical formulas. So you could use it to automatically find more optimal low-level implementations for components used in libraries such as PyTorch and TensorFlow which usually call out to hand-written low-level implementations.
- p1esk 9y ago
  How can it be both general, and fast? For example, CuDNN ops are fast because they are very specialized and highly tuned. Is convolution written with TC going to be as fast as CuDNN convolution? Or, if TC's strength is in its generality, then what are the advantages over something like CuPy for Chainer? Can someone give an example where TC shines?
  - ezyang 9y ago
    With cuDNN, some expert had to sit down and write an optimized kernel for your ops. With TC, the idea is you write the simple, tensor comprehension for your op, and then throw the autotuner at it, and get something that approaches or even beats the hand-tuned example. The diagrams in the paper give a sense for some of these situations.
  - ozinenko 9y ago
    Section 7 of the paper (https://arxiv.org/abs/1802.04730 https://arxiv.org/abs/1802.04730) has a couple of examples. In short, yes CuDNN is fast for the cases it was tuned for. It is probably faster on power-of-two sizes, but when you operate on a 26 x 1024954 x 3 tensor, TC can generate specialized code. Want 42 x 17 x 5? TC can generate differently specialized code. With almost no effort from the user (or performance engineers). Can a performance expert do better job than TC optimizer? Very likely yes, but it will very likely take much more time. TC is not a framework. It can be integrated with any framework of your liking.

smhx 9y ago

Tensor Comprehensions are not a deep learning framework, but a way to write new optimized operators for deep learning. Hence, Tensor Comprehensions can be integrated with TensorFlow.

liuliu 9y ago
There is XLA though: https://www.tensorflow.org/performance/xla/ https://www.tensorflow.org/performance/xla/ But I guess unlike TC / Halide / TVM, that is less generic and much less customization.

davesque 9y ago

My understanding is that Tensor Comprehensions provides a way to automatically generate optimized CUDA code for algorithms written in a high-level language that more closely mirrors the notation used in mathematical formulas. So you could use it to automatically find more optimal low-level implementations for components used in libraries such as PyTorch and TensorFlow which usually call out to hand-written low-level implementations.

p1esk 9y ago
How can it be both general, and fast? For example, CuDNN ops are fast because they are very specialized and highly tuned. Is convolution written with TC going to be as fast as CuDNN convolution? Or, if TC's strength is in its generality, then what are the advantages over something like CuPy for Chainer? Can someone give an example where TC shines?
- ezyang 9y ago
  With cuDNN, some expert had to sit down and write an optimized kernel for your ops. With TC, the idea is you write the simple, tensor comprehension for your op, and then throw the autotuner at it, and get something that approaches or even beats the hand-tuned example. The diagrams in the paper give a sense for some of these situations.
- ozinenko 9y ago
  Section 7 of the paper (https://arxiv.org/abs/1802.04730 https://arxiv.org/abs/1802.04730) has a couple of examples. In short, yes CuDNN is fast for the cases it was tuned for. It is probably faster on power-of-two sizes, but when you operate on a 26 x 1024954 x 3 tensor, TC can generate specialized code. Want 42 x 17 x 5? TC can generate differently specialized code. With almost no effort from the user (or performance engineers). Can a performance expert do better job than TC optimizer? Very likely yes, but it will very likely take much more time. TC is not a framework. It can be integrated with any framework of your liking.