4 ms·

My understanding is that Tensor Comprehensions provides a way to automatically generate optimized CUDA code for algorithms written in a high-level language that

My understanding is that Tensor Comprehensions provides a way to automatically generate optimized CUDA code for algorithms written in a high-level language that more closely mirrors the notation used in mathematical formulas. So you could use it to automatically find more optimal low-level implementations for components used in libraries such as PyTorch and TensorFlow which usually call out to hand-written low-level implementations.

p1esk 9y ago
How can it be both general, and fast? For example, CuDNN ops are fast because they are very specialized and highly tuned. Is convolution written with TC going to be as fast as CuDNN convolution? Or, if TC's strength is in its generality, then what are the advantages over something like CuPy for Chainer? Can someone give an example where TC shines?
- ezyang 9y ago
  With cuDNN, some expert had to sit down and write an optimized kernel for your ops. With TC, the idea is you write the simple, tensor comprehension for your op, and then throw the autotuner at it, and get something that approaches or even beats the hand-tuned example. The diagrams in the paper give a sense for some of these situations.
- ozinenko 9y ago
  Section 7 of the paper (https://arxiv.org/abs/1802.04730 https://arxiv.org/abs/1802.04730) has a couple of examples. In short, yes CuDNN is fast for the cases it was tuned for. It is probably faster on power-of-two sizes, but when you operate on a 26 x 1024954 x 3 tensor, TC can generate specialized code. Want 42 x 17 x 5? TC can generate differently specialized code. With almost no effort from the user (or performance engineers). Can a performance expert do better job than TC optimizer? Very likely yes, but it will very likely take much more time. TC is not a framework. It can be integrated with any framework of your liking.

4 ms·

My understanding is that Tensor Comprehensions provides a way to automatically generate optimized CUDA code for algorithms written in a high-level language that

by davesque 9y ago

p1esk 9y ago
How can it be both general, and fast? For example, CuDNN ops are fast because they are very specialized and highly tuned. Is convolution written with TC going to be as fast as CuDNN convolution? Or, if TC's strength is in its generality, then what are the advantages over something like CuPy for Chainer? Can someone give an example where TC shines?
- ezyang 9y ago
  With cuDNN, some expert had to sit down and write an optimized kernel for your ops. With TC, the idea is you write the simple, tensor comprehension for your op, and then throw the autotuner at it, and get something that approaches or even beats the hand-tuned example. The diagrams in the paper give a sense for some of these situations.
- ozinenko 9y ago
  Section 7 of the paper (https://arxiv.org/abs/1802.04730 https://arxiv.org/abs/1802.04730) has a couple of examples. In short, yes CuDNN is fast for the cases it was tuned for. It is probably faster on power-of-two sizes, but when you operate on a 26 x 1024954 x 3 tensor, TC can generate specialized code. Want 42 x 17 x 5? TC can generate differently specialized code. With almost no effort from the user (or performance engineers). Can a performance expert do better job than TC optimizer? Very likely yes, but it will very likely take much more time. TC is not a framework. It can be integrated with any framework of your liking.