3 ms·
How can it be both general, and fast? For example, CuDNN ops are fast because they are very specialized and highly tuned. Is convolution written with TC going t
by p1esk 9y ago
How can it be both general, and fast? For example, CuDNN ops are fast because they are very specialized and highly tuned. Is convolution written with TC going to be as fast as CuDNN convolution?
Or, if TC's strength is in its generality, then what are the advantages over something like CuPy for Chainer?
Can someone give an example where TC shines?
- ezyang 9y agoWith cuDNN, some expert had to sit down and write an optimized kernel for your ops. With TC, the idea is you write the simple, tensor comprehension for your op, and then throw the autotuner at it, and get something that approaches or even beats the hand-tuned example. The diagrams in the paper give a sense for some of these situations.
- ozinenko 9y agoSection 7 of the paper (https://arxiv.org/abs/1802.04730 https://arxiv.org/abs/1802.04730) has a couple of examples. In short, yes CuDNN is fast for the cases it was tuned for. It is probably faster on power-of-two sizes, but when you operate on a 26 x 1024954 x 3 tensor, TC can generate specialized code. Want 42 x 17 x 5? TC can generate differently specialized code. With almost no effort from the user (or performance engineers). Can a performance expert do better job than TC optimizer? Very likely yes, but it will very likely take much more time. TC is not a framework. It can be integrated with any framework of your liking.