2 ms·

I think this is a very interesting project. I am curious what direction you are going. Does it support dilated convolutions, like Wavenet uses? Can it implem

I think this is a very interesting project. I am curious what direction you are going. Does it support dilated convolutions, like Wavenet uses? Can it implement transformers?

37ef_ced3 6y ago
As for dilated (and also grouped, like ResNeXt) convolutions, there is full support The dilated (general-purpose, fallback) convolution algorithm executes FMADDs not much slower than the 1x1 convolution. It makes slow im2col-style approaches unnecessary. Example here: https://nn-512.com/example/4 https://nn-512.com/example/4 The idea is to split the input and dilated weight tensors up according to the stride, and then do 1x1 convolutions with accumulation at heightwise and widthwise offsets No transformers at the moment. You can see what is supported here: https://nn-512.com/docs/graph https://nn-512.com/docs/graph