4 ms·
I definitely agree that if you stray outside the "traditional" CNN, LSTM, etc. it is very hard to implement custom layers in most frameworks. Especially if they
by IshKebab 7y ago
I definitely agree that if you stray outside the "traditional" CNN, LSTM, etc. it is very hard to implement custom layers in most frameworks. Especially if they are recurrent. I tried to implement a custom RNN in Tensorflow and gave up. All of the documentation is "just call tf.Lstm()" or whatever.
The exception is CNTK. That makes it really really easy to describe any network including recurrent ones. Sadly it doesn't seem to have caught on at all.
- acollins1331 7y agoTensorflow has pretty poor documentation. Looks like you've been reading the higher level stuff. It's pretty easy to make custom layers and such with tf if you read some of the source. Though, honestly, numpy might be the simplest method to create a custom layers inside an architecture. If you use TF 2.0 this is simple because it doesn't even have to be part of the graph, per se.
- sdenton4 7y agoYou should be able to just inherit from the rnn layer interface, and then use it directly. https://www.tensorflow.org/api_docs/python/tf/keras/layers/RNN https://www.tensorflow.org/api_docs/python/tf/keras/layers/R...
- orbifold 7y agoI agree that it is super painful and confusing to implement RNNs in TensorFlow and had a similar experience starting out. You should take a look at https://www.tensorflow.org/api_docs/python/tf/scan https://www.tensorflow.org/api_docs/python/tf/scan, that is one of the easier ways of implementing one I think. Other than that Pytorch is also great for experimentation and even has a nice example https://pytorch.org/tutorials/advanced/cpp_extension.html https://pytorch.org/tutorials/advanced/cpp_extension.html of implementing a variant of an LSTM. Contributing to this problem is that there is a ton of low quality blog posts on all of these topics.