3 ms·
I'm glad this is happening and honestly surprised this didn't happen earlier. fchollet, the Keras main author, works at Google and had integrated TensorFlow int
by Smerity 10y ago
I'm glad this is happening and honestly surprised this didn't happen earlier. fchollet, the Keras main author, works at Google and had integrated TensorFlow into Keras very early on (given he'd known about the existence long before the general public). Even though Keras is backend agnostic, you don't really lose any of the flexibility that TensorFlow would give you - it's a pretty transparent abstraction[3]. Best of all, if a new underlying tensor library appears tomorrow that's better than TensorFlow, you'll be getting support for that too!
Keras goes beyond simply being a concise API, a variety of examples, or a strong community with the "best practices included" philosophy. Opinionated but quite useful.
As an example, the settings for an LSTM are complex and require reasonably thorough understanding of many topics. There's dropout (and the many debated ways one could apply it), there's the forget bias, there's weight initialization, there's ... You get the idea.
If you use `keras.layers.recurrent.LSTM` however, bam, you get an opinionated version of these for free.
Initialization is Glorot uniform for most of the weights but then orthogonal[2] for the inner weights.
The forget bias is set to one - as I hope every library has by default now but wasn't the case for some time.
Dropout is variational inference based dropout - recent, likely what you want, and zero complexity.
At some point you'll likely want to learn about all the details - and this provides a smooth easy transition for that as you go "wait, what's a Glorot?" - but for getting your feet wet and/or solving a specific task, "best practices included" seems the best combination. I've successfully recommended this to high school students and they've been up and running with neural networks in short order!
Given all of this, whilst I'm a researcher who works on fiddly novel architectures that require some pretty specific features so use Chainer at work, I turn to Keras for my fun side projects as it keeps me sane and happy :)
Full disclosure, I've committed examples to the Keras codebase and know Francois in person.
[1]: https://keras.io/layers/recurrent/#lstm https://keras.io/layers/recurrent/#lstm
[2]: https://smerity.com/articles/2016/orthogonal_init.html https://smerity.com/articles/2016/orthogonal_init.html
[3]: https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html https://blog.keras.io/keras-as-a-simplified-interface-to-ten...
[4]: http://chainer.org/ http://chainer.org/
- agibsonccc 10y agoWhat does chainer have that keras is missing?
- Eridrus 10y agoChainer has a "define by run" architecture for defining computation graphs, so you can have very dynamic computation graphs. TF/Theano have static computation graphs, though an optimized define by run library is coming to TF in ~February, not sure if there are any plans to make it play nice with Keras though.
- agibsonccc 10y agoOhh that's very similar to ours actually. Our computation graphs just work relative to their context. We've found it very easy to add arbitary vertices because of it. Not that you care, but someone might: We ended up using this style because we opted for not having auto diff in our layers due to speed as well as the state of our framework at the time (layers being hand implemented already). We got the use case for auto diff but not being focused on research we just decided it was easier to hand implement the layers and just have the graphs be defined at the layer level. Thanks for clarifying.