7 ms·
Keras will be added to core TensorFlow at Google
- du_bing 10y agoGreat News!
- kyloon 10y agoThis is amazing news as using Keras is my opinion the best way to start building models in TensorFlow for most people (and also having support for Theano is definitely a plus).
- captainarab 10y agoSomewhat off topic -- HN introduced me to fast.ai and I have found their hands-on/practical approach to teaching particularly useful!
- jph00 10y agoI'm glad you liked it :) As it happens, I literally just now posted the curriculum for part 2 of the course - http://www.fast.ai/2017/01/17/curriculum2/ http://www.fast.ai/2017/01/17/curriculum2/ . If you're near SF, you may want to join us. Either way, I'd love to get feedback on the curriculum - anything you'd like to see added? Anything from part 1 that you'd like to know more about?
- rrherr 10y agoI'm very excited for this in Part 2: "A key teaching goal for us is that you come away from the course feeling much more comfortable reading, understanding, and implementing research papers. We’ll be sharing some simple tricks that make it much easier to quickly scan and get the key insights from a paper." My interest is musical style transfer. I'd like to replicate these examples from Sony Computer Science Lab-Paris: http://www.flow-machines.com/odetojoy/ http://www.flow-machines.com/odetojoy/ They've published papers, but not code (except for DeepBach).
- captainarab 10y agoI am just getting started actually, so I will provide you more feedback as I complete part 1! For starters, I found the pairing of the lecture, the code, AND the documentation to be particularly useful. The setup in anaconda really enables to you compare/understand inputs and outputs, which at least for me, is very helpful! Big fan of learning through practical application, which the aforementioned combo is well suited for, imo. Kudos and thank you (and the team) very much for all of the hard work! I am not sure I'll be able to attend part 2 in person but I will be sure to follow along online. If I am ever in SF at the time of a course, I will certainly apply! All the best.
- chrisfosterelli 10y ago> because TensorFlow’s API is verbose and confusing, and because Keras has the most thoughtfully designed, expressive API I’ve ever experienced This doesn't feel fair to say, just like it wouldn't if you replaced "Tensorflow" with "C" and "Keras" with "Python". They operate on fundamentally different levels and provide a similar trade off in control/ease of use. We started using Keras for a project a few months ago, and it was great while it supported what we were doing. Once we needed to go outside of the box a little bit we essentially had to rewrite it in just Tensorflow. This is great news though! Hopefully it will make the barrier to entry much lower for getting started with Tensorflow.
- taion 10y agoI strongly recommend checking out TensorFlow-Slim if you're looking for a lightweight abstraction for using TensorFlow – at least the layers, anyway. The API looks similar to something like Keras or Lasagne, but the layers are just simple operations on tensors, so it integrates seamlessly with vanilla TensorFlow in a way that something like TF-Keras won't. At the same time, though, the layers are the same foundation that TF-Keras look like they'll use, so you get much of the same expressive power, but without sacrificing flexibility.
- AlexCoventry 10y ago> it integrates seamlessly with vanilla TensorFlow in a way that something like TF-Keras won't. Google also released prettytensor, which is designed to address the same problem. https://github.com/google/prettytensor https://github.com/google/prettytensor
- jph00 10y agoThe Keras functional API are just simple operations on tensors too.
- wodenokoto 10y agoThey are also comparing tf unfavourably to theano. Personally I've found a somewhat obscure low-level and (comparatively) little documented DyNet [[1] much easier to understand than tensorflow, so the critique rings true to me. [1] http://dynet.readthedocs.io http://dynet.readthedocs.io
- Smerity 10y agoI'm glad this is happening and honestly surprised this didn't happen earlier. fchollet, the Keras main author, works at Google and had integrated TensorFlow into Keras very early on (given he'd known about the existence long before the general public). Even though Keras is backend agnostic, you don't really lose any of the flexibility that TensorFlow would give you - it's a pretty transparent abstraction[3]. Best of all, if a new underlying tensor library appears tomorrow that's better than TensorFlow, you'll be getting support for that too! Keras goes beyond simply being a concise API, a variety of examples, or a strong community with the "best practices included" philosophy. Opinionated but quite useful. As an example, the settings for an LSTM are complex and require reasonably thorough understanding of many topics. There's dropout (and the many debated ways one could apply it), there's the forget bias, there's weight initialization, there's ... You get the idea. If you use `keras.layers.recurrent.LSTM` however, bam, you get an opinionated version of these for free. Initialization is Glorot uniform for most of the weights but then orthogonal[2] for the inner weights. The forget bias is set to one - as I hope every library has by default now but wasn't the case for some time. Dropout is variational inference based dropout - recent, likely what you want, and zero complexity. At some point you'll likely want to learn about all the details - and this provides a smooth easy transition for that as you go "wait, what's a Glorot?" - but for getting your feet wet and/or solving a specific task, "best practices included" seems the best combination. I've successfully recommended this to high school students and they've been up and running with neural networks in short order! Given all of this, whilst I'm a researcher who works on fiddly novel architectures that require some pretty specific features so use Chainer at work, I turn to Keras for my fun side projects as it keeps me sane and happy :) Full disclosure, I've committed examples to the Keras codebase and know Francois in person. [1]: https://keras.io/layers/recurrent/#lstm https://keras.io/layers/recurrent/#lstm [2]: https://smerity.com/articles/2016/orthogonal_init.html https://smerity.com/articles/2016/orthogonal_init.html [3]: https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html https://blog.keras.io/keras-as-a-simplified-interface-to-ten... [4]: http://chainer.org/ http://chainer.org/
- agibsonccc 10y agoWhat does chainer have that keras is missing?
- ma2rten 10y agoThe blog post links to a tweet and that tweet has a couple more details. The new keras is going be tensorflow-only and build on top of tf.contrib.layers (so a complete rewrite). https://twitter.com/fchollet/status/820746845068505088 https://twitter.com/fchollet/status/820746845068505088
- mgreg 10y agoIt actually sounds like Keras as we know it will continue to exist and support both Theano and TF. Theano support will continue for as long as Keras exists. This integration changes nothing for existing Keras users, only for TF users. - https://twitter.com/fchollet/status/821090410659344384 https://twitter.com/fchollet/status/821090410659344384 Rather than a "new" Keras it sounds (I could be wrong) as if the Keras API will be included now with TF as an alternative way of interacting with TF (as it has been for some time) and simple bundled with TF.
- attractivechaos 10y agoI wonder how Keras will be integrated into TF and how keras-in-tf will evolve with the separate Keras. More specifically, how many more APIs will keras-in-tf have in addition to the current keras? Wouldn't this hurt keras as a backend-agnostic library? Will the keras-in-tf APIs always keep in sync with standalone-keras? I will be curious to see how these pan out before getting too excited.
- nl 10y agoThat isn't what that Tweet says at all. Notability he makes this very explicit: Theano support will continue for as long as Keras exists. This integration changes nothing for existing Keras users, only for TF users. https://mobile.twitter.com/fchollet/status/821090410659344384 https://mobile.twitter.com/fchollet/status/82109041065934438...
- dzhiurgis 10y agoCan you use Keras to train on GCP?
- gooshan 10y agoFor what I see, TensorFlow is getting what Torch users have had for awhile with NN.
- chenzhekl 10y agoFor me, `tf.contrib.layers` is high-level enough. Besides, it is easier to be integrated to other TensorFlow functionalities than Keras.
- amelius 10y agoBut what will happen to support for the Theano backend?
- fauigerzigerk 10y agoThat is potentially very bad news. I fear that from now on the Python API will be the best documented API used by most example code, and the low level API will over time become something obscure that is hard to approach and badly documented, just like Torch's C API. I don't like this split between a core system written in C or C++ and a high level API written in a language that's too slow and memory hungry to write the core system in. This architecture is probably meant to reflect an existing split between users and implementors of the system and I can understand the arguments in favor. But I think it also creates and reinforces that split, which isn't a good thing at all. We were actually looking into moving to TensorFlow from Torch because the Torch C API is considered an internal thing not meant for public consumption.
- cinquemb 10y agoI don't really understand why people want to funnel themselves into these ml frameworks when you can have the freedom to implement arguably more interesting things in c++/fortran with libs like armadillo, openblas, superlu, etc? I mean, i get it if people just wanna copy paste the functions they see used in blogs/press/toy demo releases and get on with things, is that it?
- fauigerzigerk 10y agoI have an uneasy feeling about these frameworks as well, but I'm not an expert in this field. So if I'm told that it's not worth the effort to recreate all of that stuff from scratch I'm going to have to believe it. What I do insist on is that we keep the capability to selectively dig deeper where we need to and combine different libraries with our own code. That's why I'm always trying to reduce the frameworkness and increase the librariness of any third party code we use. Scripting language runtimes are a major roadblock in that approach.
- cinquemb 10y ago> I have an uneasy feeling about these frameworks as well, but I'm not an expert in this field. So if I'm told that it's not worth the effort to recreate all of that stuff from scratch I'm going to have to believe it. I wouldn't consider myself an expert in the "ml" field either, and only really informed because of my academic/contract research work in neuroimaging wrt DSP and beamforming and having to rely on performative code involving what at the end of the day comes down to manipulating matrices with some set of mathematical/physics constraints; I wished reviewers gave us an easier time with our paper and just believed our results without taking months before accepting it haha. But if you are able to use these frameworks/hack around the quirks for your use case with minimum effort and solve your problems to satisfactory degree, I completely understand that they can be good enough. >…we keep the capability to selectively dig deeper… >That's why I'm always trying to reduce the frameworkness and increase the librariness of any third party code we use. Scripting language runtimes are a major roadblock in that approach. I don't disagree with your sentiments at all, but it seems like you are already at the intersection where you just need to go down your own road because its probably unrealistic to expect Google's needs to be aligned with yours in the long term. Then again, I'm the type of person who would fork the nightly version of firefox and remove and add the stuff I want out of the box so maybe this is not the best suggestion >.<
- Ralfp 10y agoThis looks very interesting to me. Being responsive for campaign sites allowing people to upload images, I would love to be able to use neural network in some of my projects, even if for simple "flag this picture as suspicious" because people are supposed to upload pictures of their cakes, but they are uploading something else instead. Over few years of running the site we've accumulated human-curated database of few dozens of thousands of images of cakes, and with traffic ever growing, I would love to be able to include neural network to prioritize suspicious submissions for moderators. As person without any eduction in the field of maths or neural networks, examples from Keras docs and blog reasonated with me far more than Tensor Flow ones.
- ktamiola 10y agoExcellent news!
- agibsonccc 10y agoDisclaimer: Very biased player in the space. I'd like to personally see keras as the industry standard as an interface. We are starting to give classes with it for our customers and the feedback has been positive. CNTK is adding one: https://github.com/Microsoft/CNTK/issues/797 https://github.com/Microsoft/CNTK/issues/797 Mxnet is there: https://github.com/fchollet/keras/issues/1313 https://github.com/fchollet/keras/issues/1313 TF being there obviously helps. //Begin blantant self promotion :D No one will care about ours as much (research crowd,python,etc etc,..), but I'll throw our hat in the ring here too, keras has been amazing for our use case and we are personally pushing more python people towards that: https://github.com/deeplearning4j/deeplearning4j/pull/2562 https://github.com/deeplearning4j/deeplearning4j/pull/2562 ^^ that will bridge the spark ecosystem and friends so you can run on yarn/mesos/spark etc from a familiar interface. //End blatant self promotion
- pantulis 10y agoIs this bad news for TFlearn?
- ilaksh 10y agoIs there any way to do Keras or Tensorflow on AMD or will there be? I know AMD was pushing something new called ROCm or whatever. But I looked at the OpenCL thread on Tensorflow and they seem to be pretending ROCm isn't relevant. I am considering just using WebGL or something like Turbo.js to experiment with my Radeon hardware for AI. Maybe AMD should hire some people to work full time on Tensorflow. Anyway at this point next thing I buy will probably be an nvidia card and chalk up the amd graphics card purchase to ignorance. At least I played a few games on it.
- transcranial 10y agoHave you seen Keras.js? It uses WebGL to run certain computations on the GPU. https://github.com/transcranial/keras-js https://github.com/transcranial/keras-js
- ilaksh 10y agoNo didn't know about that thanks!
- attractivechaos 10y agoPretty cool! Just to make sure: keras.js so far only supports inference/prediction, but not training (including autodiff, SGD, etc). Is that correct?
- bbctol 10y agoI like this news; as someone coming from an abstract, non-programming background, Keras is gorgeous.