5 ms·
This is great news! One of the most intimidating things about getting started with deep learning if you want to understand and extend cutting-edge work is the T
by aab0 10y ago
This is great news! One of the most intimidating things about getting started with deep learning if you want to understand and extend cutting-edge work is the Tower of Babel situation: aside from mastering some quite difficult and opaque concepts, you need to learn multiple frameworks in multiple languages, some of which are quite uncommon. (Want to use Torch? You have to learn Lua. Want to use Theano or Caffe? Need to learn Python too. Need to implement optimizations? Hope you've mastered C++.)
And DeepMind's research output was a major reason to need to use Torch, and hence have to learn Lua.
But by switching over to TensorFlow, this means you now have one language to learn which is supported well by all the major frameworks - Python - and you can benefit from several frameworks (Theano, Keras, TensorFlow). So the language barrier is reduced and you can focus on the framework and actual NN stuff. Further, this will also drive consolidation onto TensorFlow, reducing the framework mental overhead. As long as TF is up to the job, and it reportedly is, this will benefit the deep learning community considerably.
I'd been wondering myself what language and framework I should focus on when I start studying NNs, and this settles it for me: Python and TensorFlow.
- mrdrozdov 10y agoCan you speak for the implementing of algorithms in TensorFlow? My impression is that it's still quite difficult to introduce something interesting that doesn't already exist. This is basically the case for all the frameworks, but I've found that even though Torch is written Lua, there is much much less abstraction.
- pigscantfly 10y agoI can't speak for TensorFlow, but I've personally found Theano to be the most easily extensible among the other widespread deep/reinforcement/gradient learning libraries (Caffe, Torch, Keras, Chainer). There does seem to be a performance hit between Theano and Caffe/Torch, but it's great for prototyping.
- sja 10y agoImplementing a new operation in TensorFlow does require a fair amount of overhead, and you'll need to know CUDA if you want to create something that works on (NVIDIA) GPUs. The "Creating an Op" how-to[1] goes over the basic steps necessary to implement an operation, but there are a couple of things missing. Notably, proper documentation for the various functions and classes used when writing operations, as well as information on writing (and registering) a Python wrapper for additional functionality. All this said, if you want to commit your work to the GitHub master, the Google team working on the repository does an excellent job of walking contributors through the steps necessary to get their work over the hump. [1]: https://www.tensorflow.org/versions/master/how_tos/adding_an_op/index.html https://www.tensorflow.org/versions/master/how_tos/adding_an...
- clickok 10y agoTensorFlow makes some things difficult in truly baffling ways. For example, the last time I checked there's no built-in way to get gradients w/r/t say a single weight, rather you get a gradient sum. So if you want to do anything with the Jacobian (e.g. contractive autoencoders) you are better off using Theano.
- argonaut 10y agoYou're overstating things a bit. There has never been a "need to learn multiple frameworks in multiple languages." As a beginner, you pick one and go with that (as evidenced by the fact that... you've picked one!). This announcement doesn't change that situation. Nobody needs to use multiple frameworks unless you're someone (not a beginner) who wants to be able to take the code from research papers or something.
- tedmiston 10y agoResearchers need multiple frameworks because the feature sets aren't the same in each. I haven't done anything with deep learning, but I worked in a research lab with others who did. On the image processing side, we prototyped code in OpenCV, both with Python and C++, and MATLAB (with toolboxes) regularly because of this. At the end of the day they have a limited amount of time and just want to test their idea the fastest way they can.
- argonaut 10y agoI don't think this is the reason. You can do pretty much anything in Torch/Theano. But, researchers are lazy and writing code is only a means to an end. They will shamelessly copy/hack as much open source code as possible, so if a framework has some quick module for a desired algorithm, or if a researcher wrote their paper in some framework, and you want to build off that paper, then you'll just copy that.
- tedmiston 10y agoThat's definitely correct. One related unsolved problem in that type of research is getting people to share their actual code. Especially when multiple universities are at the bleeding edge in a field, they often publish just enough [1] to prove their point without giving everyone else the same foundation to build on easily. Even in science it gets political... who knew. 1: i.e., just their algorithms, or their code without very useful implementation-level optimizations
- 10y ago
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]