6 ms·

If Tensorflow becomes the default library for Deep learning, is this a good thing or bad thing? Does it help in that all researchers can focus on what's importa

If Tensorflow becomes the default library for Deep learning, is this a good thing or bad thing? Does it help in that all researchers can focus on what's important (the data and results) or does it hurt in that Google now controls an important paradigm for the next generation of computing?

li4ick 9y ago
Researchers use Tensorflow for backend only + Keras for implementation. However, PyTorch and Caffe are much more popular among researchers. I personally enjoy PyTorch a lot.
- niyazpk 9y ago
  >> I personally enjoy PyTorch a lot Why?
halflings 9y ago
I fail to see what's wrong with this: it's open-source, and anybody can write different libraries, or libraries that use all the low-level work ($$$ paying experienced/skilled engineers) that Google pours into making tensorflow. Alternatives exist (PyTorch by facebook, theano, caffe, etc.) in any case, and the best solution wins. (in the same way scikit-learn once eclipsed all other ML libraries, but some would still use xgboost which had superior Gradient Boosting Trees).
- sillysaurus3 9y ago
  the best solution wins Minor quibble, but: not really. The earliest solution wins, e.g. rails. It has to be good enough, but after a certain point "best" stops mattering.
  - omot 9y ago
    > The earliest solution wins Minor minor quibble, but: not really. The earliest solution wins, unless there is a 10x better solution, e.g Google. There were a lot of good enough options, but after a certain benchmark "earliest" stops mattering.
- ehsankia 9y ago
  Absolutely. To me, it's very similar to programming languages. At the end of the day, it doesn't really matter if people use Google's Go, C++, or some other language. What matters is what's produced by it: the actual research and results. When you see a useful program that helps you be much more productive, you don't really care if it was coded in c++ or python. The one part Google does care about is being able to sell Cloud TPU, but I don't think that by using this tool you're instantly in under their control.
  - davedx 9y ago
    Most of the time I agree with this, but some areas (where e.g. Google make Dart but also make Chrome), I think there are risks.
deleted 9y ago
[deleted]
Hyperbolic 9y ago
A popular framework for iterative control sequences in deep learning (e.g. Stack LSTM) is Dynet. It's being used more and more in the NLP community.
TuringNYC 9y ago
I think it is an incredibly positive thing that TensorFlow, Caffe2, Torch and multiple open-source systems are competing actively in the space. These are all active projects, with open-access support facilities. This is all of enormous societal value and we all reap the benefits in numerous, usually invisible, ways.
blueyes 9y ago
PyTorch is becoming much more popular among researchers, even if TensorFlow use is widespread among data scientists. Would it be bad if TensorFlow became the default library for deep learning? For companies that aren't Google, yes. Many organizations don't like how Google's controlled Android development. But I think an important part of Dean's lecture are the slides that talk about making ML/DL expertise obsolete. That's the future, whether you use TensorFlow or not.
- deepnotderp 9y ago
  It's cute, but isn't going to completely obsolete ML/DL expertise. (And believe me, as a prospective supplier of DL compute, I would love for the answer to be "more compute"). To give you an idea of how much compute it took, they spent two million dollars for one run on a relatively toy dataset, CIFAR-10. Imagine how much it would take on an imagenet sized dataset! Can your company afford ~$20 million+ per dataset? I do think that hyperparameter twiddling aspect might get automated, but believe me, that is much welcomed by the DL research community! I would much rather spend my time on new ideas rather than trying ten different initializations :)
  - blueyes 9y ago
    Sure, but that's $2m of today's compute on today's chips. The constraints on chips and cost are moving, and Google is pushing them.
    - deepnotderp 9y ago
      Sure, so it becomes ~$2 million for an imagenet scale dataset with TPUs and ~$200K with our chips. Still pretty expensive :)
- londons_explore 9y ago
  Android is a very closed opensource project. For example, if you write a pull request to add a feature as an outsider, the chances of it getting into the main project is very low. Chromium is very open. In many cases, Google has to ask permission from outside contributors to commit code, since the outside contributors are the "boss" of part of the codebase. Tensorflow is closer to chromiums model. There are a lot of external contributors.
tanilama 9y ago
It won't, however it won't go away either.
harigov 9y ago
I believe their game plan is more hardware based differentiation rather than software alone. From that perspective, TensorFlow will probably play a similar role to Google's Cloud that DirectX played to Microsoft Windows. As other cloud vendors aren't THAT far behind (notably Microsoft), I don't believe Google will win with that high of a margin, if it wins.
agibsonccc 9y ago
I personally think this would lead to stagnation in the space. Because of competition, tensorflow is adding better ETL tools, dynamic computation graphs,.. I would be careful about having only 1 company's use cases served. Competition forces parties to innovate. For example mxnet and cntk or even more recently caffe2 serve different use cases and companies. Giving you a recent one with our deep learning framework (I compete with all the above frameworks): We compete in the big data niche and recently found a few bottlenecks we missed because people run comparisons.

li4ick 9y ago

Researchers use Tensorflow for backend only + Keras for implementation. However, PyTorch and Caffe are much more popular among researchers. I personally enjoy PyTorch a lot.

niyazpk 9y ago
>> I personally enjoy PyTorch a lot Why?

halflings 9y ago

I fail to see what's wrong with this: it's open-source, and anybody can write different libraries, or libraries that use all the low-level work ($$$ paying experienced/skilled engineers) that Google pours into making tensorflow. Alternatives exist (PyTorch by facebook, theano, caffe, etc.) in any case, and the best solution wins. (in the same way scikit-learn once eclipsed all other ML libraries, but some would still use xgboost which had superior Gradient Boosting Trees).

sillysaurus3 9y ago
the best solution wins Minor quibble, but: not really. The earliest solution wins, e.g. rails. It has to be good enough, but after a certain point "best" stops mattering.
- omot 9y ago
  > The earliest solution wins Minor minor quibble, but: not really. The earliest solution wins, unless there is a 10x better solution, e.g Google. There were a lot of good enough options, but after a certain benchmark "earliest" stops mattering.
ehsankia 9y ago
Absolutely. To me, it's very similar to programming languages. At the end of the day, it doesn't really matter if people use Google's Go, C++, or some other language. What matters is what's produced by it: the actual research and results. When you see a useful program that helps you be much more productive, you don't really care if it was coded in c++ or python. The one part Google does care about is being able to sell Cloud TPU, but I don't think that by using this tool you're instantly in under their control.
- davedx 9y ago
  Most of the time I agree with this, but some areas (where e.g. Google make Dart but also make Chrome), I think there are risks.

deleted 9y ago

[deleted]

Hyperbolic 9y ago

A popular framework for iterative control sequences in deep learning (e.g. Stack LSTM) is Dynet. It's being used more and more in the NLP community.

TuringNYC 9y ago

I think it is an incredibly positive thing that TensorFlow, Caffe2, Torch and multiple open-source systems are competing actively in the space. These are all active projects, with open-access support facilities. This is all of enormous societal value and we all reap the benefits in numerous, usually invisible, ways.

blueyes 9y ago

PyTorch is becoming much more popular among researchers, even if TensorFlow use is widespread among data scientists. Would it be bad if TensorFlow became the default library for deep learning? For companies that aren't Google, yes. Many organizations don't like how Google's controlled Android development. But I think an important part of Dean's lecture are the slides that talk about making ML/DL expertise obsolete. That's the future, whether you use TensorFlow or not.

deepnotderp 9y ago
It's cute, but isn't going to completely obsolete ML/DL expertise. (And believe me, as a prospective supplier of DL compute, I would love for the answer to be "more compute"). To give you an idea of how much compute it took, they spent two million dollars for one run on a relatively toy dataset, CIFAR-10. Imagine how much it would take on an imagenet sized dataset! Can your company afford ~$20 million+ per dataset? I do think that hyperparameter twiddling aspect might get automated, but believe me, that is much welcomed by the DL research community! I would much rather spend my time on new ideas rather than trying ten different initializations :)
- blueyes 9y ago
  Sure, but that's $2m of today's compute on today's chips. The constraints on chips and cost are moving, and Google is pushing them.
  - deepnotderp 9y ago
    Sure, so it becomes ~$2 million for an imagenet scale dataset with TPUs and ~$200K with our chips. Still pretty expensive :)
londons_explore 9y ago
Android is a very closed opensource project. For example, if you write a pull request to add a feature as an outsider, the chances of it getting into the main project is very low. Chromium is very open. In many cases, Google has to ask permission from outside contributors to commit code, since the outside contributors are the "boss" of part of the codebase. Tensorflow is closer to chromiums model. There are a lot of external contributors.

tanilama 9y ago

It won't, however it won't go away either.

harigov 9y ago

I believe their game plan is more hardware based differentiation rather than software alone. From that perspective, TensorFlow will probably play a similar role to Google's Cloud that DirectX played to Microsoft Windows. As other cloud vendors aren't THAT far behind (notably Microsoft), I don't believe Google will win with that high of a margin, if it wins.

agibsonccc 9y ago

I personally think this would lead to stagnation in the space. Because of competition, tensorflow is adding better ETL tools, dynamic computation graphs,.. I would be careful about having only 1 company's use cases served. Competition forces parties to innovate. For example mxnet and cntk or even more recently caffe2 serve different use cases and companies. Giving you a recent one with our deep learning framework (I compete with all the above frameworks): We compete in the big data niche and recently found a few bottlenecks we missed because people run comparisons.