5 ms·
That said, there's been a religious flamewar over which-is-better-TensorFlow/Keras-or-PyTorch that's dragged down a lot of productive discussion about ML/DL fra
by minimaxir 9y ago
That said, there's been a religious flamewar over which-is-better-TensorFlow/Keras-or-PyTorch that's dragged down a lot of productive discussion about ML/DL frameworks.
- curuinor 9y agochainer is the best. plain numpy means that somebody else thinks about broadcasting rules and other stuff that has nothing specific to do with neural stuff and cupy just implements them.
- amelius 9y agoImho, as a user, it is rather silly to find that a framework requires me to define a dependency graph instead of just coding the operations directly. In my career so far, no compiler has asked me to do the same. So why would neural networks be an exception?
- cjalmeida 9y agoOptimization and embedded devices. In theory, knowing the full graph upfront allows the framework to optimize and fuse some operations. For embedded devices you may not have access to Python. You can precompile the graph to the target device in such cases. Note that in practice, PyTorch is as fast or faster than Tensorflow and, and newer version allow you to "post compute" the graph and export to ONNX to allow embedded inference using Caffe2
- improbable22 9y agoIndeed. Frameworks seem like embryonic languages, which are extremely awkward to work with (compared to Python) but easy to compile to the GPU. A relevant essay: "On Machine Learning and Programming Languages" https://julialang.org/blog/2017/12/ml&pl https://julialang.org/blog/2017/12/ml&pl
- jerf 9y agoBecause if you are using a language designed around one execution model, but you need another execution model, defining a graph is one of the better ways to do it, even if it is annoying. You're basically writing a compiler that takes your problem and converts it into another representation that will actually be run. In the maximally abstract sense, Python isn't necessarily the best choice for this, but as the closest manifestation to the correct way to do this that I know of is probably Haskell, that seems unlikely to beat out Python any time soon. There's certainly a lot of worse languages. Ultimately, you'll want a language that has the necessary primitives built in natively, but it doesn't sound like we're to that level of maturity in the field yet.
- stealthcat 9y agoChainer does have its tradeoff though. It's not exactly smooth to generate CUDA kernels from pure Python code. Also Chainer is bogged down by the fact it is made and maintained by Japanese company/people or at least majority of them are Japanese. Most of the community itself are Japanese users, and many model implementations are made by Japanese users, hence in blogged, documented in Japanese. Chainer had similar fate like Ruby. Language barrier is real.
- p1esk 9y agoIt's not exactly smooth to generate CUDA kernels from pure Python code. Can you give an example? What's not smooth?
- stealthcat 9y agohttps://www.reddit.com/r/MachineLearning/comments/7lb5n1/d_chainer_vs_pytorch/drlu145/ https://www.reddit.com/r/MachineLearning/comments/7lb5n1/d_c...
- pixelHD 9y agoYo! I've read articles on your blog quite a few times while working on my thesis. Great work! I've been using Pytorch for quite a while, and had to use tensorflow recently. May be its my pytorch _priors_, but using tensorflow felt weird, and un-intuitive. Any good resources for pytorch folks to get into tensorflow?
- cjalmeida 9y agoSpecially since the answer is obviously PyTorch. :) Seriously though, we're in the early days of ML and still understanding the correct patterns and abstractions. Web dev been through this to the point we had "JS fatigue" until things started settling on the reactive model and React / Vue / Angular 2
- pedrosorio 9y ago"Seriously though, we're in the early days of ML" In the early days of "differentiable programming" perhaps. Certainly not ML.
- cookiecaper 9y agoThis has got to be one of the most important things to keep an ecosystem active. If you don't have the spectacle from the fireworks and heat from the flame war to keep peoples' interest, you are in danger. Controversy and religion are excellent and important health indicators, because they reflect passion within your community, while giving passerbys a fascinating train wreck to watch and discuss. Attention is an exceedingly precious commodity, potentially the most precious and valuable commodity in the marketplace. Skilled developers must learn how to direct this so that it stops being commandeered by posers all the time.
- deepGem 9y agoTensorflow's eager execution might challenge Pytorch. I for one love the imperative style of Pytorch but I also realize the advantages of pre-computed graph style of TF, especially when it comes to distribution. Interesting times!