5 ms·

chainer is the best. plain numpy means that somebody else thinks about broadcasting rules and other stuff that has nothing specific to do with neural stuff and

chainer is the best. plain numpy means that somebody else thinks about broadcasting rules and other stuff that has nothing specific to do with neural stuff and cupy just implements them.

amelius 9y ago
Imho, as a user, it is rather silly to find that a framework requires me to define a dependency graph instead of just coding the operations directly. In my career so far, no compiler has asked me to do the same. So why would neural networks be an exception?
- cjalmeida 9y ago
  Optimization and embedded devices. In theory, knowing the full graph upfront allows the framework to optimize and fuse some operations. For embedded devices you may not have access to Python. You can precompile the graph to the target device in such cases. Note that in practice, PyTorch is as fast or faster than Tensorflow and, and newer version allow you to "post compute" the graph and export to ONNX to allow embedded inference using Caffe2
- improbable22 9y ago
  Indeed. Frameworks seem like embryonic languages, which are extremely awkward to work with (compared to Python) but easy to compile to the GPU. A relevant essay: "On Machine Learning and Programming Languages" https://julialang.org/blog/2017/12/ml&pl https://julialang.org/blog/2017/12/ml&pl
- jerf 9y ago
  Because if you are using a language designed around one execution model, but you need another execution model, defining a graph is one of the better ways to do it, even if it is annoying. You're basically writing a compiler that takes your problem and converts it into another representation that will actually be run. In the maximally abstract sense, Python isn't necessarily the best choice for this, but as the closest manifestation to the correct way to do this that I know of is probably Haskell, that seems unlikely to beat out Python any time soon. There's certainly a lot of worse languages. Ultimately, you'll want a language that has the necessary primitives built in natively, but it doesn't sound like we're to that level of maturity in the field yet.
stealthcat 9y ago
Chainer does have its tradeoff though. It's not exactly smooth to generate CUDA kernels from pure Python code. Also Chainer is bogged down by the fact it is made and maintained by Japanese company/people or at least majority of them are Japanese. Most of the community itself are Japanese users, and many model implementations are made by Japanese users, hence in blogged, documented in Japanese. Chainer had similar fate like Ruby. Language barrier is real.
- p1esk 9y ago
  It's not exactly smooth to generate CUDA kernels from pure Python code. Can you give an example? What's not smooth?
  - stealthcat 9y ago
    https://www.reddit.com/r/MachineLearning/comments/7lb5n1/d_chainer_vs_pytorch/drlu145/ https://www.reddit.com/r/MachineLearning/comments/7lb5n1/d_c...

chainer is the best. plain numpy means that somebody else thinks about broadcasting rules and other stuff that has nothing specific to do with neural stuff and cupy just implements them.

amelius 9y ago

Imho, as a user, it is rather silly to find that a framework requires me to define a dependency graph instead of just coding the operations directly. In my career so far, no compiler has asked me to do the same. So why would neural networks be an exception?

cjalmeida 9y ago
Optimization and embedded devices. In theory, knowing the full graph upfront allows the framework to optimize and fuse some operations. For embedded devices you may not have access to Python. You can precompile the graph to the target device in such cases. Note that in practice, PyTorch is as fast or faster than Tensorflow and, and newer version allow you to "post compute" the graph and export to ONNX to allow embedded inference using Caffe2
improbable22 9y ago
Indeed. Frameworks seem like embryonic languages, which are extremely awkward to work with (compared to Python) but easy to compile to the GPU. A relevant essay: "On Machine Learning and Programming Languages" https://julialang.org/blog/2017/12/ml&pl https://julialang.org/blog/2017/12/ml&pl
jerf 9y ago
Because if you are using a language designed around one execution model, but you need another execution model, defining a graph is one of the better ways to do it, even if it is annoying. You're basically writing a compiler that takes your problem and converts it into another representation that will actually be run. In the maximally abstract sense, Python isn't necessarily the best choice for this, but as the closest manifestation to the correct way to do this that I know of is probably Haskell, that seems unlikely to beat out Python any time soon. There's certainly a lot of worse languages. Ultimately, you'll want a language that has the necessary primitives built in natively, but it doesn't sound like we're to that level of maturity in the field yet.

stealthcat 9y ago

Chainer does have its tradeoff though. It's not exactly smooth to generate CUDA kernels from pure Python code. Also Chainer is bogged down by the fact it is made and maintained by Japanese company/people or at least majority of them are Japanese. Most of the community itself are Japanese users, and many model implementations are made by Japanese users, hence in blogged, documented in Japanese. Chainer had similar fate like Ruby. Language barrier is real.

p1esk 9y ago
It's not exactly smooth to generate CUDA kernels from pure Python code. Can you give an example? What's not smooth?
- stealthcat 9y ago
  https://www.reddit.com/r/MachineLearning/comments/7lb5n1/d_chainer_vs_pytorch/drlu145/ https://www.reddit.com/r/MachineLearning/comments/7lb5n1/d_c...