12 ms·

There is NO save on computation cost. CPU is slow for NN whatever language you are using. OP has a point, if I want to learn PRACTICAL front end programming, I

There is NO save on computation cost. CPU is slow for NN whatever language you are using. OP has a point, if I want to learn PRACTICAL front end programming, I will choose javascript, not Python. Same for Neural networks, just switch the place.

mschuetz 8y ago
What's wrong with js for neural networks? JS/V8 is faster than python, not accounting for python bindings to native NN libs that may as well be bound to js.
- karmasimida 8y ago
  There is nothing wrong to use js, like there is nothing wrong to call it impractical. JS is faster than Python, but in the land of DL, there is C++ and everyone else. No one is using Python to do the actual computation anyway.
  - zamadatix 8y ago
    "No one is using Python to do the actual computation anyway." Depends what "actual" computation is. If you're definition of "actual computation" is something which requires extreme optimization then your definition precludes the question. In the real world there is an extraordinary amount of computation done with inefficient languages though simply because development time costs very often outweighs run time costs.
    - heavyset_go 8y ago
      Python is used because there are C/C++ bindings for working with NNs. The point is that none of computations are done in pure Python. Python just provides a convenient wrapper over non-Python code.
- deleted 8y ago
  [deleted]
- yonkshi 8y ago
  JS/V8 may beat Python in general operations but definitely not in math specialized libraries. A lot of python packages support OpenBLAS and AVX/FMA by default which gives huge boosts to math/tensor operations. Correct me if I'm wrong, I don't think the js ecosystem is quite there yet except for experimental projects.
zawerf 8y ago
Cost as in money. To deploy a model in js (as a web page) all you need is a static S3 or GCS bucket. You don't even need a webserver and it can automatically handle infinite scale. Show me a python solution that can do the same.
- yonkshi 8y ago
  There is currently no NN learning algorithm that can handle massively paralleled training. We can use some simple fixes such as mean gradient but they have severe limitations, and their limitation grows as you scale up. Currently even with a handful paralleled GPU training, the gradient computation needs to wait for all GPU batch to complete and then return to CPU before the next batch, so your idea of infinite scaling is just a pipe dream at the moment. Sure Python can't be run on a million pages, but heck, no NN architecture can even handle more than a dozen paralleled computation in a stable manner yet.
  - zawerf 8y ago
    Yea the infinite scaling was referring to deployment of inference, not training.
- karmasimida 8y ago
  > To deploy a model in js (as a web page) all you need is a static S3 or GCS bucket. First of all, what you have described is far from the reality. Had this come true, only inference will be in javascript, through some language agnostic standardized model format, not training. The model is just a blackbox function for the js runtime to call. The amount of javascript to make this happen will be surprisingly slim anyway.
  - zawerf 8y ago
    > First of all, what you have described is far from the reality. Did you try any of the links I included? This is the reality for all of them and they are a few years old. They have a model file loaded from bucket url and never make another network request thereafter. It's actually the recommended workflow from https://js.tensorflow.org/ https://js.tensorflow.org/ where you'd find tons of other examples. But you're right, this is for inference only. I would not do training in JS.

mschuetz 8y ago

What's wrong with js for neural networks? JS/V8 is faster than python, not accounting for python bindings to native NN libs that may as well be bound to js.

karmasimida 8y ago
There is nothing wrong to use js, like there is nothing wrong to call it impractical. JS is faster than Python, but in the land of DL, there is C++ and everyone else. No one is using Python to do the actual computation anyway.
- zamadatix 8y ago
  "No one is using Python to do the actual computation anyway." Depends what "actual" computation is. If you're definition of "actual computation" is something which requires extreme optimization then your definition precludes the question. In the real world there is an extraordinary amount of computation done with inefficient languages though simply because development time costs very often outweighs run time costs.
  - heavyset_go 8y ago
    Python is used because there are C/C++ bindings for working with NNs. The point is that none of computations are done in pure Python. Python just provides a convenient wrapper over non-Python code.
deleted 8y ago
[deleted]
yonkshi 8y ago
JS/V8 may beat Python in general operations but definitely not in math specialized libraries. A lot of python packages support OpenBLAS and AVX/FMA by default which gives huge boosts to math/tensor operations. Correct me if I'm wrong, I don't think the js ecosystem is quite there yet except for experimental projects.

zawerf 8y ago

Cost as in money. To deploy a model in js (as a web page) all you need is a static S3 or GCS bucket. You don't even need a webserver and it can automatically handle infinite scale. Show me a python solution that can do the same.

yonkshi 8y ago
There is currently no NN learning algorithm that can handle massively paralleled training. We can use some simple fixes such as mean gradient but they have severe limitations, and their limitation grows as you scale up. Currently even with a handful paralleled GPU training, the gradient computation needs to wait for all GPU batch to complete and then return to CPU before the next batch, so your idea of infinite scaling is just a pipe dream at the moment. Sure Python can't be run on a million pages, but heck, no NN architecture can even handle more than a dozen paralleled computation in a stable manner yet.
- zawerf 8y ago
  Yea the infinite scaling was referring to deployment of inference, not training.
karmasimida 8y ago
> To deploy a model in js (as a web page) all you need is a static S3 or GCS bucket. First of all, what you have described is far from the reality. Had this come true, only inference will be in javascript, through some language agnostic standardized model format, not training. The model is just a blackbox function for the js runtime to call. The amount of javascript to make this happen will be surprisingly slim anyway.
- zawerf 8y ago
  > First of all, what you have described is far from the reality. Did you try any of the links I included? This is the reality for all of them and they are a few years old. They have a model file loaded from bucket url and never make another network request thereafter. It's actually the recommended workflow from https://js.tensorflow.org/ https://js.tensorflow.org/ where you'd find tons of other examples. But you're right, this is for inference only. I would not do training in JS.