4 ms·

I hope web pages don't grow to hundreds of megabytes just because they have a Neural Network embedded.

nsthorat 9y ago
There is lots of work being done in model compression (quantization, simple factorization tricks, better conv kernels like depthwise separable convs, etc). We won’t let that happen!
- jorgemf 9y ago
  I am aware of that research, but even with a 20x decrease in size some models are still too big for web (think about world wide web, not internet in US).
  - nsthorat 9y ago
    Often times researchers train huge models, but don't think about model size (because they don't have to). We've seen ~200MB production models get down to ~4MB and not lose much precision. I'm quite confident we'll continue that trend. Don't forget that folks were saying this about the web when images / rich media were becoming prevalent!
    - niyazpk 9y ago
      >> We've seen ~200MB production models get down to ~4MB and not lose much precision. Details please. What techniques are used to reduce the model size?
      - hidenotslide 9y ago
        I saw a talk on this paper a couple years ago. https://arxiv.org/abs/1503.02531 https://arxiv.org/abs/1503.02531 The method is to train a smaller model on the predictions of a large model or ensemble. I'd be interested in knowing other techniques as well.
    - jorgemf 9y ago
      200MB is still a small model and 4MB is almost the double of an average web page (including images). 10MB web pages is really bad, more for countries that are still developing their infrastructure.
onion2k 9y ago
I do. I also hope internet bandwidth grows to the point where hundreds of megabytes takes milliseconds to download, and that computer memory increases until hundreds of megabytes is trivial, and that the economics of ISPs mean I get truly unlimited data without incurring charges, but when all that happens I definitely do hope we're utilising technology in ways that includes embedding complex neural networks in webpages as a normal, typical thing we do. That particular future will be awesome.
- jorgemf 9y ago
  Resources don't grow to the infinite, there are physics limits to everything. Be aware of that. You can see the perfect example in CPUs, now we are adding cores but peak speed In a single core don't grow very much in new generations as they did few years ago. With the web it is going to be similar at some point (even the TCP protocol has some limits that forbid to reach top speed with current connections for small transfers).
  - onion2k 9y ago
    The theoretical maximum speed of 5G wireless is 10,000Mbps with a 1ms latency - a 300MB file would download in 251ms. That's current technology being tested in the field at the moment. Real phones will use it in a couple of years. We won't get that speed, but we'll get quite close in major cities. Today's flagship desktop computers have 16GB of RAM, but there's no particular reason why they can't have 10 times that much. Servers already do. Phones have 4GB, but again they could have much more. CPU speeds are fast, but a neural networks run on GPUs which are considerably faster for that task (remember this isn't training the network, it's just running one you've downloaded). This is what's actually happening right now. This is the near future.
    - jorgemf 9y ago
      I am going to just forget all the countries that are still on GSM. Let's say we have 5G right now. 1) The theoretical speed is one thing, the real one is another one. 2) There is more than one node connected to the network 3) TCP needs some package interchange to increase the window size 4) Latency with the server where the page is hosted is not going to be 1ms, you are lucky if it is around 20ms. 5) Serves have more RAM because they use better CPUs, it means they are more expensive. 6) Who connects to websites nowadays? Mobile is the way to go now (It means slower CPUs, less RAM, less power consumption, etc) 7) Why are you going to transfer a complete netural network instead of sending the data to the server. The data is going to be smaller than the network and you avoid the risk of someone cloning the neural network for their web site. 8) There are no useful applications in web sites for neural networks. It is a non sense for me (please probe me wrong and show me an use case where it makes sense)

There is lots of work being done in model compression (quantization, simple factorization tricks, better conv kernels like depthwise separable convs, etc). We won’t let that happen!

jorgemf 9y ago
I am aware of that research, but even with a 20x decrease in size some models are still too big for web (think about world wide web, not internet in US).
- nsthorat 9y ago
  Often times researchers train huge models, but don't think about model size (because they don't have to). We've seen ~200MB production models get down to ~4MB and not lose much precision. I'm quite confident we'll continue that trend. Don't forget that folks were saying this about the web when images / rich media were becoming prevalent!
  - niyazpk 9y ago
    >> We've seen ~200MB production models get down to ~4MB and not lose much precision. Details please. What techniques are used to reduce the model size?
    - hidenotslide 9y ago
      I saw a talk on this paper a couple years ago. https://arxiv.org/abs/1503.02531 https://arxiv.org/abs/1503.02531 The method is to train a smaller model on the predictions of a large model or ensemble. I'd be interested in knowing other techniques as well.
  - jorgemf 9y ago
    200MB is still a small model and 4MB is almost the double of an average web page (including images). 10MB web pages is really bad, more for countries that are still developing their infrastructure.

onion2k 9y ago

I do. I also hope internet bandwidth grows to the point where hundreds of megabytes takes milliseconds to download, and that computer memory increases until hundreds of megabytes is trivial, and that the economics of ISPs mean I get truly unlimited data without incurring charges, but when all that happens I definitely do hope we're utilising technology in ways that includes embedding complex neural networks in webpages as a normal, typical thing we do. That particular future will be awesome.

jorgemf 9y ago
Resources don't grow to the infinite, there are physics limits to everything. Be aware of that. You can see the perfect example in CPUs, now we are adding cores but peak speed In a single core don't grow very much in new generations as they did few years ago. With the web it is going to be similar at some point (even the TCP protocol has some limits that forbid to reach top speed with current connections for small transfers).
- onion2k 9y ago
  The theoretical maximum speed of 5G wireless is 10,000Mbps with a 1ms latency - a 300MB file would download in 251ms. That's current technology being tested in the field at the moment. Real phones will use it in a couple of years. We won't get that speed, but we'll get quite close in major cities. Today's flagship desktop computers have 16GB of RAM, but there's no particular reason why they can't have 10 times that much. Servers already do. Phones have 4GB, but again they could have much more. CPU speeds are fast, but a neural networks run on GPUs which are considerably faster for that task (remember this isn't training the network, it's just running one you've downloaded). This is what's actually happening right now. This is the near future.
  - jorgemf 9y ago
    I am going to just forget all the countries that are still on GSM. Let's say we have 5G right now. 1) The theoretical speed is one thing, the real one is another one. 2) There is more than one node connected to the network 3) TCP needs some package interchange to increase the window size 4) Latency with the server where the page is hosted is not going to be 1ms, you are lucky if it is around 20ms. 5) Serves have more RAM because they use better CPUs, it means they are more expensive. 6) Who connects to websites nowadays? Mobile is the way to go now (It means slower CPUs, less RAM, less power consumption, etc) 7) Why are you going to transfer a complete netural network instead of sending the data to the server. The data is going to be smaller than the network and you avoid the risk of someone cloning the neural network for their web site. 8) There are no useful applications in web sites for neural networks. It is a non sense for me (please probe me wrong and show me an use case where it makes sense)