5 ms·

The non cloud version is your website goes offline for a day due to high traffic. Depending on the website that may or may not be preferable to variable costs.

CriticalCathed 7y ago
This isn't 2005. It really doesn't need to happen anymore.
- londons_explore 7y ago
  Says the person not paying the $10k/day unexpected bill...
bildung 7y ago
The Twitter feed says the $10k+ are for serving about 60k Visitors a day. This isn't exactly high traffic. A dedicated server from Hetzner with unlimited Traffic and 1gbit/s uplink, a 16 core Threadripper, 1TB SSD and 128G RAM costs €140/Month and would be absolute overkill for 60k Visitors. It doesn't have GPUs, though.
- neysofu 7y ago
  It absolutely is high traffic if each and every one of them is downloading your NN model
  - bildung 7y ago
    I see the model size is 6G - a bit larger than I thought :) 100gbit/s upstream would cost an additional 1k€/month. The larger problem is that people didn't actually download the model, but apparently got custom server instances that got a copy of the model plus a gpu to run them on.
    - orbifold 7y ago
      The server instances are free though, google only charged them for the datatransfer to those instances.
- shandor 7y ago
  For similar price point (€105.62), from e.g. Azure you can get their "E2 v3" VM, with 2 vCPUs and 16 GB RAM, plus 50GB temporary storage. That's...quite a difference.
- jrockway 7y ago
  They are seeing 60k downloads of 6GB models per day, which is about 33Gbps of bandwidth (assuming no burstiness in when people visit, which is a poor assumption). That is starting to get out of the range of what is easily available. 10Gbps circuits are commodities (I had one at my desk at my last job), but 100Gbps circuits are still pretty pricey. And, it's not necessarily trivial to get that kind of throughput on file serving out of the box; this bandwidth is something like CPU <-> video card, not disk <-> cpu, or cpu <-> network. Some tweaking is for sure going to be necessary if you are self-hosting this, and now you're tweaking network parameters and writing a custom file server instead of writing your game. The cloud here is making something possible that should never have been possible, which is pretty cool. Being able to go from 0 infrastructure to 30Gbps of file serving without lifting a finger is somewhat impressive... but with that fast iteration times, comes the entity that did all the work wanting their cut. It seems fair to me, though perhaps not economically viable. Such is life.
  - krick 7y ago
    > 60k downloads of 6GB models per day Wait... you mean the actual computation is running client in the browser? I didn't even open this "game", but I assumed such high cost is because there is a separate GPT-2 running on a GPUs for each and every user.
    - jsty 7y ago
      No, the models are running in Colab - but in the end-user's account - so each 'run' costs the lab 6GB of internet egress when the model is downloaded from GCS to the Colab VM.
    - lis 7y ago
      No, the computation is not running on the client in the browser. This is the traffic to transfer the model from GCS to Google Colab. This is what makes the price so surprising - you are copying data from one Google Service to another, but it's billed as egress.
      - jstanley 7y ago
- mark_l_watson 7y ago
  About a year and a half ago, I rented a Hetzner server with a 1080 GPU, 16 core i7, etc. it was only about $105/month. I had a job queue for TensorFlow/Keras experiments and set it to mine ether when there were no ML jobs. I did this until I bought a System76 laptop with a 1070 GPU.

CriticalCathed 7y ago

This isn't 2005. It really doesn't need to happen anymore.

londons_explore 7y ago
Says the person not paying the $10k/day unexpected bill...

bildung 7y ago

The Twitter feed says the $10k+ are for serving about 60k Visitors a day. This isn't exactly high traffic. A dedicated server from Hetzner with unlimited Traffic and 1gbit/s uplink, a 16 core Threadripper, 1TB SSD and 128G RAM costs €140/Month and would be absolute overkill for 60k Visitors. It doesn't have GPUs, though.

neysofu 7y ago
It absolutely is high traffic if each and every one of them is downloading your NN model
- bildung 7y ago
  I see the model size is 6G - a bit larger than I thought :) 100gbit/s upstream would cost an additional 1k€/month. The larger problem is that people didn't actually download the model, but apparently got custom server instances that got a copy of the model plus a gpu to run them on.
  - orbifold 7y ago
    The server instances are free though, google only charged them for the datatransfer to those instances.
shandor 7y ago
For similar price point (€105.62), from e.g. Azure you can get their "E2 v3" VM, with 2 vCPUs and 16 GB RAM, plus 50GB temporary storage. That's...quite a difference.
jrockway 7y ago
They are seeing 60k downloads of 6GB models per day, which is about 33Gbps of bandwidth (assuming no burstiness in when people visit, which is a poor assumption). That is starting to get out of the range of what is easily available. 10Gbps circuits are commodities (I had one at my desk at my last job), but 100Gbps circuits are still pretty pricey. And, it's not necessarily trivial to get that kind of throughput on file serving out of the box; this bandwidth is something like CPU <-> video card, not disk <-> cpu, or cpu <-> network. Some tweaking is for sure going to be necessary if you are self-hosting this, and now you're tweaking network parameters and writing a custom file server instead of writing your game. The cloud here is making something possible that should never have been possible, which is pretty cool. Being able to go from 0 infrastructure to 30Gbps of file serving without lifting a finger is somewhat impressive... but with that fast iteration times, comes the entity that did all the work wanting their cut. It seems fair to me, though perhaps not economically viable. Such is life.
- krick 7y ago
  > 60k downloads of 6GB models per day Wait... you mean the actual computation is running client in the browser? I didn't even open this "game", but I assumed such high cost is because there is a separate GPT-2 running on a GPUs for each and every user.
  - jsty 7y ago
    No, the models are running in Colab - but in the end-user's account - so each 'run' costs the lab 6GB of internet egress when the model is downloaded from GCS to the Colab VM.
  - lis 7y ago
    No, the computation is not running on the client in the browser. This is the traffic to transfer the model from GCS to Google Colab. This is what makes the price so surprising - you are copying data from one Google Service to another, but it's billed as egress.
    - jstanley 7y ago
mark_l_watson 7y ago
About a year and a half ago, I rented a Hetzner server with a 1080 GPU, 16 core i7, etc. it was only about $105/month. I had a job queue for TensorFlow/Keras experiments and set it to mine ether when there were no ML jobs. I did this until I bought a System76 laptop with a 1070 GPU.