3 ms·

I remember this article from 2018: https://medium.com/the-mission/why-building-your-own-deep-learning-computer-is-10x-cheaper-than-aws-b1c91b55ce8c https://medi

I remember this article from 2018: https://medium.com/the-mission/why-building-your-own-deep-learning-computer-is-10x-cheaper-than-aws-b1c91b55ce8c https://medium.com/the-mission/why-building-your-own-deep-le... Hackernews discussion for the article: https://news.ycombinator.com/item?id=18063893 https://news.ycombinator.com/item?id=18063893 It really is interesting how this is changing the dynamics of neural network training. Now it is affordable to train a useful network on the cloud, whereas 2 years ago that would be reserved to companies with either bigger investments or an already consolidated product.

qayxc 6y ago
> Now it is affordable to train a useful network on the cloud I honestly don't see how anything changed significantly in past 2 years. Benchmarks indicate that a V100 is barely 2x the performance of an RTX 2080 Ti [1] and a V100 is • $2.50/h at Google [2] • $13.46/h (4xV100) at Microsoft Azure [3] • $12.24/h (4xV100) at AWS [4] • ~$2.80/h (2xV100, 1 month) at LeaderGPU [5] • ~$3.38/h (4xV100, 1 month) at Exoscale [6] Other smaller cloud providers are in a similar price range to [5] and [6] (read: GCE, Azure and AWS are way overpriced...). Using the 2x figure from [1] and adjusting the price for the build to a 2080 Ti and an AMD R9 3950X instead of the TR results in similar figures to the article you provided. Please point me to any resources that show how the content of the article doesn't apply anymore, 2 years later. I'd be very interested to learn what actually changed (if anything). NVIDIA's new A100 platform might be a game changer, but it's not yet available in public cloud offerings. [1] https://lambdalabs.com/blog/best-gpu-tensorflow-2080-ti-vs-v100-vs-titan-v-vs-1080-ti-benchmark/ https://lambdalabs.com/blog/best-gpu-tensorflow-2080-ti-vs-v... [2] https://cloud.google.com/compute/gpus-pricing https://cloud.google.com/compute/gpus-pricing [3] https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/ https://azure.microsoft.com/en-us/pricing/details/virtual-ma... [4] https://aws.amazon.com/ec2/pricing/on-demand/ https://aws.amazon.com/ec2/pricing/on-demand/ [5] https://www.leadergpu.com/#chose-best https://www.leadergpu.com/#chose-best [6] https://www.exoscale.com/gpu/ https://www.exoscale.com/gpu/
- solidasparagus 6y ago
  You are missing TPU and spot/preemptible pricing, which need to be considered when we are talking about training cost. The big one to me is the ability to consistently train on V100s with spot pricing, which was not possible a couple of years ago (there wasn't enough spare capacity). Also, the improvement in cloud bandwidth for DL-type instances has helped distributed training a lot.
- gchamonlive 6y ago
  I don't really know if those hardware breakthroughs that the article refers to already reflects in Cloud GPU performance, but software reflects nonetheless. So even though pricing has fluctuated marginally since 2018, it is just plain faster to train a neural network today because of software advances, from what I understood.
  - qayxc 6y ago
    But that's not what the actual data says. Here's some figures from an actual benchmark [1] w.r.t. training costs: 1. [Mar 2020] $7.43 (AlibabaCloud, 8xV100, TF v2.1) 2. [Sep 2018] $12.60 (Google, 8 TPU cores, TF v1.11) 3. [Mar 2020] $14.42 (AlibabaCloud, 128xV100, TF v2.1) -- Training time didn't go down exponentially either [1]: 1. [Mar 2020] 0:02:38 (AlibabaCloud, 128 x V100, TF v2.1) 2. [May 2019] 0:02:43 (Huawei Cloud, 128 x V100, TF v1.13) 3. [Dec 2018] 0:09:22 (Huawei Cloud, 128 x V100, MXNet) So again, I have to ask where exactly do these magical improvement occur (regarding training - inference is another matter entirely, I understand that)? I've yet to find a source that supports 4x to 10x cost reductions. [1] https://dawn.cs.stanford.edu/benchmark/index.html https://dawn.cs.stanford.edu/benchmark/index.html
    - gchamonlive 6y ago
      I guess I should have been more skeptical of the articles figures. But still, if we give the benefit of the doubt, is there any scenario we might see the reduction mentioned? 1000 to 10 USD?
      - 6y ago
mtgp1000 6y ago
I trained a useful neural network and prototyped a viable [failed] startup technology something like 4 years ago on a 1080ti with a mid range CPU. It was enough to get me meetings with a couple of the largest companies in the world. Yeah it took 12-24 hours to do what I could login to AWS and accomplish in minutes with parallel GPUs...but practical solutions were already in reach. The primary changes now are buzz and possibly unprecedent rate of research progress.

qayxc 6y ago

> Now it is affordable to train a useful network on the cloud I honestly don't see how anything changed significantly in past 2 years. Benchmarks indicate that a V100 is barely 2x the performance of an RTX 2080 Ti [1] and a V100 is • $2.50/h at Google [2] • $13.46/h (4xV100) at Microsoft Azure [3] • $12.24/h (4xV100) at AWS [4] • ~$2.80/h (2xV100, 1 month) at LeaderGPU [5] • ~$3.38/h (4xV100, 1 month) at Exoscale [6] Other smaller cloud providers are in a similar price range to [5] and [6] (read: GCE, Azure and AWS are way overpriced...). Using the 2x figure from [1] and adjusting the price for the build to a 2080 Ti and an AMD R9 3950X instead of the TR results in similar figures to the article you provided. Please point me to any resources that show how the content of the article doesn't apply anymore, 2 years later. I'd be very interested to learn what actually changed (if anything). NVIDIA's new A100 platform might be a game changer, but it's not yet available in public cloud offerings. [1] https://lambdalabs.com/blog/best-gpu-tensorflow-2080-ti-vs-v100-vs-titan-v-vs-1080-ti-benchmark/ https://lambdalabs.com/blog/best-gpu-tensorflow-2080-ti-vs-v... [2] https://cloud.google.com/compute/gpus-pricing https://cloud.google.com/compute/gpus-pricing [3] https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/ https://azure.microsoft.com/en-us/pricing/details/virtual-ma... [4] https://aws.amazon.com/ec2/pricing/on-demand/ https://aws.amazon.com/ec2/pricing/on-demand/ [5] https://www.leadergpu.com/#chose-best https://www.leadergpu.com/#chose-best [6] https://www.exoscale.com/gpu/ https://www.exoscale.com/gpu/

solidasparagus 6y ago
You are missing TPU and spot/preemptible pricing, which need to be considered when we are talking about training cost. The big one to me is the ability to consistently train on V100s with spot pricing, which was not possible a couple of years ago (there wasn't enough spare capacity). Also, the improvement in cloud bandwidth for DL-type instances has helped distributed training a lot.
gchamonlive 6y ago
I don't really know if those hardware breakthroughs that the article refers to already reflects in Cloud GPU performance, but software reflects nonetheless. So even though pricing has fluctuated marginally since 2018, it is just plain faster to train a neural network today because of software advances, from what I understood.
- qayxc 6y ago
  But that's not what the actual data says. Here's some figures from an actual benchmark [1] w.r.t. training costs: 1. [Mar 2020] $7.43 (AlibabaCloud, 8xV100, TF v2.1) 2. [Sep 2018] $12.60 (Google, 8 TPU cores, TF v1.11) 3. [Mar 2020] $14.42 (AlibabaCloud, 128xV100, TF v2.1) -- Training time didn't go down exponentially either [1]: 1. [Mar 2020] 0:02:38 (AlibabaCloud, 128 x V100, TF v2.1) 2. [May 2019] 0:02:43 (Huawei Cloud, 128 x V100, TF v1.13) 3. [Dec 2018] 0:09:22 (Huawei Cloud, 128 x V100, MXNet) So again, I have to ask where exactly do these magical improvement occur (regarding training - inference is another matter entirely, I understand that)? I've yet to find a source that supports 4x to 10x cost reductions. [1] https://dawn.cs.stanford.edu/benchmark/index.html https://dawn.cs.stanford.edu/benchmark/index.html
  - gchamonlive 6y ago
    I guess I should have been more skeptical of the articles figures. But still, if we give the benefit of the doubt, is there any scenario we might see the reduction mentioned? 1000 to 10 USD?
    - 6y ago

mtgp1000 6y ago

I trained a useful neural network and prototyped a viable [failed] startup technology something like 4 years ago on a 1080ti with a mid range CPU. It was enough to get me meetings with a couple of the largest companies in the world. Yeah it took 12-24 hours to do what I could login to AWS and accomplish in minutes with parallel GPUs...but practical solutions were already in reach. The primary changes now are buzz and possibly unprecedent rate of research progress.