5 ms·
That $6.50/hr rate might be the big deal here. Amazon does offer instances with a V100 GPU (https://aws.amazon.com/ec2/pricing/on-demand/ https://aws.amazon.com
by minimaxir 9y ago
That $6.50/hr rate might be the big deal here. Amazon does offer instances with a V100 GPU (https://aws.amazon.com/ec2/pricing/on-demand/ https://aws.amazon.com/ec2/pricing/on-demand/, the P3 instances), but if you're training something like ImageNet, you'll want the biggest image (p3.16xlarge) at $24.48/hr.
Attaching a VM of similar power to a TPU on Google Compute Engine is much cheaper (https://cloud.google.com/compute/pricing https://cloud.google.com/compute/pricing, n1-highmem-64, +$3.78/hr to the TPU cost for $10.28/hr total).
Per recent benchmarks for training ImageNet (https://dawn.cs.stanford.edu/benchmark/ https://dawn.cs.stanford.edu/benchmark/), training ImageNet on a p3.16xlarge cost $358, when this post claims it'll cost less than $200. (EDIT: never mind; the benchmark uses ImageNet-152, and Google compares TPU performance against ImageNet-50) Interesting.
- twtw 9y agoWhy does it make any sense to compare the price/hour for a single TPU (4 ASICs) to the price/hour for p3.16xlarge, which has 8x V100? Also, that benchmark cost of $358 is for Resnet-152, not Resnet-50.
- minimaxir 9y agoWhoops, I misread, added edit.
- douglasfshearer 9y agoP3.16x benchmark is ResNet-152, TPU cost of $200 was for ResNet-50. Tensorflow benchmarks show ResNet-152 resulting in 2.4x lower throughput than ResNet-50. [0] [0] https://www.tensorflow.org/performance/benchmarks https://www.tensorflow.org/performance/benchmarks
- Cyph0n 9y agoA better comparison would be the f1.16xlarge[1] instance @ ~$4/hr. It comes with 8 FPGAs (12 Gbps link) and 64 vCPUs. [1]: https://aws.amazon.com/ec2/instance-types/f1/ https://aws.amazon.com/ec2/instance-types/f1/ Edit: I'm genuinely curious about why this comment is getting downvotes.
- dharma1 9y agoDo much deep learning training with the FPGAs?
- Cyph0n 9y agoNope, but I'm sure others do.
- jongomez 9y agoMicrosoft uses FPGAs for Deep Learning. Source: https://techcrunch.com/2017/08/22/microsoft-brainwave-aims-to-accelerate-deep-learning-with-fpgas/ https://techcrunch.com/2017/08/22/microsoft-brainwave-aims-t...
- twtw 9y ago... for inference. I don't know of anyone who takes training on FPGAs seriously. They tend to get crushed by GPU/TPU/other ASIC in throughput, perf/watt, and perf/$.
- p1esk 9y agoTraining on FPGAs starts to make a lot of sense when you consider low bit precision computation (e.g. DoReFa-Net).
- boulos 9y agoDisclosure: I work on Google Cloud. I didn’t downvote you, but presumably people disagree with “Here’s an FPGA” as comparable to being given a working piece of hardware. That is, would you have said that the best comparison to a V100 is this same FPGA box? I (and others) get what you were trying to say: TPUs are ASICs that aren’t general purpose at all, so an FPGA is a better comparison than a more general purpose GPU. As an end user, that just isn’t true though. If someone hands you an f1.16xlarge, you have to build your own psuedo-chip for machine learning. While with this offering, TensorFlow handles the acceleration / offload for you.
- borramakot 9y agoBack of the envelope, a TPU costs a little more than 2x as much as a Volta on AWS P3, and delivers a little less than 2x the performance (180 TOPs for the TPU, 100 for Volta). On a raw performance/$ metric, I'm not sure the TPU is that interesting. It might be worth it if I were willing to pay a huge amount to get back results from an experiment faster, by using lots of TPUs- distributed learning on GPUs doesn't seem easy yet.
- surajrmal 9y agoIt'll make a lot more sense when the TPU pods they alluded to come out.
- jlebar 9y agoI know people don't know what to expect from tpu performance, but does anyone actually get 100tops out of Volta? I thought you'd have to spin the tensorcores and never touch memory, which is...not realistic. I know you hedged by saying "back of the envelope", but I'd much rather compare on real benchmarks than based on cited peak performance numbers, which are kind of meaningless.
- deepnotderp 9y agoThis is true of the TPU as well, check out their paper's utilization numbers. If you ignore one outlier at ~90% utilization, their utilization plummets. I'm glad people are finally looking past the b.s. "peak" numbers for once though.
- jlebar 9y agoHas Google published data on the memory bandwidth of TPU v2 (aka "cloud TPU")? I'm having trouble finding it. In any case I agree, we shouldn't be looking at the stated peak compute of either of the chips. (Disclaimer: I work at Google on XLA, and have in the past worked on TPUs.)
- boulos 9y ago
- staticfloat 9y agoJust a minor nitpick that ResNet is the model you are referring to, ImageNet is the dataset that ResNet is trained on.
- boulos 9y agoDisclosure: I work on Google Cloud. Note that the post says “less than $200” not $200. There are lots of values between 0 and 200. What we’d love is for third-party folks like yourself to do the comparison (which I know you can, Max!)