7 ms·

Not sure about the exact pricing shown here, but it looks like Azure blows AWS GPU pricing out of the water. Comparing specs: AWS g2.2xlarge: * DRAM: 15GB

by n00b101 10y ago

Not sure about the exact pricing shown here, but it looks like Azure blows AWS GPU pricing out of the water. Comparing specs: AWS g2.2xlarge: * DRAM: 15GB * GPU processor: 1xGK104 * 1536 CUDA cores * 2.29Tflops/s peak single precision throughput * GPU memory: 4GB * 160GB/s peak bandwidth * Price: $0.65/hr (US East, Linux) Azure NC6: * DRAM: 56GB * GPU processors: 2xGK210 * 4992 CUDA cores * 8.73Tflops/s peak single precision throughput * GPU memory: 24 GB * 480GB/s peak bandwidth * Price: $0.66/hr I expect AWS will introduce a new GPU fleet and reduce pricing of the existing G2 fleet, in response to Azure.

JoshGlazebrook 10y ago
I wish one of the main cloud platforms would significantly cut their per/GB bandwidth costs. It feels like there has been no adjustments in years. Also funny story, I was 14 or 15 and was trying to figure out a name for a stupid little chat client I was trying to make in C#, and somehow azure popped into my head. Not surprisingly azure.com was taken and just had a generic microsoft landing page (this was a little before it was announced in 2008).
- blahi 10y ago
  That's a major lock-in mechanism. Data stays in for the compute. Which is a major pity for Microsoft since all the marketing measurement startups got built on top of AWS while Azure has superior analytics tools.
- snuxoll 10y ago
  I mean, it doesn't necessarily solve anything if you're using it for public facing apps, but if you're doing a hybrid solution with Azure you can buy unmetered ExpressRoute ports and at least avoid the metered bandwidth costs back to a datacenter.
dharma1 10y ago
it sure does. I hope AWS and GCE will step up their GPU instance game
- jjallen 10y ago
  GCE should offer Tensorflow instances sometime soon. Closed alpha now. This will be a nice offering from Google which is no doubt part of the reason they open sourced Tensorflow. Which GPUs they offer and how much, nobody knows.
  - n00b101 10y ago
    > Which GPUs they offer and how much, nobody knows. They might not offer GPUs at all. It may be that Google is planning to build a cloud using their own proprietary "Tensor Processing Unit" chip designed for TensorFlow (https://en.wikipedia.org/wiki/Tensor_processing_unit https://en.wikipedia.org/wiki/Tensor_processing_unit)
    - visarga 10y ago
      TPUs are only useful for prediction, not for training.
      - mikecb 10y ago
        Unless I missed something, whether or not they are for training or inference has not been clarified.
        Eridrus 10y ago
        They mention that they use them for inference in one of the TF papers; also the focus on per watt efficiency rather than speed/memory means it's probably for inference.
  - dharma1 10y ago
    Nice. Would still prefer GPU instances I can do anything I want with
Eridrus 10y ago
Does Azure offer spot pricing? You can get those AWS instances for (iirc) 10-15 cents/hr without much interruption.
- ryan-c 10y ago
  The spot pricing seems to go up pretty quickly if you try to run more than one instance.
  - gcr 10y ago
    I've run 6 of those GPU instances for $0.60ish/hr. Pricing does go up near computer vision conference deadlines though. Many research labs are too poor to buy our own hardware. :-)
- KirinDave 10y ago
  No, but it doesn't do hourly billing so it can still be cheaper depending on your workload. Also, you tend to be getting more per instance hour on both Azure and GCE, assuming you do things those environments are good at. In Azure's case if you can get your software running on windows then it's actually quite good performance for I/O intensive things.
  - dharma1 10y ago
    Why Windows? You can run Linux on Azure
    - KirinDave 10y ago
      So... I haven't checked in the last 6 months but when I did do a comparison the Windows vms were noticably faster than the linux vms. I should check that again, it's been awhile. But it depends on your platform. Its an unpopular but somewhat verifiable fact that the Windows kernels have much more powerful async I/O support and when software targets that, it is very fast compared to linux counterparts. If your toolkit supports this, then a strong argument can be made that you don't really care so long as SSH works. Powershell is serviceable for maintenance and most deploys are fairly agnostic to other aspects of the platform. I've been doing skunkworks projects. I tried to get some stuff working in GCE and their docs are a mess and it never worked. Azure has some warts in its git push deploy stuff, but it worked the first time and I managed to diagnose the other issues.
dogma1138 10y ago
Just a small correction the K80 is a dual GPU setup which means you get 2 GPU's with 2496 "CUDA cores" each and 12GB assigned to each GPU. This isn't the same as having 1 GPU with 4992 cores and 24GB of memory. CUDA kernel scaling is pretty decent but it's not 1:1 with additional GPU's it's also very much task specific so tasks that cannot be easily parallelized will not scale well or at all at least not without you splitting the workload manually. As for the memory scaling CUDA supports P2P memcopy so you can easily move memory from 1 GPU to another (or between hosts in a network distributed configuration) but under best case scenarios this goes through a PCIE bridge (in the K80 it's onboard) but no NVLINK support on the K80 as it's 2 gens behind Pascal, if you are using P2P memcopy you do not double the memory capacity (even lose some since memcopy requires another buffer iirc). CUDA also allows you to share the memory directly between multiple GPU's (and the CPU/Host) NVIDIA calls this Unified Addressing (OpenCL 2.0 later implemented something similar as Shared Virtual Memory) but this also comes at some cost the latency is greatly increased for memory access and you really need stellar code to make everything work properly if 1 GPU tries to dereference a pointer on another GPU while it's still is in use whelp "don't panic and move on" probably won't work :P
- n00b101 10y ago
  Thanks. I was glossing over the details. The point I was trying to emphasize is that the Azure instance gives you the whole K80 board (i.e. two GK210 processors), unlike the AWS G2 instance which effectively gives you a virtualized half of a K10 board (i.e. one GK104 processor)
  - dogma1138 10y ago
    Oh yeah the Kepler 2 K80's blow the K10 out of the water. I wonder just how expensive the enterprise grade Pascal boards are that even Microsoft doesn't offer them as an option. It's really amazing just how much money NVIDIA seems to be squeezing out of the compute market (almost makes you feel bad for AMD) especially considering that Kepler is a 4 almost 5 year old architecture by now. The K80 is still going for like 4000-5000$ a pop I would love to know just how many GPU's Microsoft ended up buying.
gcr 10y ago
What about comparing against spot market prices? Those AWS GPU instances are frequently $0.10 or $0.20 rather than $0.66. Not clearly a better deal anymore.
- visarga 10y ago
  Yes, if you are ok with having your neural net suddenly killed mid-training.
  - dharma1 10y ago
    you can set your max spot price high, and/or you can periodically save the trained weights (to an EBS volume) and continue from where you left off, same as recovering from a crash. Probably not a huge deal. Here's how you do it with Tensorflow: https://www.tensorflow.org/versions/r0.10/how_tos/variables/index.html#checkpoint-files https://www.tensorflow.org/versions/r0.10/how_tos/variables/... The sweet spot would be updated AWS GPU instances at spot prices. You could spin up XX instances for a relatively short time to train fast. Needs some Kubernetes etc cluster setup for distributed training but should be doable. If the spot price is $0.1-$0.2/hr that's pretty attractive, considering that is close to the price of electricity alone
pilooch 10y ago
Well, check this one out: https://www.runabove.com/titan-x-gpu-servers.xml https://www.runabove.com/titan-x-gpu-servers.xml
- dharma1 10y ago
  That's nice, but the whole point of cloud instances is to be able to spin them up and down in large numbers when you need them, and pay by the hour
  - benaadams 10y ago
    Or pay by the minute on Azure
KirinDave 10y ago
Have they historically done so though? AWS's market leader position seems to persist even though they're actually quite a bit more expensive than Google Cloud and Azure (in particular due to the billing practices).

by n00b101 10y ago

JoshGlazebrook 10y ago

I wish one of the main cloud platforms would significantly cut their per/GB bandwidth costs. It feels like there has been no adjustments in years. Also funny story, I was 14 or 15 and was trying to figure out a name for a stupid little chat client I was trying to make in C#, and somehow azure popped into my head. Not surprisingly azure.com was taken and just had a generic microsoft landing page (this was a little before it was announced in 2008).

blahi 10y ago
That's a major lock-in mechanism. Data stays in for the compute. Which is a major pity for Microsoft since all the marketing measurement startups got built on top of AWS while Azure has superior analytics tools.
snuxoll 10y ago
I mean, it doesn't necessarily solve anything if you're using it for public facing apps, but if you're doing a hybrid solution with Azure you can buy unmetered ExpressRoute ports and at least avoid the metered bandwidth costs back to a datacenter.

dharma1 10y ago

it sure does. I hope AWS and GCE will step up their GPU instance game

jjallen 10y ago
GCE should offer Tensorflow instances sometime soon. Closed alpha now. This will be a nice offering from Google which is no doubt part of the reason they open sourced Tensorflow. Which GPUs they offer and how much, nobody knows.
- n00b101 10y ago
  > Which GPUs they offer and how much, nobody knows. They might not offer GPUs at all. It may be that Google is planning to build a cloud using their own proprietary "Tensor Processing Unit" chip designed for TensorFlow (https://en.wikipedia.org/wiki/Tensor_processing_unit https://en.wikipedia.org/wiki/Tensor_processing_unit)
  - visarga 10y ago
    TPUs are only useful for prediction, not for training.
    - mikecb 10y ago
      Unless I missed something, whether or not they are for training or inference has not been clarified.
      - Eridrus 10y ago
        They mention that they use them for inference in one of the TF papers; also the focus on per watt efficiency rather than speed/memory means it's probably for inference.
- dharma1 10y ago
  Nice. Would still prefer GPU instances I can do anything I want with

Eridrus 10y ago

Does Azure offer spot pricing? You can get those AWS instances for (iirc) 10-15 cents/hr without much interruption.

ryan-c 10y ago
The spot pricing seems to go up pretty quickly if you try to run more than one instance.
- gcr 10y ago
  I've run 6 of those GPU instances for $0.60ish/hr. Pricing does go up near computer vision conference deadlines though. Many research labs are too poor to buy our own hardware. :-)
KirinDave 10y ago
No, but it doesn't do hourly billing so it can still be cheaper depending on your workload. Also, you tend to be getting more per instance hour on both Azure and GCE, assuming you do things those environments are good at. In Azure's case if you can get your software running on windows then it's actually quite good performance for I/O intensive things.
- dharma1 10y ago
  Why Windows? You can run Linux on Azure
  - KirinDave 10y ago
    So... I haven't checked in the last 6 months but when I did do a comparison the Windows vms were noticably faster than the linux vms. I should check that again, it's been awhile. But it depends on your platform. Its an unpopular but somewhat verifiable fact that the Windows kernels have much more powerful async I/O support and when software targets that, it is very fast compared to linux counterparts. If your toolkit supports this, then a strong argument can be made that you don't really care so long as SSH works. Powershell is serviceable for maintenance and most deploys are fairly agnostic to other aspects of the platform. I've been doing skunkworks projects. I tried to get some stuff working in GCE and their docs are a mess and it never worked. Azure has some warts in its git push deploy stuff, but it worked the first time and I managed to diagnose the other issues.

dogma1138 10y ago

Just a small correction the K80 is a dual GPU setup which means you get 2 GPU's with 2496 "CUDA cores" each and 12GB assigned to each GPU. This isn't the same as having 1 GPU with 4992 cores and 24GB of memory. CUDA kernel scaling is pretty decent but it's not 1:1 with additional GPU's it's also very much task specific so tasks that cannot be easily parallelized will not scale well or at all at least not without you splitting the workload manually. As for the memory scaling CUDA supports P2P memcopy so you can easily move memory from 1 GPU to another (or between hosts in a network distributed configuration) but under best case scenarios this goes through a PCIE bridge (in the K80 it's onboard) but no NVLINK support on the K80 as it's 2 gens behind Pascal, if you are using P2P memcopy you do not double the memory capacity (even lose some since memcopy requires another buffer iirc). CUDA also allows you to share the memory directly between multiple GPU's (and the CPU/Host) NVIDIA calls this Unified Addressing (OpenCL 2.0 later implemented something similar as Shared Virtual Memory) but this also comes at some cost the latency is greatly increased for memory access and you really need stellar code to make everything work properly if 1 GPU tries to dereference a pointer on another GPU while it's still is in use whelp "don't panic and move on" probably won't work :P

n00b101 10y ago
Thanks. I was glossing over the details. The point I was trying to emphasize is that the Azure instance gives you the whole K80 board (i.e. two GK210 processors), unlike the AWS G2 instance which effectively gives you a virtualized half of a K10 board (i.e. one GK104 processor)
- dogma1138 10y ago
  Oh yeah the Kepler 2 K80's blow the K10 out of the water. I wonder just how expensive the enterprise grade Pascal boards are that even Microsoft doesn't offer them as an option. It's really amazing just how much money NVIDIA seems to be squeezing out of the compute market (almost makes you feel bad for AMD) especially considering that Kepler is a 4 almost 5 year old architecture by now. The K80 is still going for like 4000-5000$ a pop I would love to know just how many GPU's Microsoft ended up buying.

gcr 10y ago

What about comparing against spot market prices? Those AWS GPU instances are frequently $0.10 or $0.20 rather than $0.66. Not clearly a better deal anymore.

visarga 10y ago
Yes, if you are ok with having your neural net suddenly killed mid-training.
- dharma1 10y ago
  you can set your max spot price high, and/or you can periodically save the trained weights (to an EBS volume) and continue from where you left off, same as recovering from a crash. Probably not a huge deal. Here's how you do it with Tensorflow: https://www.tensorflow.org/versions/r0.10/how_tos/variables/index.html#checkpoint-files https://www.tensorflow.org/versions/r0.10/how_tos/variables/... The sweet spot would be updated AWS GPU instances at spot prices. You could spin up XX instances for a relatively short time to train fast. Needs some Kubernetes etc cluster setup for distributed training but should be doable. If the spot price is $0.1-$0.2/hr that's pretty attractive, considering that is close to the price of electricity alone

pilooch 10y ago

Well, check this one out: https://www.runabove.com/titan-x-gpu-servers.xml https://www.runabove.com/titan-x-gpu-servers.xml

dharma1 10y ago
That's nice, but the whole point of cloud instances is to be able to spin them up and down in large numbers when you need them, and pay by the hour
- benaadams 10y ago
  Or pay by the minute on Azure

KirinDave 10y ago

Have they historically done so though? AWS's market leader position seems to persist even though they're actually quite a bit more expensive than Google Cloud and Azure (in particular due to the billing practices).