8 ms·
New Compute Engine A2 VMs–First Nvidia Ampere A100 GPUs in the Cloud
- EwanToo 6y agoThere's some more technical information on the A100 here https://www.anandtech.com/show/15801/nvidia-announces-ampere-architecture-and-a100-products https://www.anandtech.com/show/15801/nvidia-announces-ampere...
- anon102010 6y agoI wish google / aws would avoid the overlapping names where possible. the "A" series on AWS = AMD instances The "A" series on GCP = Nvidia instances. I know - probably on no ones radar at all :)
- boulos 6y agoDisclosure: I work on Google Cloud. Even worse is that for GCE, A was for AMD originally (and N was for iNtel). In any case, this A is for Accelerator.
- jeffbee 6y agoAre there any papers or blogs about how these GPUs are attached to the host? I find it interesting that you can get a VM with 96 vCPUs, which I assume amounts to a whole box (2x24-core hyperthreaded Xeon CPUs?) but either 8 or 16 GPUs. How does that keep from stranding 8 GPUs? Is there some kind of rack-wide PCIe switch that can attach GPUs to various hosts or ??
- boulos 6y agoWe sadly don’t talk about how we rack these at all, but the folks at Facebook have made their OCP designs public for vaguely similar systems. However, I’ll note that the 16 A100s here are way more expensive than the cpu cores (and we can just run vanilla VMs on those left over cores if really needed).
- mcspiff 6y agoAlso ARM, sometimes, on AWS. A1 = first generation of Gravitron. Of course second generation uses names like m6g to keep us on our toes..
- sradman 6y agoAnd the new server-side ARM competitor to the Graviton is the Ampere Altra [1] which has nothing to do with the NVIDIA Ampere. [1] https://news.ycombinator.com/item?id=22475036 https://news.ycombinator.com/item?id=22475036
- cperciva 6y agoIt's even worse than that. On AWS, A1 is ARM. The successor to A1 is M6G. Meanwhile M5A, as you note, is AMD.
- codemk8 6y agoFor azure, NC is for vms with NVidia card. Seems that they deliberately choose different terms.
- txcwpalpha 6y agoWorse than that, "A"/"a" on AWS can actually mean AMD or ARM. An "a1.large" instance is an instance with a first-gen Graviton ARM CPU (whereas the second-gen Graviton2 ARM CPUs are something like "c6g.large"). A "c5a.large" is an instance with a x86 AMD CPU.
- ricardo81 6y agoI've only had a very slight dip into the water of the GPGPU world in the past 5 years, but it is obvious that Nvidia have the lion's share of the market when it comes to hosted solutions. Still not sure why competitors don't have their GPU's offered in cloud services. At the time, it seemed the alternatives offered a more economical alternative. I was building a GPU version of Hashcash at the time fwiw.
- fluffything 6y ago> Still not sure why competitors don't have their GPU's offered in cloud services. AMD has many GPGPUs, but while they have kind of ok-ish raw numbers, their software stack makes quite poor use of them. Users care about perf/$, and nvidia is king there. For deep-learning and other apps, hardware without software is useless.
- JoeAltmaier 6y agoAnd if the population is not uniform (and it never is, as its a growing market and new resources have different feature sets) it can be hard to deploy some hardware-sensitive applications. We needed hardware support for encryption in our teleconferencing mix node. Had to contract specially, so we could deploy only on hardware that supported that (was modern enough).
- Nokinside 6y agoNvidia has software advantage. Unlike with AMD, you have libraries and ways to build software that runs fast: CUDA, cuDNN.
- BadInformatics 6y agoThey do, but (as described elsewhere in this thread), the level of polish isn't quite up to that of the CUDA stack. Add onto that the relatively abysmal developer evangelism from AMD and it's not surprising that adoption is so low...
- pjmlp 6y ago
- NiekvdMaas 6y agoIn case you're wondering about pricing: the V100 costs $1260/GPU/month and this A100 will have about 2.5x its performance [1]. A n2d-highmem-96 instance is $4377 per month. So for the maxed out a2-megagpu-16g I would expect around $54k per month before usage discounts etc. 1. https://developer.nvidia.com/blog/nvidia-ampere-architecture-in-depth/ https://developer.nvidia.com/blog/nvidia-ampere-architecture...
- fluffything 6y agoWhat's the perf/$ ratio for these different instances ? I've also read that the A100 can deliver 4x the performance on some DL workloads.
- selectodude 6y agoI feel like if you’re maxing one of these clusters for an entire month at a time, it might still be cheaper to just buy one.
- manigandham 6y agoThat's true for any cloud server.
- 867-5309 6y agountrue Intel Xeon Gold 5120 14Core @ £1200 used renting 1Core @ £1.20/mo. would take 6 years to fully pay for that single core alone, and that's excluding the 512MB RAM, 10GB SSD and unlimited 400Mb/s bandwidth
- sbierwagen 6y agoHere's a completed ebay auction from April 20th that sold a 5120 for $699: https://www.ebay.com/itm/Intel-Xeon-Gold-5120-SR3GD-14-Core-2-20GHz-10-40GT-s-UPI-19-25MB-LGA3647-U159103/313059429725?hash=item48e3cbcd5d:g:gCoAAOSwni1ejMs1 https://www.ebay.com/itm/Intel-Xeon-Gold-5120-SR3GD-14-Core-... I know the pound hasn't been doing well lately, but surely it hasn't been that bad.
- zetazzed 6y agoI think this is the most in-depth article on Ampere: https://developer.nvidia.com/blog/nvidia-ampere-architecture-in-depth/ https://developer.nvidia.com/blog/nvidia-ampere-architecture... Lots of architectural changes like MIG, new floating point formats, etc. Great to see GCP getting VMs out pretty soon after launch so people can start kicking the tires.
- g_airborne 6y agoVery cool! Does anyone know how is software support for all these features? It seems that TF doesn’t support the TP32/16 types as of yet. Is this something only CUDA engineers can use right now? It does seem a little fishy to me that NVIDIA often boasts with figures like 10x performance upgrade whilst in practice those are only possible if you use one of their non-default float types which are hardly supported in most deep learning libraries :(
- zetazzed 6y agoBoth PyTorch and Tensorflow teams have announced they'll support TF32. By design, it interoperates well with existing code, since calling code can just treat it as a regular FP32 value.
- beamatronic 6y agoImagine a Beowulf cluster of these
- alfonsodev 6y agoAs a ML beginner, seeing this new offerings, does a local setup(2x nvidia 2080 8gb) make sense or it's better to learn using the cloud and let hardware get even cheaper. The cloud it's a bit scary from learners perspective because it's not exactly clear how much power one would need to practice the core concepts and see actual results. In the other hand a pc build, it's an upfront investment, less scary because it's a fix cost, but also feels risky in case things progress quickly and hardware gets outdate soon.
- dna_polymerase 6y agoIf you have those GPUs already use them. If you are looking into buying them, check out Paperspace or Colab first. Think about the fact, that you will probably not use your own cards the whole time and with Paperspace you only pay for your usage (and some fixed amount for disk usage I think) and Colab is even cheaper in that regard.
- boulos 6y agoDisclosure: I work on Google Cloud. Honestly, I’d start with Colab until you decide you need “dedicated hardware”. It’s better to focus on learning before you decide “Okay, I’m serious about this now”.
- txcwpalpha 6y agoI would completely avoid things like this A2 instance (and other accelerated instances on other cloud providers) if you’re a self-described beginner. The GPU-accelerated instances in public clouds are terrifyingly expensive and way overkill for any kind of beginner/practice work. Only approach these types of instances once you actually have a handle on how much power you need.
- ksec 6y agoGCP seems to be getting new hardware way faster than AWS. Both AMD's Zen 2 and Nvidia's Ampere.
- rjhacks 6y agoI understand it's exciting to see introductions of new machine types and new GPUs, but for it to mean anything Google should instead get its house in order on the GPUs they already offer. Getting an n1 instance with a Tesla T4 GPU in any datacenter I've tried has a <50% success rate on any given day ("resource unavailable" more often than not, they just don't seem to have enough of them), which is _hugely_ damaging to our ability to rely on the cloud for our workload. Worse, there's no way for me to work around it: I'd be willing to switch zones, or machine type, or GPU type, but there is no dashboard or support guidance that'll tell me if there's any such configuration that'll be reliably available. Because of that, seeing this A100 announcement is just a bummer, as I fear it'll be just another "resource unavailable" GPU...
- ckleban 6y agoDisclaimer: I work at google cloud. Sorry to hear you have experienced this. Customers can experience stock outs sometimes based on a variety of factors but we can surely help you out as we have T4 GPU capacity available and like you said, we may direct you to a different zone or region. Open up a support case on the issue and we can help you out.
- fhrifjr 6y ago100G of network bandwidth for 8-16 GPUs? Is it a joke? One GPU will consume 50G easily. Who is going to use it?