11 ms·
New P2 Instance Type for Amazon EC2 – Up to 16 GPUs
- clishem 10y agoPricing?
- moonbug 10y agohttps://aws.amazon.com/ec2/instance-types/p2/ https://aws.amazon.com/ec2/instance-types/p2/ From 0.90-14.40 USD/hr.
- cm2187 10y agoStupid question: are GPU safe to be shared by two tenants in a datacenter? I read previously that there are very few security mechanisms in GPU, in particular that the memory is full of interesting garbage. So I would assume no hardware enforced separation between the VMs too.
- moonbug 10y agoThe GPUs aren't shared between VMs, each GPU is dedicated to the VM it is allocated to. The GPU memory gets wiped on initialisation, just like host memory, so there's no leakage between successive VMs allocated the same hardware. If you do absolutely care about not being multitennanted, you can stipulate exclusive hw allocation when requesting the VM.
- _pmf_ 10y ago> The GPU memory gets wiped on initialisation Well, an attempt is made. As we can see in the case of WebGL, there are a multitude of interesting effects.
- moonbug 10y agoCan't speak to what happens on a Windows box, but across AWS instances it's definitely the case.
- jon-wood 10y agoAt the sort of scale these instances run I'd be surprised if you're sharing any physical hardware at all, my guess is that they provision a single VM on each GPU instance.
- boulos 10y agoThat's not mentioned in the article, did you work on this? Disclosure: I work on Google Cloud.
- corysama 10y agoNvidia has been doing a lot of work on hardware virtualization of GPUs. A goal of which is to run dozens of low-requirements desktops (think users running MS Office in separate Windows VMs) off of a single high-end server with a single GPU. http://www.nvidia.com/object/grid-technology.html http://www.nvidia.com/object/grid-technology.html
- asendra 10y agoDamn, I would love to have an excuse to play around with this.
- Smerity 10y ago$0.9 per K80 GPU per hour, while expensive, opens up so many opportunities - especially when you can get a properly connected machine. Just as an example of the change this entails for deep learning, the recent "Exploring the Limits of Language Modeling"[1] paper from Google used 32 K40 GPUs. While the K40 / K80 are not the most recent generation of GPU, they're still powerful beasts, and finding a large number of them set up well is a challenge for most. In only 2 hours, their model beat previous state of the art results. Their new best result was achieved after three weeks of compute. With two assumptions, that a K80 is approximately 2 x K40 and that you could run the model with similar efficiency, that means you can beat previous state of the art for ~$28.8 and could replicate that paper's state of the art for ~$7257.6 - all using a single P2 instance. While the latter number is extreme, the former isn't. It's expensive but still opens up so many opportunities. Everything from hobbyists competing on Kaggle competitions to that tiny division inside a big company that would never be able to provision GPU access otherwise - and of course the startup inbetween. * I'm not even going to try to compare the old Amazon GPU instances to the new one as they're not even in the same ballpark. They have far less memory and don't support many of the features required for efficient use of modern deep learning frameworks. [1]: https://arxiv.org/abs/1602.02410 https://arxiv.org/abs/1602.02410
- deleted 10y ago[deleted]
- lqdc13 10y agoStill seems much better to buy your own gtx 1080 for $700 which you would have spent in a month playing with parameters on these instances. And at the end of the month you still have a modern GPU to play video games on etc. Of course if you have money to burn, these are great.
- madeofpalk 10y agoI question the amount of people that would be buying these for research AND gaming.
- 10y ago
- mrb 10y agoKeep in mind the Nvidia K80 is a 2 years old Kepler GPU. Nvidia launched 2 newer microarchitectures since then: Maxwell, Pascal. I would expect to see some P100 Pascal GPU "soon" on AWS. Maybe 6 months? (Maxwell's severely handicapped double precision performance reduces its utility for many workloads.)
- gcp 10y agoThis is a big issue because multiple DCNN optimizations in CuDNN are only applicable to the newer generations. You could be looking at an order of magnitude worse performance than expected.
- n00b101 10y ago> Nvidia K80 is a 2 years old Kepler GPU ... I would expect to see some P100 Pascal GPU "soon" on AWS. Maybe 6 months? Not likely. The AWS CG1 instance type was introduced in 2010 (NVIDIA Tesla C2050 GPU). The G2 instance type was introduced in 2013 (NVIDIA GRID K520 GPU). Now the P2 instance type has been introduced in 2016 (NVIDIA Tesla K80 GPU). At this rate, we probably won't see another GPU fleet in AWS until 2019.
- mrb 10y agoI never realized Amazon was so consistently late in deploying current-gen GPUs. How lame. Sounds like an opportunity for a competitor to beat them.
- deleted 10y ago[deleted]
- raverbashing 10y agoOne thing I discovered recently is that for GPU machines your initial limit on AWS is 0 (meaning you have to ask support before you start one for yourself) (This might be an issue of my account though - having had only small bills so far)
- fletchowns 10y agoWonder if it has anything to do with stolen AWS credentials being used for crypto currency mining.
- raverbashing 10y agoYes, this might also be a possibility
- hueving 10y agoDoubtful, if you're using stole credentials you don't need to care about efficiency and you can just use whatever instances are available to the account. They're likely doing it as a way to control demand spikes because they have such a smaller pool of GPU instances to go around. Also, it prevents Joe Schmo from wasting one to run a web server because he doesn't understand performance and "just picked the best one".
- RossM 10y agoInteresting - we started launching GPU workloads recently without bumping into that. Might be limited to VPC-only?
- raverbashing 10y agoI guess if you are a regular user (and have paid some invoices already) the limits will be adjusted automatically
- vacri 10y agoIf you've previously asked for a limit increase, it may be grouped with the other server types - I notice that when the limits are increased, all server types increase together.
- paukiatwee 10y agoFor anyone who interested in ML/DL on cloud. Google Cloud Platform just released Cloud ML beta with different pricing model, see https://cloud.google.com/ml/ https://cloud.google.com/ml/ Cloud ML costing $0.49/hour to $36.75/hour, compared to AWS $0.900/hour to $14.400/hour The huge different of $36.75/hour (Google) compared to $14.400/hour (AWS) make me wonder what Cloud ML are using, they mentioned GPU (TPU?) but not exact GPU model.
- viksit 10y agoFor some definition of "release". This is something they've had in beta for a while, and its only accessible to people who are already in the "beta" program for cloud ML. Unlike AWS which offers it _now_.
- paukiatwee 10y agoI believe previously is "alpha" release that required signup invitation form. Now "beta" release is open to anyone with Google Account.
- aub3bhat 10y agoThere are two issues, Google offers a black box service where your Tensor Flow model gets optimized. While AWS offers a generic instance which is far more flexible and can be used with multiple frameworks. Second AWS offers spot pricing on GPU instances (though not on P2 instances yet) which can offer significant discount depending on the type of workload. I would say that while I still have to evaluate Google Cloud ML, AWS offers a far far better experience and cost effectiveness.
- nik736 10y agoAnyone knows if video transcoding on GPUs (with FFMPEG) is viable nowadays? If yes, what are the gains?
- mwilcox 10y agoThey have chips dedicated to video encoding/decoding with the latest cards doing h265. Nvidia has an ffmpeg patch here https://developer.nvidia.com/ffmpeg https://developer.nvidia.com/ffmpeg
- nenreme 10y agoDecoding is good but hw encoding usually has very bad quality/size ratio compared to best software encoders.
- scrabble 10y agoThis depends on if you're doing live transcoding or VOD transcoding. I work with live transcode, and it can be beneficial to run on g2 instances. Running c4.4xlarge I can transcode a good number of 1080@30 in with 1080/720/480/360/240@30 out. With a proper g2 instance I can transcode more simultaneously. So cost efficiency really depends on sustained traffic levels. I scale out currently using haproxy and custom code that monitors my pool and scales appropriately. But I monitor sustained traffic levels to know when it makes financial sense to scale up. If your main concern is transcode speed, CPU is likely sufficient -- I am unable to transcode faster than real time with live transcode.
- corobo 10y ago> I am unable to transcode faster than real time with live transcode Well, yeah. Maybe quantum computing will change that one day! What is it you're working on? Colour me intrigued.
- grhmc 10y agoI'm not sure if you're kidding or not, but just in case: it is desirable to be able to encode faster than real time (ie: 1 hour of media takes less than 1 hour of time to transcode) so you have leftover processing cycles to do other work.
- seanwilson 10y agoHow do Amazon (or any other cloud provider) make sure they have enough of these machines to cope with the demand for them without getting too many?
- tachion 10y agoYou pretty much answered your own question. First, they have 'too many', second, if they dont, they oversubscribe, third, after running your business for a while, you can do some math on your stats and predict future demand with accuracy that's 'good enough'.
- raverbashing 10y agoI don't think you can oversubscribe GPUs But at the price they're charging, if demand exists, they just call nVidia and order another batch of cards
- DonHopkins 10y agoAnd fortunately, they have enough computing power to do the math.
- dmichulke 10y agoThey lowball with the offer (#instances) and have price regulate the demand side. After a few months they can estimate the demand and increase supply accordingly (iff profitable)
- dx034 10y agoA lot of people will use spot instances to get cheaper computer power for non-critical calculations. So they can control demand by the price for spot instances. Also, they'll get a lot of instances up and running before publishing the post. At the moment, p2 spot prices seem very low (~9c for p2.xlarge). Especially in the GPU space, a lot of calculations will likely not be very time-sensitive. The spot market is optimal to control demand and ensure high utilization.
- idunno246 10y agoThey estimate, sometimes well sometimes not. Recently Ive seen a region that ran out of an instance type, launching gives you an error, and you're sol for days if you can't adjust. Reserved instances are more than just a price reduction
- topbanana 10y agoAll of the instances are powered by an AWS-Specific version of Intel’s Broadwell processor, running at 2.7 GHz. Does anyone have any more information about this? Are the chips fabricated separately or is it microcode differences?
- idunno246 10y agohttp://m.youtube.com/watch?v=JIQETrFC_SQ http://m.youtube.com/watch?v=JIQETrFC_SQ I think this is the right talk by him, but there's one where he implies some differences. For instance, that because they can promise exact temperature ranges they can clock them differently. My guess is same fab, just binned differently, or maybe different packaging.
- ComodoHacker 10y agoPerhaps with overclocking unlocked via microcode.
- phs318u 10y agoMy first thought: "I wonder what the economics are like, re: cryptocurrency mining?" My second thought: "I wonder if Amazon use their 'idle' capacity to mine cryptocurrency?" With respect to my second thought, at their scale, and at the cost they'd be paying for electricity, it could quite possibly be a good hedge.
- dharma1 10y agoThey sell their idle capacity at spot prices. I am really happy they have upgraded the GPU instances
- hueving 10y agoAs far as I understand it, the only way to profitably mine bitcoin is to have super cheap power and ASICs specific to bitcoin mining. Essentially, GPUs won't cut it because you're competing with Chinese miners using ASICs attached almost directly to dirty, cheap electricity generation.
- askmike 10y agoYou are right about Bitcoin specifically, but there is a second market for altcoins which use different PoW algorithms [0]. For a lot of these ASICs are either not on the market, or not economically feasible, in which cases GPUs still dominate. But most of these markets are very small compared to Bitcoin. [0] https://whattomine.com/ https://whattomine.com/
- kobeya 10y agoHe didn't say bitcoin.
- cloudjacker 10y agoYou don't understand it. Mine other cryptocurrencies that are profitable, or may become profitable. It is VERY, as in EXTREMELY, profitable to mine large stakes in cryptocurrencies that the market hasn't discovered yet. Especially ones that aren't even listed on an exchange yet and mining is the only way to earn it. The Darkcoin (now renamed to the politically non-threatening name DASH) instamine was a notoriously profitable use of GPU clusters on AWS.
- sysexit 10y agoPretty classless how Jeff describes TensorFlow as an "Open Source library," without atributing it to Google.
- lsaferite 10y agoDoes every mention of an Open Source library or project morally require attribution to the original authors or something?
- cloudjacker 10y agoNo, lol. The license stipulates the conditions, and the common licenses only have the condition that the code or final product make the attribution somewhere in the GUI.
- sysexit 10y ago> Does every mention of an Open Source library or project morally require attribution to the original authors or something? No. But in my view had this been written by an independent journalist, they would describe it as "TensorFlow by Google". The project is virtually completely driven by Google, so "By Google" is a more distinguishing classification than "Open Source". We can speculate but strong impression is that Amazon doesn't want to give kudus to Google.
- bdcravens 10y agoNot sure that Google would agree. Looking to Google's own words about open sourcing the project: "We hope to build an active open source community that drives the future of this library, both by providing feedback and by actively contributing to the source code."
- __mp 10y agoMeteoSwiss uses a 12 node cluster with 16 GPUs each to compute its weather forecasts (Piz Escha/Kesch). The machine is in operation since more than a year. We were able to disable the old machine (Piz Albis/Lema) last week. The 1.1km forecast runs on 144 GPUs, the 2.2km probabilistic ensemble forecast is computed on 168 GPUs (8 GPUs or 1/2 node per ensemble member). The 7km EU forecast is run on 8 GPUs as well.
- n00b101 10y agoWhat kinds of numerical algorithms are used for this? e.g. Monte Carlo or Finite Element Analysis?
- __mp 10y agoIt's a simple finite differences discretization on a rectangular grid. The computations are horizontally explicit and vertically implicit (data dependencies in the vertical direction). Here's the scientific description of the model: http://www2.cosmo-model.org/content/model/documentation/core/cosmoDyncsNumcs.pdf http://www2.cosmo-model.org/content/model/documentation/core...
- deleted 10y ago[deleted]
- epberry 10y agoYess thank you, thank you, thank you. I was just signing up for the Azure N Series preview but we're good to go now :).
- ravenstine 10y agoSounds like a great way to build a custom render farm. My home comouter has a dirt cheap GPU but it works well enough for basic modeling & animation. Terrible for rendering, though. I've been thinking of using ECS to build a cluster of renderers for Maya that I can spin up when needed and scale to the appropriate size. I don't know for certain if it's cheaper than going with a service, but it sounds like it is(render farm subscriptions cost hundreds), and I would get complete control over the software being used. I am glad to hear that Amazon is doing this. Granted, I'm more of a hobbyist in this arena, so maybe it wouldn't work for someone more serious about creating graphics.
- ajaimk 10y agoPriced this config (or close enough) on http://www.thinkmate.com/system/gpx-xt24-2460v3-8gpu http://www.thinkmate.com/system/gpx-xt24-2460v3-8gpu Comes to just under $50,000 for the server or roughly 4.5 months @ $14.40
- visarga 10y agoThat's months of continuous use. If you have periods when you don't use it, then you'd have to take the occupation percentage into consideration. It's pretty wasteful not to use a 50K$ server at maximum.
- smw 10y agoAlso, doesn't include electricity, cooling, data center space, all of which might be significant?
- chrisconley 10y agoThis is great - we'll try to get our Tensorflow and Caffe AMI repo updated soon: https://github.com/RealScout/deep-learning-images https://github.com/RealScout/deep-learning-images
- ivan_ah 10y agoAccording to [1], the K80 GPUs have the following specs: Chips: 2× GK210 Thread processors: 4992 (total) Base clock: 560 MHz Max Boost: 875 MHz Memory Size: 2× 12288 Clock: 5000 Bus type: GDDR5 Bus width: 2× 384 Bandwidth: 2× 240 GB/s Single precision: 5591–8736 GFLOPS (MAD or FMA) Double precision: 1864–2912 GFLOPS (FMA) CUDA compute ability: 3.7 Is that a good deal for $1/hour? (I'm not sure if a p2.large instance corresponds to use of one K80 or half of it) How much would it cost to "train" ImageNet using such instances? Or perhaps another standard DDN task for which the data is openly available? ______ [1] https://en.wikipedia.org/wiki/Nvidia_Tesla#cite_ref-19 https://en.wikipedia.org/wiki/Nvidia_Tesla#cite_ref-19
- spullara 10y agoIt is interesting to compare this to NVidia's DGX-1 system. That server is based on the new Tesla P100 and uses NVLink rather than PCIe (about 10x faster). It boasts about 170 Tflops vs the p2.16xlarge's 64 Tflops. If you run the p2.16xlarge full time for a year it would cost about the same as buying a DGX-1. Presumably Amazon releases their GPU instances on older hardware for cost savings. http://www.nvidia.com/object/deep-learning-system.html http://www.nvidia.com/object/deep-learning-system.html