10 ms·
GPUs for Google Cloud Platform
- kozikow 10y agoNow it would be great if kubernetes on GKE would work nicely with GPUs. It's still in the works: https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/gpu-support.md https://github.com/kubernetes/kubernetes/blob/master/docs/pr... .
- n00b101 10y agoGreat news!
- timdorr 10y ago> Google Cloud GPUs give you the flexibility to mix and match infrastructure. You’ll be able to attach up to 8 GPU dies to any non-shared-core machine... Wow, that's impressive. One thing I've loved about GCE has been the custom sizing. This takes it even further, so we don't have to buy what we don't need. Looking forward to seeing the pricing on this. Looks like they're going to heavily compete with AWS on this stuff.
- boulos 10y agoYes, I foresee a lot of 8 vCPUs VMs with 8 giant GPU dies attached to them. And that's just fine by us! Disclosure: I work on Google Cloud (and want to sell you some GPUs!)
- matt_wulfeck 10y agoOne of the HUGE advanatages of GCE/AWS is that they will gobble up 100% of the unused resources for their own computation. Nothing is wasted, and the machines basically pay for themselves. Compare this was something like oracle, which simply can't consume the unused resources in order to discount the hardware effectively. They can't beat GCE/AWS at the cloud game until this changes.
- alberth 10y agoCan you explain this in more detail about GCE/AWS bs Oracle. What "unused resources" are used for "their own compitation"? How does this "pay for itself"?
- ddorian43 10y agoHe's saying they have their own apps (like google search) that can use vps that aren't currently rented to do their own computations (like pagerank etc).
- reacharavindh 10y agoDid you mean to say GCE/AWS could gobble up "unpaid for" resources for their own computation? Curious because if I pay for using a P100 for 2 hours - it is 100% available to me and not virtualized of any sort right? However, if I use it only for 2 hours that day, and no one else were ready and waiting to buy the time on those, then Google uses it internally until...
- boulos 10y agoFrom our post: > GPUs are offered in passthrough mode to provide bare metal performance. Up to 8 GPU dies can be attached per VM instance including custom machine types. When you're using it, it's 100% yours. Disclosure: I work on Google Cloud (and pitched in on this!)
- matt4077 10y agoI'm not sure if Amazon has enough batch-processing type work to make this effective. You can't run amazon.com with these resources because their needs are inflexible. Actually not sure about Google either – their preemtible instances are sold at an 80% discount, so that puts an upper bound on the usefulness of these machines for them.
- matt_wulfeck 10y agoAmazons largest "batch" job is their holiday shopping season. They must purchase the horsepower to get them through that season, and then the rest of the year they can resell it. I believe this was the original motivation for AWS.
- deleted 10y ago[deleted]
- AlexCoventry 10y agoIs there any public access to the TPUs?
- mikecb 10y agoIf you use the cloud ml service.
- mnbbrown 10y agoReally? I was under the impression they just use standard compute instances to power Cloud ML?
- boulos 10y agoNot direct access, yet (they're not sufficiently programmable and isolatable). But as mikecb alludes to, we have a number of services like Translate, Vision, Speech, and a few others in Alpha that may be trained using TPUs (see https://cloud.google.com/ml https://cloud.google.com/ml for more). For models you write yourself and have the Cloud ML service train, they will use our regular VMs today and as this blog post says, GCE will soon have GPUs (and therefore, Cloud ML will expose them, too). Disclsoure: I work on Google Cloud (and sorry for the confusion!)
- boxerab 10y agoVery very happy to finally see AMD GPUs in cloud.
- wyldfire 10y agoI love to root for an underdog. But I've been pulling for AMD for years and they just can't get their linux support to the same level of NVIDIA's. I speculate that they have undiagnosed design errors in their driver.
- boxerab 10y agoYou should check out ROCm http://hothardware.com/news/amd-rocm-13-adds-support-for-polaris-and-opencl http://hothardware.com/news/amd-rocm-13-adds-support-for-pol... They are revamping their entire linux stack for HPC, heterogenous compute.
- slizard 10y agoStill no OpenCL, still only promised as a "preview" for mid Dec. I'm still very worried that AMD will fail, not because of the hardware, but due to their lacking software stack.
- wyldfire 10y agoFool me several times, shame on me.... ;) Really -- ever since the ATI acquisition I thought AMD could make it happen. And it's true that they have come a long way since then. "New Linux Driver and Runtime Stack optimized for HPC & Ultra-scale class computing" sounds great, let's hope this time they can do it.
- boxerab 10y ago:) I've only been deeply involved with compute for the past two years, so I am still an optimist. I think AMD pretty much has no choice but to get it right this time: they need server and HPC dough to stay afloat.
- largote 10y agoI wonder what kinds of cores will be available and whether that will be visible. Optimizing your code for a particular GPU architecture can have massive performance differences, much more so than for GPUs.
- boulos 10y ago9300x2s, K80s and P100s. From the post: > Google Cloud will offer AMD FirePro S9300 x2 that supports powerful, GPU-based remote workstations. We'll also offer NVIDIA® Tesla® P100 and K80 GPUs for deep learning, AI and HPC applications that require powerful computation and analysis. GPUs are offered in passthrough mode to provide bare metal performance. Up to 8 GPU dies can be attached per VM instance including custom machine types. Disclosure: I work on Google Cloud and pitched in on this.
- alimbada 10y agoThey have pictures in the post if you don't want to read the rest of the article...
- eudoxus 10y agoThis amazing!!! First cloud provider to have P100! Amazing opportunities ahead with compute power like that.
- science404 10y agoThey aren't the first.. https://www.nimbix.net/blog/2016/10/04/ibm-nvidia-powerful-gpu-cloud/ https://www.nimbix.net/blog/2016/10/04/ibm-nvidia-powerful-g...
- eudoxus 10y agoDidn't know the added those recently. I knew that had Titan X's and K80s. This is awesome too. Nice to have a full cloud provider with a lengthy feature set AND P100s though.
- eDameXxX 10y agoWhat about this[1]? "GPU-Accelerated Microsoft Cognitive Toolkit Now Available in the Cloud on Microsoft Azure and On-Premises with NVIDIA DGX-1" DGX-1 is powered by 8 x P100 [1]http://nvidianews.nvidia.com/news/nvidia-and-microsoft-accelerate-ai-together http://nvidianews.nvidia.com/news/nvidia-and-microsoft-accel...
- thesandlord 10y agoAFAIK the DGX-1 is for running on premise, not on the cloud
- redbeard0x0a 10y agoFirst with the P100, lets see if they can be the first with full IPv6 support at the instance and/or container level! (I'm not holding my breath)
- dkobran 10y agoKudos to Google for making moves here. Having spent the last year+ tackling GPUs in the datacenter, super curious how custom sizing works. It's a huge technical feat to get eight GPUs running (let alone, in a virtual environment), but the real challenge is making sure the blocks/puzzle pieces all fit together so there's no idle hardware sitting around There's a reason why Amazon's G/P instances require that you double the RAM/CPU if you double the GPU. Another example would be Digital Ocean's linear scale-up of instance types. In any case, we'll have to see what pricing comes out to. Shameless plug, if you want raw access to a GPU in the cloud today, shoot me an email at daniel at paperspace.com We have people doing everything from image analysis to genomics to a whole lot of ML/AI.
- rosstex 10y ago>Your password is too weak This better be worth it! :P
- habosa 10y agoPaperspace looks awesome, but I found the pricing on the homepage misleading. It says in big font "Starting at $5/month". Then I clicked "Pricing" and the cheapest I see is "$15/month". What's the $5 plan?
- Macuyiko 10y agoMaybe the $5 means hourly usage (which has a $5 fixed monthly storage cost) and then not actually using it? Strange indeed, but looks very interesting!
- Veen 10y agoPaperspace looks pretty awesome. Given the exorbitant expense of the new MacBook Pros, I'm thinking of getting a iPad Pro instead, except that you still can't really do dev work conveniently on an iPad. This looks like it might be a cool solution to that problem. Use iOS most of the time (I'm a writer) and the Paperspace VM on the iPad for coding (I'm also a wannabe developer). Only fly in the ointment is that it's not available in Europe, and I'm worried about latency over a 4g connection.
- kesor 10y agoAm I the only one annoyed that their "announcement" talks about something that will happen in the future? What kind of asshole move is this? Why not just say "here, you can use it now, good luck".
- puzzle 10y agoPresumably you can contact them and ask for early access.
- boulos 10y agoYes, that's the goal of the survey: > Tell us about your GPU computing requirements and sign up to be notified about GPU-related announcements using this survey. Additional information is available on our webpage. Survey: https://goo.gl/mgEI9X https://goo.gl/mgEI9X Landing Page: https://cloud.google.com/gpu/ https://cloud.google.com/gpu/ (which also links to the survey and indicates as you rightly presumed, that if you fill it in, we'll add you to our waitlist) Disclosure: I work on Google Cloud.
- eDameXxX 10y agoSimiliar: http://nvidianews.nvidia.com/news/nvidia-and-microsoft-accelerate-ai-together http://nvidianews.nvidia.com/news/nvidia-and-microsoft-accel...
- slizard 10y agoKudos for Google and happy to see that at least in principle AMD is still an option. I wonder what kind of device driver does GCE use with AMD, the new ROCm? What about Power8 + NVLink harware? Does anybody know if the current NVIDIA GPUs, in particular the P100s are all on x86?
- boulos 10y agoI'll just answer your driver question: you bring whatever you want. Our VMs just boot off of bytes you have stored in your image (which is in turn just stored on GCS). If those bytes happen to be Debian 8 with some AMD driver that supports the 9300, awesome ;). We will be working with both vendors to make sure we highlight which drivers work most reliably on our stack though (virtualization + GPUs is way too rare!). We want to work closely with both vendors to do the qualification, so you at least know what is known good. Disclosure: I work on Google Cloud.
- puzzle 10y agoWhen is Kubernetes support going to be added? Right now it has it only in a very basic form for NVIDIA GPUs.
- eudoxus 10y agoThere is a PR[1] to extend the current alpha GPU implentation. Hopefully it will get merged soon. [1] - https://github.com/kubernetes/kubernetes/pull/28216 https://github.com/kubernetes/kubernetes/pull/28216
- kesor 10y agoThis happened some months ago ... how does it compare? Anyone in the know can pitch in on a short comparison? https://aws.amazon.com/about-aws/whats-new/2016/09/introducing-amazon-ec2-p2-instances-the-largest-gpu-powered-virtual-machine-in-the-cloud/ https://aws.amazon.com/about-aws/whats-new/2016/09/introduci...
- wmf 10y agoAmazon: up to 16 K80s and ~700 GB of RAM Google: up to 8 K80s and ~200 GB of RAM but much more flexible
- alecco 10y agoIs it possible to have a non-shared machine? Is it virtualized anyway?
- boulos 10y agoEven if we had a "You get this box all for you", we'd still run our hypervisor on it. We wouldn't give someone a raw, unfettered machine in our datacenter (and I hope you wouldn't either!). What is your concern with virtualization in this context? Disclosure: I work on Google Cloud (and for a long time specifically Compute Engine).
- pktgen 10y ago> We wouldn't give someone a raw, unfettered machine in our datacenter (and I hope you wouldn't either!). I think you're alluding to security issues with renting bare metal hardware to potentially untrusted customers, right? I've often wondered about the security implications of that - think backdooring BIOS/UEFI or other component firmware. But there are many large providers that rent dedicated hardware without a hypervisor. Are they just ignoring these concerns, hoping that this kind of tampering would be rare anyway? > What is your concern with virtualization in this context? I'm not the parent poster, but performance was probably the concern. There's still some (marginal, these days) overhead with virtualization in general, and VT-d specifically.
- boulos 10y agoFrom our perspective: yes, we don't feel comfortable letting untrusted users run side by side without a sandbox. In this context though of GPU compute, you are just talking sporadically to a PCIe attached device directly. There isn't any virtualization or sharing of the device. And yes, I assumed that the complaint would be performance, but especially with just a pass through GPU, it's going to be honestly funny to find the right "oh wow this is 20% slower because virtualization specifically".
- pktgen 10y ago
- fulafel 10y agoWhat's the assurance like regarding security against other concurrent users on the same hardware? Historically multitenancy with GPUs has been quite iffy and not much security research around, even if there theoretically are IOMMU's.
- boulos 10y agoWe care a lot about security, that's part of the motivation for passthrough instead of any of the "split up a GPU". So as I quoted above from our post: > GPUs are offered in passthrough mode to provide bare metal performance. Up to 8 GPU dies can be attached per VM instance including custom machine types. So when your VM boots up, the GPUs you attach will not be shared with others. I agree that the state of GPU virtualization is not yet where it needs to be to make me personally comfortable enough. Disclosure: I work on Google Cloud (and worked on this a little).
- shaklee3 10y agoDoes anyone know what the cost will be for these? AWS is quite high for the K80.
- otto_ortega 10y agoAwesome news! The Tesla P100 is a monster, this will push ML development to new heights.
- dylanz 10y agoThere are a lot of excited posts here about this announcement! For someone that doesn't use GPU's in everyday life, can someone explain why this is great and maybe touch on the current landscape around GPU usage and the cost landscape?
- mattlondon 10y agoGPUs for certain workloads are much more powerful than CPUs, meaning you can get stuff done much quicker. Machine Learning is one such field, batch-rendering for CG studios etc is another. You could throw 100 CPUs at something that perhaps 1 GPU might do in the same time (made up numbers but it gives you an idea of why people want it)
- slizard 10y agoThe excitement comes from the fact that some workloads benefit greatly from GPUs -- mostly those where the GPU arch offers some niche-advantage that can give GPUs >=10x perf[/W]. A good example is deep learning where Intel is struggling to compete [1], and even in linear algebra there is a 4-5x power efficiency difference [2, slide 6]. At the same time, in many other complex applications the performance/W/$ advantage has not been all that clear, especially since GPUs have been quite behind in process technology. A good example is a simulation code I work on where a 2-4x speedup from GPUs, which (due to inherent overhead of using accelerators) gradually vanishes in strong scaling, means that when price is considered, only cheap consumer cards can imrpove the performance/buck metric significantly (that's time-to-solution/buck in our case), professional cards don't offer significant significant improvements other than performance density [3, figures 6,7]. There are plenty more examples that NVIDIA collects [4], but always take the claims with a grain of salt, especially the ones with "incredible" >10x speedup claims :) Last, I'd note that with the recent 14-16nm jump, the gap between the traditional CPUs and the simpler accelerator architectures has increased (I've just seen MAGMA BLAS on Tesla P100 results which show >10x GFlops/W, more than double that of the previous arch) and I expect it to keep increasing partly due to the manufacturing/process technology gap shrinking between Intel and the rest and partly due to architecture and programmability improvements of accelerators. [1] https://www.nvidia.com/object/gpu-accelerated-applications-tensorflow-benchmarks.html https://www.nvidia.com/object/gpu-accelerated-applications-t... [2] http://on-demand.gputechconf.com/gtc/2015/presentation/S5476-Stanimire-Tomov.pdf http://on-demand.gputechconf.com/gtc/2015/presentation/S5476... [3] https://www.academia.edu/13753737/Best_bang_for_your_buck_GPU_nodes_for_GROMACS_biomolecular_simulations https://www.academia.edu/13753737/Best_bang_for_your_buck_GP... [4] https://www.nvidia.com/object/gpu-applications.html https://www.nvidia.com/object/gpu-applications.html
- nojvek 10y agoNvidia got a massive bump in share price. I was quite sad because I sold all my shares after election downfall. I think this announcement might have caused the huge peak. Could have made 10% in one day.
- kayoone 10y agoThey announced very good results for the quarter, i think that was the reason for this spike. Their datacenter/compute revenue grew by >150%
- Veratyr 10y agoThe large recent bump in share price happened while the market was closed between last Thursday and Friday, when they announced earnings: http://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-third-quarter-fiscal-2017 http://nvidianews.nvidia.com/news/nvidia-announces-financial... The Google announcement didn't move their stock much at all.
- jaspervdmeer 10y agoMine ALL the bitcoins