3 ms·
$0.9 per K80 GPU per hour, while expensive, opens up so many opportunities - especially when you can get a properly connected machine. Just as an example of th
by Smerity 10y ago
$0.9 per K80 GPU per hour, while expensive, opens up so many opportunities - especially when you can get a properly connected machine.
Just as an example of the change this entails for deep learning, the recent "Exploring the Limits of Language Modeling"[1] paper from Google used 32 K40 GPUs. While the K40 / K80 are not the most recent generation of GPU, they're still powerful beasts, and finding a large number of them set up well is a challenge for most.
In only 2 hours, their model beat previous state of the art results.
Their new best result was achieved after three weeks of compute.
With two assumptions, that a K80 is approximately 2 x K40 and that you could run the model with similar efficiency, that means you can beat previous state of the art for ~$28.8 and could replicate that paper's state of the art for ~$7257.6 - all using a single P2 instance.
While the latter number is extreme, the former isn't. It's expensive but still opens up so many opportunities. Everything from hobbyists competing on Kaggle competitions to that tiny division inside a big company that would never be able to provision GPU access otherwise - and of course the startup inbetween.
* I'm not even going to try to compare the old Amazon GPU instances to the new one as they're not even in the same ballpark. They have far less memory and don't support many of the features required for efficient use of modern deep learning frameworks.
[1]: https://arxiv.org/abs/1602.02410 https://arxiv.org/abs/1602.02410
- deleted 10y ago[deleted]
- lqdc13 10y agoStill seems much better to buy your own gtx 1080 for $700 which you would have spent in a month playing with parameters on these instances. And at the end of the month you still have a modern GPU to play video games on etc. Of course if you have money to burn, these are great.
- madeofpalk 10y agoI question the amount of people that would be buying these for research AND gaming.
- onli 10y agoI saw quite a few asking for PC builds for both. Might not be too common, but is also not something that happens only once a year.
- gcp 10y agoThe demographic overlap between "gamer" and "machine learning researcher" is likely very high.
- angry_octet 10y agoLikely how??
- mappu 10y agoI guess: University students.
- madeofpalk 10y agoUniversity students can afford $700 graphic cards? I would assume they have (or have access to) a computer fitted out in a research lab or something which would be seperate from their "gaming rig".
- detaro 10y agoFor many students, I suspect the "computer fitted out in a research lab" is some dell desktop box someone shoved a gaming GPU in 3 years ago and that they have to share with 5-20 other students. Better not break it. If you are into the topic and want to work on things outside of assigned projects or course work, upgrading a personal machine doesn't seem far-fetched if you can make room in the budget for it. $700 isn't cheap, but doable for many as a one-off purchase if they want it.
- sangnoir 10y ago> Likely how?? Those who are predisposed to buying & installing high-end graphics cards are likely to be either gamers, interested in machine learning or both
- egeozcan 10y agoBy the way, I own a 1080 since two weeks and I can't overstate how powerful this thing is. Even if you are not into gaming, getting a 1080 is still a considerable option if you want to experiment with deep learning.
- deleted 10y ago[deleted]
- visarga 10y agoGet an GTX 1080 for your local box and use it to write and test the model, then run in cloud for training. Both are necessary, in fact, they are complementary. If you want to run random hyperparameter search (train 50 times) then you benefit from many GPUs, but a single 1080 doesn't scale.
- feelix 10y agoYou can set up the volume a CPU only machine (even on a free instance), and then just launch that volume with a machine with these big expensive GPU's on it. I did it a few months ago with a TensorFlow model I was training. Configured it for free then all the training time was fully used after configuration. I assume it'll be the same with these new instances. I have a backlog of stuff I wanted to train but had no GPU to train it on (I travel a lot so couldn't buy a desktop, and was almost biting the bullet to buy one of those monster laptops with dual 1080's, although really I was after at least 12GB of VRAM so this is fantastic news for me)
- eddiepierce 10y agoHave you considered buying/building a desktop that acts as a server and then using ssh to connect to it from your laptop?
- visarga 10y agoThat's what I do.
- imh 10y agoI'm not the person you posed the question to, but what's the secure, convenient way to do that? If I want to ssh into my desktop from within my house? If I want to ssh into it from elsewhere?
- greensoap 10y agoAm I missing the question? Maybe, maybe not. The simple answer is build your new "server" and install some version linux, like Ubuntu. Set up the accounts and make it so you can login using a private key. Turn off remote login with password so that you can only login with the private key. Install Fail2Ban to block automated hacking attempts---almost the second you expose your server to remote traffic you will start seeing attempts. Configure your internet router so that incoming port 22 traffic is routed to the server (or all ports if you don't care about security). Signup for a free dynamic DNS service for when your IP changes. Now you can login to your home server from anywhere using a your hostname which is tied to your home server external ip address. Its secure because there is no password to brute force and Fail2Ban is killing external attempts. You can secure it further using more research, but this is the basic answer.
- p0rkbelly 10y agoIf you're a regulated financial institution, say performing complex risk valuations, you need the ECC ram. It's a consumer device vs an enterprise, professional device.
- Beltiras 10y agoThat's not the use-case for this product. You can fiddle around with a single card till you get it right and then run the real job on EC2 where you most likely order dozens of these machines and run them for weeks/months. It's rarely economical to purchase hardware at that scale, unless if you are AWS renting out computation to others. At that scale you have to think about way more cost factors than raw hardware cost.
- visarga 10y agoWhile you're right that buying your own GPU makes a lot of sense for personal use or small tasks, if you factor in the cost of keeping a ML team waiting for the network to finish training, then it might be cheaper to invest $7000 and have it run in a few hours instead of weeks. Also, consider the efficiency of owning such a 16 GPU beast: if you don't have tasks to run 100% of the time, then you're losing a lot by just keeping it idle. Cloud providers can reuse the same GPU for other clients in the meantime.
- comboy 10y agoYou will need time to buy that gpu, upgrade your PC, realise that it doesn't fit into your existing case and that you need a better power supply. Then some time to find a better place to be than sitting next to that really loud and hot PC. And it all takes optimistically days, while you can start computing on EC2 within in an hour. Plus it doesn't scale, so you cannot just speed up the results by paying some more. You may need to wait a month (and then realise the param was wrong and you need to redo it) Surely there are some cases when you are better off computing on your own GPU, but I think in all of them you assign pretty low value to the time spent.
- thdeltei 10y agoAzure N Series are about 35% cheaper[1], although only up to 4 Tesla k80 GPU per instance. [1] https://buildazure.com/2016/08/09/azure-n-series-vms-and-nvidia-gpus-in-the-cloud/ https://buildazure.com/2016/08/09/azure-n-series-vms-and-nvi...
- visarga 10y agoI've been waiting for access for a long time. I have no idea when they will accept users other than a few hand picked beta testers.
- feelix 10y agoUsing spot pricing you tend to get it far cheaper than that. I was using the previous GPU instances a few months ago which are meant to be $0.65 an hour. But using spot pricing, setting the maximum to about $50 so I wouldn't get shutdown in the middle of training, I seemed to spend an average of about $0.20 an hour. Now I'm using a P2 instance to see how much faster it runs the same neural net. I set the max price to $0.50 in a spot price bid and got it immediately. I'm not sure how to find out exactly how much it is being charged however.
- indeed30 10y agohttps://aws.amazon.com/ec2/spot/pricing/ https://aws.amazon.com/ec2/spot/pricing/ has the current p2 spot prices
- web007 10y agoAnd as of 9am PDT you can get a single GPU Linux instance for less than ten cents an hour. I doubt it will last, but if it did you could get a year's worth of compute for $800 and still have the flexibility and reliability of AWS infrastructure.