6 ms·
Ask HN: Anybody using Amazon Machine Image for AWS Deep Learning?
Hello, I do not own GPU hardware and am trying to understand if public amis on AWS are worth the fuss at professional level or at least good enough for the likes of course.fast.ai moocs. Newest here https://aws.amazon.com/marketplace/pp/B06VSPXKDX for Ubuntu. Any experience? Thanks!
- lekker 10y agoGPU hardware on Amazon is quite pricey. I would be careful since it can get out of hand rather quickly.
- benmanns 10y agoI don't have experience with the Amazon AMIs for deep learning, but I'll give the tip that you can use spot instances with limited lifespans (x instance at $y price for at most z hours) to reduce cost and the risk you forget to shut something down.
- Fripplebubby 10y agoDoing spot requests is a great way to keep the costs down if you just want to rent a GPU for a while. But bear in mind that GPU instance types tend to spike in price all-at-once when somebody brings a huge compute job online, which will exceed your spot request's max bid with no warning and immediately terminate your instance. So make sure whatever you're doing can be interrupted!
- bhouston 10y agoGPUs in the cloud right now are so pricy, in part I think because NVIDIA is making people put in K40s and K80 - the Quadro and Telsa class cards that are super expensive. We started to put on premise machines that have GPUs in them to reduce our costs. We can get a desktop class Core i7 machine with a GeForce 1060 6GB for around $800 all in, where as on Amazon you have to have a G2 2.xlarge on demand it is $400/month, or $3478 one year term (that is for 1/4 of a G2 8.xlarge achine, thus you get one of its 4 Quadro K5000 4GB cards to yourself.) I could put 2 1060s in that machine for another $200 or so. This just beats AWS pricing in <4 months of use. My expectation is that these 1060s will be good for two years, maybe even three. Because our GPU tasks are low bandwidth and non-critical we can run them on our existing office internet. Our rent also includes power. So there are not too many additional costs over the bare machine costs.
- mabbo 10y agoI think you're definitely winning on price- provided you plan to run things continuously for that time period. The AWS GPU instances are probably not aimed at you though. It's far more likely to be aimed at the Master's student in ML who has this new awesome model but it takes goddamn forever to train on his laptop and he just needs 3 days of time with a big-ass GPU instance. He spins up the instances, runs his model, spins it down and is out what, $100 maybe? That fits into his limited budget. Additional benefit: if I buy a kickass GPU, I'm going to reduce my overall productivity in research because now I can finally play some of these hot new games on full graphics!
- tmsldd 10y agoHopefully everything works fine for the student at the first run ;)
- bhouston 10y ago> It's far more likely to be aimed at the Master's student in ML I am pretty sure AWS isn't targeting its services at Master's students.
- chrisfosterelli 10y ago> It's far more likely to be aimed at the Master's student in ML As a counter example, I'm a master's student in ML and my lab wants to push more for an onsite machine since it's easier to expense one machine than a bunch of monthly AWS bills for each student in the lab :) Additionally, most Master's students (at least in Canada) already have access to one or more very large compute clusters for jobs like this. I have used AWS a few times when I was short on time though!
- Baeocystin 10y agoNo. CUDA-capable hardware is dirt cheap, and once you have your own, you can run experiments 24/7.
- chaosmail 10y agoI used the ami with a p2.large for a deep learning project at university. In my opinion it was quite pricey and reading from disk storage was disappointingly slow (we had a 60gb dataset). What i found really great was using the same machine first as a t2.micro instance for setup and testing s3 connection, a m4.large for some preprocessing and finally the p2 for deep learning, all on the same AMI and SSD.
- uberneo 10y agoIf you are just getting started then https://www.floydhub.com/ https://www.floydhub.com/ is a good place as they provide initial 100 mins on GPU for free and then later on approx 50% cheaper than AWS
- neil1023 10y agoCorrection: 100 hours
- gm-conspiracy 10y agoSo, what is a recommended, buildable GPU rig (bang for buck)? i7 w/ dual 1060 GPUs?
- lloyd-christmas 10y agomy year and a half old rig is 2x 980ti and a 5930k, which is also my generic home computer. I only use AWS when it's a long running job and the weather is hot. It can really heat up your home on long running jobs, so keep that in mind.
- StavrosK 10y agoI've been wondering: Is information processing "free" in terms of energy? I.e. if I put 100 watts in my GPU to make it do computations, will it heat my room for 100 watts' worth of heat? I'm guessing yes, because where else is the energy going to go, but then we should probably be using heaters that are made of old/cheap GPUs, rather than just a resistor.
- ivan_ah 10y agoAs a Canadian, I always feel good running compute-intensive jobs in the winter. It's heating as usual, with a side-benefit of computation.
- Neliquat 10y agoI have a system to duct in, out or recurculate inside air as the seasons change. I can heat my home with 'waste' energy at idle, or switch to cool outside air for heavy jobs or late night gaming. In summer I exhaust outside, pulling cooler air from underground. It is slowly spawning its own system, only 1.5 years in.
- graphene 10y agoYes, according to the first law of Thermodynamics, energy is conserved; regardless of whether there is computation being done, 100W of electric power will result in 100W of heat being dissipated. About heating with computers, There is a Dutch company doing this, https://cloud.nerdalize.com/ https://cloud.nerdalize.com/. It's interesting to think about how their economics work, because the heaters will be switched off for large parts of the year, and for much of the day. They must be banking that computation won't get much cheaper over time (in terms of FLOPS/$) because otherwise it'd be hard to recoup their initial investment. In a sense, this almost looks like a bet against Moore's law!
- Fripplebubby 10y agoI've been using this AMI for CNTK since the AMI was made available recently. It's been working well for me - they seem to update the AMI for big tagged releases of the libraries, and all the CUDA drivers are good to go out of the box. Personally I found it easier to use than Docker images, and both are easier than manual install. With a spot request you can get a p2.xlarge for around $0.22/hr, which for me is sufficient for moocs and side projects, but for beefier instances the prices get high.
- jph00 10y agoTo answer your question about the http://course.fast.ai http://course.fast.ai MOOC: the course provides an AMI that has everything you need pre-installed. Just follow the directions in the setup video, which includes a script that sets up the instance for you. To save around 80% (but slightly less convenient) there are two ways to use spot instances provided on http://forums.fast.ai http://forums.fast.ai that are largely automated and work well. In part 2 of the course you'll learn how to set up your own server. As others have mentioned that's much better value if you're using deep learning somewhat regularly. (I'm the founder/teacher of this course.)
- shoshin23 10y agoI can totally vouch for the ease with which you can setup your deep learning AMI. I followed the instructions on the site and the scripts provided by Jeremy abstract a lot of the stuff away. Super cool work!
- farhanhubble 10y agoI'm using p2.xlarge for Udacity Self Driving card nanodegree and Fast.AI assignments. It's super pricey as everyone has mentioned but I prefer using AWS to owning my own rig because I can download large datasets quickly.
- minimaxir 10y agoA helpful on-topic note: Amazon has 1x K80 cloud GPU instances for $0.90/hr (not prorated), but very recently, Google announced GPU instances of the same caliber (https://cloud.google.com/compute/docs/gpus/add-gpus https://cloud.google.com/compute/docs/gpus/add-gpus) for ~$0.75/hr (prorated to the minute), although Google Compute Engine requires configuring the GPU drivers first.
- rocketcity 10y agoI have been using it a good bit for some of the deep learning assignments on Siraj Raval's channel (https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A). I have had a really good experience so far. The trick is to find the AMI id for whichever region you want to use the image in and then just put in a spot request for your instance. I just checked the current going rate for a P2.XL in us-east-1 and it was at 0.2355 per hour.
- ShirsenduK 10y agoSiraj is _the_ bomb! Fellow wizard here :)
- torbjorn 10y agoSiraj has great material.
- ap46 10y agoJust get the GTX 1060 6GB variant & that should be enough RAM to run most models without suffering pagination.
- KaiserPro 10y agoI'm using them at $work. But they are expensive, but so is EC2. Unless you are able to get a decent deal (far better than reserved instances) then for raw CPU/GPU power hosting your own is better. Its the whole package that makes AWS attractive. (RDS, S3, instant spinup) Spot prices is a good place to start if you are doing batches. just make sure you checkpoint little and often. However if you are on your own, with limited funds, Get a second hand workstation (dell or hp z620/600) they tend to have decent PSUs, loads of ram and decent support, and a couple of Geforces with more Vram. (Quadros you are paying for the double float performance and ram. transfer Bandwidth and speed are the same or slightly better on geforce class cards.)
- hackpert 10y agoYeah. I mostly end up using p2.xlarge spot instances, and they have worked fine for my jobs so far. Be sure to checkpoint often and you'll be okay with spot instances.
- chrischen 10y agoA pascal titan x is the best card for deep learning. Even my gtx 1080 is faster, albeit with slightly less ram than the K80 or M60s available in the cloud. A titan X would fix the ram issue though. The most economical would probably be to colocate some titan X boxes. Not only cheaper, but more percormant than anything you can get from IaaS right now.
- kshnell 10y agowww.paperspace.com/ml has much more affordable GPUs than AWS and a new Pascal card. There's a pre-configured Deep Learning template and a couple interesting features like a browser based terminal and desktop (you can still SSH if you prefer).
- dkobran 10y agoPaperspace co-founder here -- happy to discuss our AWS alternative which is focused on affordability, simplicity and performance. Recently, we've had a huge influx of people in moocs offloading their work to our GPUs/cloud. Would love any feedback you have.
- deleted 10y ago[deleted]
- coffeepants 10y agohttps://paperspace.com https://paperspace.com is great. Takes about 2 minutes to go from no account to running a VM with all of the ML frameworks and drivers installed. The K80 on AWS/GCP is three years old and is just not built for Deep Learning. Paperspace has newer, more modern GPUs and I heard they are adding more cards soon (forget where I saw that).
- eshvk 10y ago> is just not built for Deep Learning What does a "built for Deep Learning" GPU do that the K80 doesn't do?
- coffeepants 10y agoSorry, should have elaborated on that a bit. The K80 is Kepler architecture (from 2012) and underperforms Pascal on most Deep Learning benchmarks. Here's a comparison of several NVIDIA cards and you'll see the K80 at the bottom: http://timdettmers.com/2017/04/09/which-gpu-for-deep-learning/ http://timdettmers.com/2017/04/09/which-gpu-for-deep-learnin... This article emphasizes consumer cards which means you're building a box ie you lose all the benefits of offloading your work to the cloud. That said, there are cards that are better suited for DL that can run in a datacenter. It's not mentioned here but Pascal includes new GPU instructions designed to accelerate DL operations -- CuDNN is already taking advantage of these new capabilities. Perhaps, more importantly than the K80 card, is the P2 instance type itself, which includes significantly more RAM than you would need for DL (hence the high cost of the instance). In other words, the instance was evidently tuned for general GPU compute tasks like graphics-intensive applications/video encoding and not DL.
- dhruvp 10y agoHey OP! I've been using it extensively and it's definitely more than good enough for most tasks you'll be doing. If you're going this route, I'd highly recommend following the Keras' authors guide on how to set up the instance and connect to it with a jupyter notebook: https://blog.keras.io/running-jupyter-notebooks-on-gpu-on-aws-a-starter-guide.html https://blog.keras.io/running-jupyter-notebooks-on-gpu-on-aw.... This will boost your productivity a ton in the long run. Best of luck!
- nanistheonlyist 10y agoI have been using vagrant to spin up boxes on various cloud providers. The important thing is to remember to turn them off when you are done! I guess it would be possible to create a script to spin up boxes for each single job but I have not figured that out yet.