10 ms·
My low cost provider of GPUs has run out of capacity. Good alternatives?
- etaioinshrdlu 7y agoHetzner has provided a quality service for a long time but now they have no more GPU servers available. That makes it hard to scale. Any other options out there without going bankrupt on AWS?
- reallydontask 7y agoHave you contacted them to see when they might get them back in stock? It might be a temporary blip in supplies, etc ...
- hanfer 7y agoYeah write them a ticket. They are really supportive, pretty sure they will come up with something for you. (as did they for us, when we were in a similar bind)
- etaioinshrdlu 7y agoAlready did. They just said, currently out of stock, working on making it available as soon as possible. No ETA provided.
- ptah 7y agohow many do you need? maybe just buy your own?
- etaioinshrdlu 7y agoOn the order of 5-25. Interestingly, these Hetzner machines are so cheap monthly that it is much easier to buy via hetzner and they provide good networking / data center ops as well. Break even time on cost is on the order of a year.
- nurettin 7y agoBreak even? Are you generating digital tokens?
- krageon 7y agoWhy would that follow? He said "on cost", which I read as "comparing the costs of buying a video card and renting GPU time".
- nurettin 7y agoBreak even point is used as a term to describe the point your gains are equal to your initial investment, hinting that after this point, you will start making net profit.
- pedrocr 7y agoBreak even applies to the crossover point between any two options. You can break-even between renting or buying a house at some point even though you are even between two options that cost you money.
- ALittleLight 7y agoI interpreted that as meaning you'd have to rent one for a year to break even buying the hardware.
- fs111 7y agoscaleway has GPUs now: https://www.scaleway.com/en/gpu-instances/ https://www.scaleway.com/en/gpu-instances/
- pantalaimon 7y ago500€ instead of 111€ though
- jfindley 7y agoI've used https://www.paperspace.com/ https://www.paperspace.com/ in the past, and they seem reasonable. They're cheaper than AWS but I don't know how they compare to hetzner.
- tex0 7y agoDid they disclose when they will be in stock again? Did you ask?
- lukeqsee 7y agoLinode just launched their instances in Newark on public beta. https://www.linode.com/gpus https://www.linode.com/gpus I'm not affiliated, just love their service!
- tmikaeld 7y ago~ 8.5x higher price than Hetzner.
- raverbashing 7y agoWell, it looks like a supply/demand issue by Hetzner
- lukeqsee 7y agoYes, they're at a much higher price point, but it's also way cheaper than AWS. I certainly understand the draw of Hetzner's price point!
- abrichr 7y agoHow is Herzner able to be so much cheaper than the competition?
- Eldt 7y agoUnmanaged (and typically old) hardware
- deleted 7y ago[deleted]
- tmikaeld 7y agoThey have addons for most support issues, including priority incidents - throughout the years we've been with them, it can take ~2-5 hours before someone even reply to complete server downtime. We mostly host proxmox nodes and use HA, so it's not much of an issue when it's used like this. OVH is very similar, although, Hetzner is often quicker. We only use server-grade (Like Xeon or EPYC), not the desktop ones (No ECC memory). If one need many IP's, OVH always wins in price due to it being a one-time cost, while Hetzner charges monthly.
- mciancia 7y agoI wonder how are they dealing with nvidia EULAs about using geforce series cards in datacenters. Maybe that is the issue?
- Donald 7y agoThey use older cards without the restrictions.
- KaiserPro 7y ago1080s are much harder to buy now, perhaps its taking time to find and qualify 2080s
- mugsie 7y agoprobably just ignore it, they are unenforceable in the EU afaik (which is a good thing - if I buy a piece of hardware, I will do what I want with it).
- dragandj 7y agoI've read somewhere (maybe in HN comments) that these kinds of clauses can not be enforced in EU. Hetzner operates out of Germany.
- etaioinshrdlu 7y agoYes, I wonder how sure HN is about this however!
- jonatron 7y agoColocation with ebay'd servers and graphics cards. Some people don't know there's somewhere in between building a datacentre and cloud servers.
- NietTim 7y agoThat's not a low cost solution though, and hard to just turn off the cost when you're done with your day
- icebraining 7y ago> hard to just turn off the cost when you're done with your day Yeah, but the same is true of OP's Hetzner machines.
- parliament32 7y agoThat's not really how Hetzner's dedis work: you typically pay a setup fee (that's about 1/2 of a month of usage) and you have to cancel a full billing cycle (usually a month) in advance. You're meant to run them for months/years at a time, not just spin them up/down on-demand.
- cryptofits 7y agoHetzner support is pretty bad took them like 6 days to answer my ticket
- jbverschoor 7y agoThey have a phone number
- mythz 7y agoNot the experience I've had with Hetzner (been with them for 5+ years), basically all responses have been within a day, 2 at most. Although their servers have been pretty reliable so I haven't had to contact them much.
- troffed 7y agoThat's not our experience. We're using Hetzner servers for 6 years and we not have any problem with their support.
- monksy 7y agoMy firend's experience has been the same. We've had issues where our wordpress instance was exploited and we got kicked off the network. Also, we've had issues where there was a upnp connection open and we got kicked off the network for that as well. In those cases it took days for them to get back via email, and then they struggled to open up a channel which we could resolve the issue and get it compliant. Also, we've had situations where there was maintenance but no update that it was going on.
- andrewl-hn 7y agoFolks at CleverCloud launched CleverGrid today https://www.clevergrid.io/ https://www.clevergrid.io/ I know a few of their engineers personally, really experienced and trustworthy engineers. They are based in France.
- baybal2 7y agoI think Ebay and TaoBao are now filled to the brim with discarded mining GPUs
- deleted 7y ago[deleted]
- blodovnik 7y agoAws and Google spot instances are easily comparable price wise to hetzner but it really depends on what you are doing, which you haven't said. For example a g3 spot instance can be as low as 19 cents per hour $138/month. Google GPU instances I've run for 14 cents an hour. I think these aren't the lowest prices either.
- minimaxir 7y agoFor one-time GPU processing tasks that only take a couple hours, AWS spot / GCP preemptible is far more cost efficient than Hetzner. However, if you need to run a GPU 24/7 and/or have massive bandwidth requirements, Hetzner is far more cost effective than AWS/GCP. It's a balance of needs.
- streetcat1 7y agoAnother option is to over provisioned spot instances, such that it would feel like a 24/7 instance, but cost like spot. However, you would probably need an automatic platform to achieve that.
- minimaxir 7y agoAn idea I've had is to use GCP preemptibles + script to automatically run a ML training job on instance start + Google Cloud Function + Cloud Scheduler to attempt to start the instance every minute if it gets preempted. The latter two are effectively free, so you'd get the cost benefits of preemptibles as long as the ML training job is resilient to random shutdowns.
- streetcat1 7y agoYes, this is basically the idea. However, there are different solutions for training and inference. For training, I would recommend that you add automatic checkpoint, and even consider model migration. For inference (which I think is the original concern), over provisioning is the key (simply because the fact that it would take a long time to load the model. Also, you also want to diversify your node types, etc.
- chx 7y agohttps://www.serverhunter.com/?search=BF6-4AC-E30 https://www.serverhunter.com/?search=BF6-4AC-E30 says the cheapest provider with a GTX GPU is Ikoula with an 1070 at 101.66 USD and then Hetzner and then Hostkey. https://www.hostkey.com/gpu-servers https://www.hostkey.com/gpu-servers It doesn't find https://dedispec.com/gpu.php https://dedispec.com/gpu.php these but I do :) I also read somewhere they even accept shipped in GPUs. I know they did with disks.
- ServerHunter 7y agoSorry about that! While we had the Dedispec GPU servers listed, our spider system didn't correctly detect the presence of a GPU, so it didn't show up as such. It has now been resolved and they show up in your search results. Thanks for using Server Hunter! :)
- etaioinshrdlu 7y agoAlso missing some GPU servers from https://www.hostkey.com/gpu-servers https://www.hostkey.com/gpu-servers ... although you got some of them.
- etaioinshrdlu 7y agoDedispec also appears to be out of stock! Oh no!
- samscully 7y agoVast AI (https://vast.ai https://vast.ai) are good, 1x GTX 1080 comes to about $170/month. You are charged by the hour unlike Hetzner. There are obviously downsides to renting time from random individuals though, so it's not suitable for a server-like workload. Good for development notebooks or training.
- gwern 7y agoI recently used Vast.ai for ~3 weeks to run an anime BigGAN ( https://www.gwern.net/Faces#biggan https://www.gwern.net/Faces#biggan ) and the stability/uptime was pretty much 100%: it never went down or caused problems. (I had problems, but they were all due to the BigGAN.) As long as a little downtime isn't too big a deal or you can script changing instances (they have a CLI tool), you probably could run as a server.
- julienfr112 7y agoWasn't it forbidden by the driver licence to use 1080 on cloud ? Nvidia tries to bully you to buy ultra expensive Tesla card for that. Maybe the germans don't care ?
- brohee 7y agoNot enforceable in Europe IIRC. So they rightfully don't care.
- etaioinshrdlu 7y agoI believe you but can we get some hard evidence? Everyone in this thread agrees but it would be nice to know for sure.
- pbhjpbhj 7y agoDo USA tyre manufacturers sell tyres that you're not allowed to use on your car if you're Uber-ing? Are there other examples of this sort of restriction that actually make sense. I can think of warnings ('don't use this keychain carabineer for climbing') but not actual use restrictions that are sensible?
- ydau 7y agoLambda Labs: https://lambdalabs.com https://lambdalabs.com has 1080 Ti GPUs for rent. We also sell GPU workstations and servers for AI.
- minimaxir 7y agoI wouldn't call $0.80/GPU/hr "low cost" compared to other options.
- matthew-wegner 7y agoElsewhere in the thread, it sounds like your needs a little more robust. But for anyone reading who might be interested: - Tesla K40 12GB cards are ~$130 on US eBay, and have been for awhile - These cards are passive, though, intending to be used in wind tunnel servers - However, you can put on 40mm screamer fans to cool them in a normal desktop. Something like https://www.thingiverse.com/thing:3032044 https://www.thingiverse.com/thing:3032044 or just wire-tie to the back grill High pressure 40mm fans are LOUD, though. I run a desktop with two K40s in my garage, and I can hear it a tiny bit standing outside in my driveway. (I mostly use that box for neural net art experiments like various style transfer workflows)
- bitL 7y agoK80 are for ~$400, 24GB, 2xGPU, newer architecture, a bit better choice.
- pbhjpbhj 7y agoYou consider $400 'low cost'?
- Johnny555 7y agoThe article link mentions an NVIDEO card, and an NVIDIA GeForce GTX 1080 8GB sells for ~ $750 new. I don't know how this $400 card compares for this use case, but it is relatively cheap.
- bitL 7y agoK80 is about as fast as 1080 in FP32 (8.2 vs 8.8TFlops) but vastly faster in FP64 (2.7 vs 0.28TFlops; FP64 supercomputers are still being built with it). It has also 3x the memory, so fitting BERT_large NLP model might be possible. 1080 is pretty much outdated at this point as not many state-of-art models can fit inside 8GB. Disadvantage of K80 are older CUDA kernels versions so customized kernels for new CUDA versions might not work, but most models don't touch CUDA directly anyway.
- tech_man7 7y agoGolem has GPUs for very cheap on its network so ask here https://www.reddit.com/r/GolemProject/ https://www.reddit.com/r/GolemProject/ or https://twitter.com/golemproject?lang=en https://twitter.com/golemproject?lang=en. Golem website is www.golem.network
- etaioinshrdlu 7y agoI don't like that website, it looks almost like a blockchain ICO site. Low on details, high on hype. I don't see prices listed. Something seems wrong. This company does something pretty similar minus the blockchain and is much more straightforward: https://www.zerosix.ai/ https://www.zerosix.ai/ , same with https://vast.ai/console/create/ https://vast.ai/console/create/
- workingpatrick 7y agoIt's exactly a blockchain ICO site :D https://golem.network/crowdfunding/ https://golem.network/crowdfunding/
- tech_man7 7y agoIt's not one of those ICO scams. Maybe you'd prefer their GitHub page in which the team has written over 100,000 lines of code. https://github.com/golemfactory/golem https://github.com/golemfactory/golem. It's a marketplace so you name your own price and at the moment, it's used for CGI rendering, but next week there are more use cases being added, specifically WASM.
- etaioinshrdlu 7y agoUpdate: they don't seem to actually be a usable option at this point. vast.ai and zerosix.ai at least do work.