3 ms·
ASK HN: Is there a market for lower spec GPU Cloud Compute?
A while a go I needed to use a GPU enabled computer to run some deep-learning models. I used AWS (naturally) and all was good except the price and performance was not great. I then had to use more GPU compute and didn't want to use AWS or GCP. I also didn't need the latest and greatest GPU specs. I found a solution that is affordable and with good performance. I am now thinking there might be other people with a need for GPU compute but not at AWS prices. People who dont have the skills to build the hw/sw set up but do need machine learning compute capabilities. Over to you HN? Does this resonate with anyone?
- version_five 5y agoCan you elaborate a bit more on what you got that wasnt available on AWS? For me (and I think this is common) there is a clear use case of experimentation on a low-end GPU to get things right (making sure code runs) and then moving to a modern GPU for real training. But Amazon had K80s and a few other low end GPUs that serve this purpose. What is different about what you propose? The only thing I can think of is I might like a GPU with a lot of memory but low throughput, so I could check batch size related stuff more easily.
- dougSF70 5y agoAWS EC2 instances tend to scale key metrics in line with one another 1 vCPU + 1 Ram Unit, 2 vCPU + 2 Ram Unit, 4 vCPU + 4 Ram Unit, etc. So it is impossible to source an instance with a High Ram / vCPU ratio. I found this annoying as I wanted lots of memory to hold a large data object but wasn't so worried about vCPU performance. Also when I wanted to upgrade on EC2 to an instance with a more powerful GPU (more cuda cores, more onboard RAM) again I encountered a similar problem. I had to use an instance with 8 GPUs - an instance type that no longer has capacity on us-east-1. This larger instance type was almost too expensive for my requirements. It was also pretty slow, probably because it was a virtualized machine (?? I am out of my depth here but it would seem this could be a cause of weak performance). I have discovered that unless you are boiling-the-ocean either mapping genomes, folding protein structures, or training CV models, you probably can get by with older technology provided you can run on an actual piece of hardware and not a virtualized instance (again could be out of my depth here). I have successfully launched two very stable, powerful enough for my purposes, GPU-enabled instances. I thought that if other people had a similar need for low-priced but powerful computers for a short period of time, there might be some community / club that could be built around this need.