4 ms·

I think it's largely about capacity. Even if you have the money they don't want you to randomly spin up up a few hundred GPU instances for 6 hours because it wi

I think it's largely about capacity. Even if you have the money they don't want you to randomly spin up up a few hundred GPU instances for 6 hours because it will end up causing capacity issues for all of the other people trying to also allocate those instances. GPU's are in crazy high demand and I doubt even GCP, Azure, or AWS can get their hands on enough of them. Contrary to what cloud providers may have you believe you can't actually spin up hundreds of instances unless you already have thousands of instances or you've talked to them and have already told them you're going to spend millions of dollars with them and got the quota pre-approved.

willcipriano 4y ago
> Contrary to what cloud providers may have you believe you can't actually spin up hundreds of instances I've had three software engineer jobs, learned this lesson on the first one and watched others get tripped up by it in the other two. Nobody ever believes me when they say "We will just spin up a instance for every file in the directory" or similar and I respond with "No you won't, they aren't going to give you a thousand instances". This is a lesson that can only be learned first hand.
- fragmede 4y ago
  For GPUs, yeah. If you're talking compute though, you just keep requesting your limits be raised until you're happy. You'll probably hit your ability to pay before you hit actual AWS limits. Talk to your AWS TAM if you have questions on what your spend has to be before they'll give you that kind of access. If you don't have a TAM then you might be too small.
  - deleted 4y ago
    [deleted]
  - nunez 4y ago
    s/might/are/
- Aeolun 4y ago
  > "No you won't, they aren't going to give you a thousand instances" AWS regularly gives me a few hundred instances at a moments notice. Sure, they’re given back just as quickly after the jobs are done, but it’s certainly not impossible.
  - willcipriano 4y ago
    Someone probably called ahead and made a arrangement. In my experience you get tatpitted after a while, the API requests take longer and longer until eventually they time out.
jiggawatts 4y ago
That's why spot pricing exists. It's basically an auction market. You set your max price to bid on instances, and will get them as long nobody else bids higher.
- superchink 4y ago
  This used to be true. As of 2017 or 2018, it is no longer a bidding system, but a published price that AWS offers based on capacity. More details here: https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pricing/ https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pri...
  - jiggawatts 4y ago
    It's still a bidding system in some sense in Azure. You set your max price, up to the on-demand price, and then you'll get your VM... or not. If you do get the VM, you still don't pay the full price in most circumstances unless demand is very high.

willcipriano 4y ago

> Contrary to what cloud providers may have you believe you can't actually spin up hundreds of instances I've had three software engineer jobs, learned this lesson on the first one and watched others get tripped up by it in the other two. Nobody ever believes me when they say "We will just spin up a instance for every file in the directory" or similar and I respond with "No you won't, they aren't going to give you a thousand instances". This is a lesson that can only be learned first hand.

fragmede 4y ago
For GPUs, yeah. If you're talking compute though, you just keep requesting your limits be raised until you're happy. You'll probably hit your ability to pay before you hit actual AWS limits. Talk to your AWS TAM if you have questions on what your spend has to be before they'll give you that kind of access. If you don't have a TAM then you might be too small.
- deleted 4y ago
  [deleted]
- nunez 4y ago
  s/might/are/
Aeolun 4y ago
> "No you won't, they aren't going to give you a thousand instances" AWS regularly gives me a few hundred instances at a moments notice. Sure, they’re given back just as quickly after the jobs are done, but it’s certainly not impossible.
- willcipriano 4y ago
  Someone probably called ahead and made a arrangement. In my experience you get tatpitted after a while, the API requests take longer and longer until eventually they time out.

jiggawatts 4y ago

That's why spot pricing exists. It's basically an auction market. You set your max price to bid on instances, and will get them as long nobody else bids higher.

superchink 4y ago
This used to be true. As of 2017 or 2018, it is no longer a bidding system, but a published price that AWS offers based on capacity. More details here: https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pricing/ https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pri...
- jiggawatts 4y ago
  It's still a bidding system in some sense in Azure. You set your max price, up to the on-demand price, and then you'll get your VM... or not. If you do get the VM, you still don't pay the full price in most circumstances unless demand is very high.