5 ms·
> Contrary to what cloud providers may have you believe you can't actually spin up hundreds of instances I've had three software engineer jobs, learned this le
by willcipriano 4y ago
> Contrary to what cloud providers may have you believe you can't actually spin up hundreds of instances
I've had three software engineer jobs, learned this lesson on the first one and watched others get tripped up by it in the other two. Nobody ever believes me when they say "We will just spin up a instance for every file in the directory" or similar and I respond with "No you won't, they aren't going to give you a thousand instances". This is a lesson that can only be learned first hand.
- fragmede 4y agoFor GPUs, yeah. If you're talking compute though, you just keep requesting your limits be raised until you're happy. You'll probably hit your ability to pay before you hit actual AWS limits. Talk to your AWS TAM if you have questions on what your spend has to be before they'll give you that kind of access. If you don't have a TAM then you might be too small.
- deleted 4y ago[deleted]
- nunez 4y agos/might/are/
- Aeolun 4y ago> "No you won't, they aren't going to give you a thousand instances" AWS regularly gives me a few hundred instances at a moments notice. Sure, they’re given back just as quickly after the jobs are done, but it’s certainly not impossible.
- willcipriano 4y agoSomeone probably called ahead and made a arrangement. In my experience you get tatpitted after a while, the API requests take longer and longer until eventually they time out.