6 ms·

EDIT: if you really need GPU compute power, go buy a GPU from the computer store around the corner - they are cheap, fast and available. I gave up on ec2 when

EDIT: if you really need GPU compute power, go buy a GPU from the computer store around the corner - they are cheap, fast and available. I gave up on ec2 when they started requiring you "request for quota" to start a gpu instance. You have to "request for quota" even if you want to run a single instance. You have to specify which specific instance type you want. You have to specify which region you want it in. Then you sit back and wait for some human to "approve your request". In my case this took 24 hours. In this time I literally could have walked (not driven) down to the local computer store, bought a computer, pushed it back to my house in a shopping cart, spent a few hours configuring it and still be left with 16 hours to have a sleep, eat and do some other things. AWS quota system is so far from "scalable" and "elastic" that it's effectively useless. You can't design any sort of infrastructure around that sort of quota system. I dumped AWS at that point. Mind you, Azure has exactly the same quota system. Seriously, just rent a server from Ionos or Hetzner or get a fast Internet connection and self host. It's faster and better and cheaper than any of the big clouds.

sabalaba 4y ago
Lambda GPU Cloud is extremely cheap as well (1/4 the price of Amazon) and we have recently added lots of capacity.
Cpoll 4y ago
> and still be left with 16 hours to have a sleep, eat and do some other things. Well there's your problem. You don't need to sit back the whole time while you're waiting for approval, it's okay to leave your chair. Jokes aside, I think this probably depends on use-case. You can't easily return the computer. And you can't easily buy (and then return) 100 computers if you have a large one-time computation to run.
toomuchtodo 4y ago
We’ve come full circle. Cloud providers have turned into dedicated server providers.
andrewstuart 4y ago
Also, I felt that Amazon was playing sketchy little games with spot instances. It's not impossible but quite technically challenging to find out how much you are actually paying for a spot instance. It felt really dodgy to me that I might have started a spot instance thinking I was paying the minimum listed rate but somewhere amongst the formulas and terms and conditions AWS actually decided I would be paying maximum rate. They don't actually tell you up front what your spot instance are costing, no doubt it would not be hard for them to do so, but it seems a deliberate strategy to hide this information. It's just not worth the game playing when self hosting or renting a server from Ionos is cheaper and takes away the uncertainty.
- KyeRussell 4y ago
  Comparing with self-hosting or renting a server is just dishonest. At that point you may as well compare with spinning up an actual EC2 instance. Your alternatives miss the main original value proposition of EC2 altogether, let alone spot instances.
  - andrewstuart 4y ago
    >> the main original value proposition of EC2 Which is? I don't really buy the idea that companies can't afford to run their own computers due to hardware and infrastructure costs, and that clouds don't require hiring or specialised technical experts.
    - easton 4y ago
      Pay per second Linux boxes with public IPs. A lot of the VPS/server hosts I used in the past took minutes to boot an instance and billed you by the hour (or maybe by the month, although that was uncommon), not so with EC2. There’s much less value if you never need to scale of course, which is true of 90% of businesses. But doing a dumb experiment on a 10 cent spot instance for an hour where it would’ve taken a week to arrange space on the company VMWare box? That is awesome.
      - 4y ago
ericpauley 4y ago
I think it’s tough to blame Amazon here. p4d server hardware costs around 100k and they were almost certainly having countless hours of use on these by stolen aws accounts and credit cards. This doesn’t excuse the requirement to specify region and instance types in advance. The region part at least can be explained by the fact that Amazon tries to make regions as independent as possible.
my123 4y ago
> I gave up on ec2 when they started requiring you "request for quota" to start a gpu instance. Sadly, the scourge of crypto mining combined with credit card fraud makes it really hard to do otherwise for any sizeable hoster. :/
- Negitivefrags 4y ago
  Maybe they should let you bypass the quota system if you pay in advance via bank transfer then?
  - andrewstuart 4y ago
    You'd think my > 10 year old AWS account with usage history would be enough, but apparently not.
    - ecnahc515 4y ago
      Devil's advocate: 10 year old accounts are probably just as likely as any to get hacked into and used for crypto mining, and honestly I bet a majority of their customers don't even use GPU instances.
      - ericpauley 4y ago
        Yep, Amazon's threat model has less to do with how reputable a given account is, since credential theft is rampant. Even at massive institutional customers on POs I've had to apply for quotas.
      - angus-prune 4y ago
        One could argue that older accounts are even more likely to get hacked, as they are more likely to have older passwords that are weaker that may have been leaked along the way, along with various other accumulated security issues (leaked API keys, out of date 2FA choices etc).
  - derefr 4y ago
    There are a lot of systems and features that are hard (impossible?) to design in a way where you can predict (every dimension of) spend; you can only react to spend events. For example, a theoretical "prepaid AWS" might allow you to put a hold on a vCPU-month of account credits to start a 1vCPU instance. But what about the bandwidth egress fees when someone makes requests to said instance? Those are going to be completely variable, depending on how much traffic the instance receives.
dragonwriter 4y ago
Google Cloud does this, too, for GPUs on their compute instances, but the approval (I assume if there is manual approval, there are conditions which trigger it that I missed) was near instant in my experience.
- carbocation 4y ago
  This is oddly downvoted, but it is accurate. When I use university-linked accounts, I get insta-approved for almost whatever I want. (2,048 spot T4 GPUs? Go for it.) When using my personal CC-backed account, my max quota is often an order of magnitude lower.
ajb 4y ago
I can understand why though. One employers first use of a GPU instance was when we were hacked and someone fired up a few to mine crypto, only a day off which cost us a few $1000 that, fortunately, AWS refunded. It's quite likely that most users don't use them and that it is a good signal that you've been hacked. They have a quota system for sending emails too, and it's not because they need to purchase any hw for sending more emails. It's because those are also a magnet for hackers.
- jldugger 4y ago
  AFAICT, its more that they can't keep GPUs in stock. Even large users of GPU have to request quotas in new AZs.
  - narrator 4y ago
    Lambdalabs used to have spot instances for GPUs and now they are completely sold out. Even vast.ai hardly has any A100 instances. We are starting to see paperclip maximizer economics start with GPUs since they are so damned useful.
    - EVa5I7bHFq9mnYK 4y ago
      Well, paperclips are clearly useful, not so sure about ChatGPTs ... so far looks like their effect is net negative - more cheating and more spam of all kinds.
- andrewstuart 4y ago
  If AWS was really clear about what you were spending then it would be easy to be running an app that tells you the AWS usage and alerts you of anomalous patterns. My bank contacts me if there are any questionable transactions. Surely AWS is capable of this?
  - leoxiong 4y ago
    AWS Cost Explorer gives a pretty good breakdown. There’s also Cost Anomaly Detection https://aws.amazon.com/aws-cost-management/aws-cost-anomaly-detection/ https://aws.amazon.com/aws-cost-management/aws-cost-anomaly-...
dylan604 4y ago
>and still be left with 16 hours to have a sleep, eat and do some other things. I hope somewhere in those 16 hours you bother to return the shopping cart. Or are you one of those that just leaves it where ever because you can't be bothered? I will be bringing this up at the next tenant's meeting.
SilasX 4y ago
Do they still wait until the very last moment before launching to tell you that the account you're logged in under doesn't actually have permission to launch EC2 instances? http://blog.tyrannyofthemouse.com/2016/02/some-of-my-geeky-tech-jokes-with.html http://blog.tyrannyofthemouse.com/2016/02/some-of-my-geeky-t...
hcarlens 4y ago
Having your own GPU is nice, but it's power hungry, heats the room, and only really makes sense if you use it a lot. For a list of cloud GPU providers with rough price comparisons, I created this page: https://cloud-gpus.com https://cloud-gpus.com
- Nextgrid 4y ago
  Having your own GPU makes sense if it works out cheaper than the cloud equivalent (not just the GPU bit but everything else - bandwidth/storage fees, etc). Given the prices of cloud infrastructure, it doesn't take much before walking to your local computer store and buying a GPU (or more!) becomes more cost-effective.
coliveira 4y ago
AWS is following the same path as other web technologies offered by big tech. In the beginning, they want to lure as many people as possible to their new, shiny product, so they offer irresistible prices. The next step, which is the important one, is to secure corporate and government contracts for that technology. Next thing you know, they will see small clients as a nuisance, and do everything to make their lives more difficult.
lucb1e 4y ago
> from the computer store around the corner They closed last year, it's now a 50-minute bus ride and then ~10 minutes walking (one way). Apparently we were one of the last customers to buy most parts for a desktop there. I fear we may not be able to refer to corner stores much longer if they're not of a type most people actually need on a weekly basis, with things moving to online sales > In my case this took 24 hours. Even shipping is faster than that, so yeah point still taken
malux85 4y ago
24 hours? You were lucky! When I requested approval it took a week
KRAKRISMOTT 4y ago
Pray tell where exactly are we supposed to find a cluster of A100s available at a reasonable price? RadioShack?

sabalaba 4y ago

Lambda GPU Cloud is extremely cheap as well (1/4 the price of Amazon) and we have recently added lots of capacity.

Cpoll 4y ago

> and still be left with 16 hours to have a sleep, eat and do some other things. Well there's your problem. You don't need to sit back the whole time while you're waiting for approval, it's okay to leave your chair. Jokes aside, I think this probably depends on use-case. You can't easily return the computer. And you can't easily buy (and then return) 100 computers if you have a large one-time computation to run.

toomuchtodo 4y ago

We’ve come full circle. Cloud providers have turned into dedicated server providers.

andrewstuart 4y ago

Also, I felt that Amazon was playing sketchy little games with spot instances. It's not impossible but quite technically challenging to find out how much you are actually paying for a spot instance. It felt really dodgy to me that I might have started a spot instance thinking I was paying the minimum listed rate but somewhere amongst the formulas and terms and conditions AWS actually decided I would be paying maximum rate. They don't actually tell you up front what your spot instance are costing, no doubt it would not be hard for them to do so, but it seems a deliberate strategy to hide this information. It's just not worth the game playing when self hosting or renting a server from Ionos is cheaper and takes away the uncertainty.

KyeRussell 4y ago
Comparing with self-hosting or renting a server is just dishonest. At that point you may as well compare with spinning up an actual EC2 instance. Your alternatives miss the main original value proposition of EC2 altogether, let alone spot instances.
- andrewstuart 4y ago
  >> the main original value proposition of EC2 Which is? I don't really buy the idea that companies can't afford to run their own computers due to hardware and infrastructure costs, and that clouds don't require hiring or specialised technical experts.
  - easton 4y ago
    Pay per second Linux boxes with public IPs. A lot of the VPS/server hosts I used in the past took minutes to boot an instance and billed you by the hour (or maybe by the month, although that was uncommon), not so with EC2. There’s much less value if you never need to scale of course, which is true of 90% of businesses. But doing a dumb experiment on a 10 cent spot instance for an hour where it would’ve taken a week to arrange space on the company VMWare box? That is awesome.
    - 4y ago

ericpauley 4y ago

I think it’s tough to blame Amazon here. p4d server hardware costs around 100k and they were almost certainly having countless hours of use on these by stolen aws accounts and credit cards. This doesn’t excuse the requirement to specify region and instance types in advance. The region part at least can be explained by the fact that Amazon tries to make regions as independent as possible.

my123 4y ago

> I gave up on ec2 when they started requiring you "request for quota" to start a gpu instance. Sadly, the scourge of crypto mining combined with credit card fraud makes it really hard to do otherwise for any sizeable hoster. :/

Negitivefrags 4y ago
Maybe they should let you bypass the quota system if you pay in advance via bank transfer then?
- andrewstuart 4y ago
  You'd think my > 10 year old AWS account with usage history would be enough, but apparently not.
  - ecnahc515 4y ago
    Devil's advocate: 10 year old accounts are probably just as likely as any to get hacked into and used for crypto mining, and honestly I bet a majority of their customers don't even use GPU instances.
    - ericpauley 4y ago
      Yep, Amazon's threat model has less to do with how reputable a given account is, since credential theft is rampant. Even at massive institutional customers on POs I've had to apply for quotas.
    - angus-prune 4y ago
      One could argue that older accounts are even more likely to get hacked, as they are more likely to have older passwords that are weaker that may have been leaked along the way, along with various other accumulated security issues (leaked API keys, out of date 2FA choices etc).
- derefr 4y ago
  There are a lot of systems and features that are hard (impossible?) to design in a way where you can predict (every dimension of) spend; you can only react to spend events. For example, a theoretical "prepaid AWS" might allow you to put a hold on a vCPU-month of account credits to start a 1vCPU instance. But what about the bandwidth egress fees when someone makes requests to said instance? Those are going to be completely variable, depending on how much traffic the instance receives.

dragonwriter 4y ago

Google Cloud does this, too, for GPUs on their compute instances, but the approval (I assume if there is manual approval, there are conditions which trigger it that I missed) was near instant in my experience.

carbocation 4y ago
This is oddly downvoted, but it is accurate. When I use university-linked accounts, I get insta-approved for almost whatever I want. (2,048 spot T4 GPUs? Go for it.) When using my personal CC-backed account, my max quota is often an order of magnitude lower.

ajb 4y ago

I can understand why though. One employers first use of a GPU instance was when we were hacked and someone fired up a few to mine crypto, only a day off which cost us a few $1000 that, fortunately, AWS refunded. It's quite likely that most users don't use them and that it is a good signal that you've been hacked. They have a quota system for sending emails too, and it's not because they need to purchase any hw for sending more emails. It's because those are also a magnet for hackers.

jldugger 4y ago
AFAICT, its more that they can't keep GPUs in stock. Even large users of GPU have to request quotas in new AZs.
- narrator 4y ago
  Lambdalabs used to have spot instances for GPUs and now they are completely sold out. Even vast.ai hardly has any A100 instances. We are starting to see paperclip maximizer economics start with GPUs since they are so damned useful.
  - EVa5I7bHFq9mnYK 4y ago
    Well, paperclips are clearly useful, not so sure about ChatGPTs ... so far looks like their effect is net negative - more cheating and more spam of all kinds.
andrewstuart 4y ago
If AWS was really clear about what you were spending then it would be easy to be running an app that tells you the AWS usage and alerts you of anomalous patterns. My bank contacts me if there are any questionable transactions. Surely AWS is capable of this?
- leoxiong 4y ago
  AWS Cost Explorer gives a pretty good breakdown. There’s also Cost Anomaly Detection https://aws.amazon.com/aws-cost-management/aws-cost-anomaly-detection/ https://aws.amazon.com/aws-cost-management/aws-cost-anomaly-...

dylan604 4y ago

>and still be left with 16 hours to have a sleep, eat and do some other things. I hope somewhere in those 16 hours you bother to return the shopping cart. Or are you one of those that just leaves it where ever because you can't be bothered? I will be bringing this up at the next tenant's meeting.

SilasX 4y ago

Do they still wait until the very last moment before launching to tell you that the account you're logged in under doesn't actually have permission to launch EC2 instances? http://blog.tyrannyofthemouse.com/2016/02/some-of-my-geeky-tech-jokes-with.html http://blog.tyrannyofthemouse.com/2016/02/some-of-my-geeky-t...

hcarlens 4y ago

Having your own GPU is nice, but it's power hungry, heats the room, and only really makes sense if you use it a lot. For a list of cloud GPU providers with rough price comparisons, I created this page: https://cloud-gpus.com https://cloud-gpus.com

Nextgrid 4y ago
Having your own GPU makes sense if it works out cheaper than the cloud equivalent (not just the GPU bit but everything else - bandwidth/storage fees, etc). Given the prices of cloud infrastructure, it doesn't take much before walking to your local computer store and buying a GPU (or more!) becomes more cost-effective.

coliveira 4y ago

AWS is following the same path as other web technologies offered by big tech. In the beginning, they want to lure as many people as possible to their new, shiny product, so they offer irresistible prices. The next step, which is the important one, is to secure corporate and government contracts for that technology. Next thing you know, they will see small clients as a nuisance, and do everything to make their lives more difficult.

lucb1e 4y ago

> from the computer store around the corner They closed last year, it's now a 50-minute bus ride and then ~10 minutes walking (one way). Apparently we were one of the last customers to buy most parts for a desktop there. I fear we may not be able to refer to corner stores much longer if they're not of a type most people actually need on a weekly basis, with things moving to online sales > In my case this took 24 hours. Even shipping is faster than that, so yeah point still taken

malux85 4y ago

24 hours? You were lucky! When I requested approval it took a week

KRAKRISMOTT 4y ago

Pray tell where exactly are we supposed to find a cluster of A100s available at a reasonable price? RadioShack?