26 ms·

How to burn the most money with a single click in Azure

• Someone1234 7y ago
  Azure has, built in, hard price/cost limits but doesn't allow the public to use them. For example if you have MSDN subscription credit you get a hard limit of up to $150/month, but you yourself cannot pick a bespoke limit to use the service more safely. Kind of makes me annoyed. I'm sure enterprises don't care/want unlimited. But solo practitioners, and people new to the platform would love a default e.g. $5K/month limit (or less). Feels like these services just want people to "gotcha" into spending a bunch of money without simple safety nets. PS - No, alerts do not accomplish the same thing, by the time you get the alert you could have spent tens of thousands.
  • wrkronmiller 7y ago
    This is only anecdotal, not personal experience, but I've read online and have had friends "oops" away large sums of money on AWS, and for the most part they seem to have at least gotten a partial discount when they contacted customer support. I strongly suspect opaque pricing and high/nonexistent limits are more about getting large organizations to transition to the cloud seamlessly (i.e. not completely caring/realizing what they're getting into for any particular migration/deployment). Tricking personal users into spending thousands by accident probably doesn't net much money compared to enterprise spend and runs the risk of alienating people who then can go into work and recommend against using a particular platform, having been burned by it on their personal accounts.
    • danmur 7y ago
      Or just not being organised enough to calculate process realistically, that's got to be pretty difficult to do
    • dkdk8283 7y ago
      A current coworker used to work at AWS and shared an anecdote. Makes sense but I’m surprised I haven’t seen it repeated: nobody wants to work on the billing code because it’s a mess and the penalty for a mistake is very high.
    • stingraycharles 7y ago
      > “oops" away large sums of money on AWS, and for the most part they seem to have at least gotten a partial discount when they contacted customer support As a counter-datapoint, we accidentally left a Redshift cluster up idling for two weeks before we started getting alerts, and after numerous attempts have failed to get compensated in any way. The reasoning was that, well, it was what we requested and they had to allocate compute power to it (which we didn’t use). All in all a very frustrating experience and it makes me fairly cynical of all these “I got my money back without problems!” comments. (For what it’s worth, it was about $4k of costs which was a lot for us at the time)
      • thanksforfish 7y ago
        It's also unnecessary. Give users cost controls, then you won't need the current mess of hoping support will write off $$$ of mistakes. With a risk of bankrupting a small shop if support doesn't help it drives risk averse users towards less dynamic offerings. Isn't AWS supposed to focus on the virtuous cycle of saving customers money (or at least reducing AWS supports need to write off customer mistakes)?
        • stingraycharles 7y ago
          I always have the feeling of a bit of “randomness” with these kinds of compensations. It makes sense, as it’s difficult to “codify” these types of things, lest they get abused and you might as well just lower your prices at that point. AWS is a large organization; I believe this type of stuff highly depends upon your “entrance” into the organization, i.e. the account manager. We were probably just unlucky with our Redshift troubles, but it did eventually trigger a move to Google Cloud / Bigquery, as the pay-as-you-go method seemed a bit safer (although it’s still too difficult imho to accurately estimate the costs of queries).
      • mepiethree 7y ago
        similar story here, but with $80k, which almost killed the company. woot.
    • rozularen 7y ago
      I, for one, had one of those "oops" when I misused some cloud services, making a VPS instance accessible from the network. When I got charged an extremely large amount I was contacted by the customer support. They explained to me what happened (I had leaked a SECRET in my repo) and then got refunded the total amount. It was quite an anecdotical experience because I wasn't expecting any of it as it was my mistake.
      • JackFr 7y ago
        I inadvertently committed SES creds to GitHub. The only way I found out by an email from AWS telling me that the creds were suspended 24 hours later for a high reported spam rate. Someone had sent 70,000 emails (which is the default daily limit at my tier). Luckily only cost ~$8.
      • scarface74 7y ago
        How do people keep doing this? It’s stated over and over again in all AWS documentation never to put access codes in code or your configuration. Locally, your keys should be stored as part of your user profile (configured with the AWS cli) and when your code is running in AWS, it should get permissions based on an IAM role attached to your EC2/ECS/Lambda.
        • applecrazy 7y ago
          My guess would be poorly configured .gitignore files. I recently had a project collaborator commit his .env file containing credentials to a MongoDB cluster because of this.
          • scarface74 7y ago
            if you follow the guidelines that are repeatedly stressed, your code should never be reading or handling AWS credentials directly. That wouldn’t be an issue. The access keys would be in your usr directory and all of the SDKs would know how to find them. When running on AWS, the SDKs get the credentials from the instance metadata.
    • wobbly_bush 7y ago
      This has happened to me when I was a student. I didn't understand EC2 billing cycle well enough and ended up using much more than my student credits. The final bill amount was high as a student (but still less than $1000). Contacted AWS support, they waived the charges but cancelled my account and told I couldn't use that email address anymore for AWS.
  • Waterluvian 7y ago
    It's their loss. I have moved off AWS for this very reason. My pet project cannot involve the risk of possibly costing me thousands or more because I made a mistake or it got super popular over night. I'd rather my site just 503.
    • klohto 7y ago
      Billing alerts + Lambda. It's not two clicks, but there is plenty of CF templates.
      • oefrha 7y ago
        Alerts are just alerts. I don’t and can’t monitor alerts 24/7.
        • klohto 7y ago
          That's why you have the Lambda there to scale it down. You don't need to sit there.
          • oefrha 7y ago
            Oh okay, I misunderstood.
        • oceanbreeze83 7y ago
          The alert automatically calls a Lambda function which turns off all your services.
          • amelius 7y ago
            Which puts a burden on the user because this needs to be tested. Also, it's not 100% safe because the user is still accountable.
            • scarface74 7y ago
  • yodsanklai 7y ago
    > people new to the platform would love a default e.g. $5K/month limit (or less). I ran into that issue when I wanted to play with AWS EC2 (few years ago, maybe it has changed since then, or maybe I didn't look hard enough). The free VMs were too slow to be usable. Considering my usage, I was unlikely to run into un-expected spendings, but I didn't want to take any risk. Can anyone recommend a similar service with a simpler customer interface where you can set up a simple safety spending limit?
    • setr 7y ago
      If you're just looking at VMs you'd probably be better off with something like linode or digital ocean, and get flat monthly fees. Though amongst those service-types, I can't really recommend beyond the fact that linode & DO didn't give me any headaches for the one month I used them
      • Someone1234 7y ago
        > If you're just looking at VMs you'd probably be better off with something like linode or digital ocean, and get flat monthly fees. Which means you won't learn AWS/Azure/etc instead, and they lose mind-share. This is actually an argument for why they SHOULD offer hard limits, not an argument against. If their goal is to push startups/newbies/hobbyists to other platform, they're definitely on the right path. If the goal is to make their cloud services safe to learn/start using, then they could do much better.
        • oefrha 7y ago
          Yeah, for my personal projects I always stick to flat fee hosts like Linode and only ever used AWS for some backup storage in S3, and GCP for a geographical region that Linode doesn’t serve well. And whenever I use the big clouds I get paranoid and have to check billing & usage very often since I’m always just one oops/DDoS away from incurring a large bill, as opposed to the flat fee hosts where I leave shit running for months or years at a time without worrying. (FWIW Amazon Lightsail might be a flat fee service, but I heard performance is pretty bad so never tried it.)
  • ghaff 7y ago
    How does that work for stateful services like S3? Should they just delete the data? (Which for some people may in fact be what they'd want.) I do realize you can get closer to a hard limit while possibly exempting some services that would let you get over the limit--I suppose. Though then people would doubtless complain that the hard and fast limit is not, in fact, a hard and fast limit.
    • thaumasiotes 7y ago
      > How does that work for stateful services like S3? Should they just delete the data? Intuitively, if you're capping out your S3 storage, the hard cutoff should look like "don't allow me to store any additional data". If you're capping out retrieval, then "don't serve the data any more".
      • ghaff 7y ago
        Yes for retrieval. Just don't serve it. But if the data is stored, the clock keeps ticking until you delete it. If I have a TB of data stored, and I hit my $1K (or whatever limit) on April 15, the only way that I don't get hit with a >$1K bill for the month is if AWS deletes anything I have stored on the service. (Or at least holds it hostage until I pay up for the overage.)
        • thaumasiotes 7y ago
          You can easily calculate what the bill will be at the end of the month if no new data is stored or deleted between now and then. So if you need a hard cutoff for storage, use that. There's enough room there for workflows where I know I'm going to delete data later that allowing configuration would be valuable. (Maybe I can set a timed expiration at the moment of storage, instead of having to store first and separately delete later? That would keep end-of-month predictions accurate.) But it isn't difficult to set the hard cutoff.
          • unreal37 7y ago
            So then your AWS/Azure service is turned off April 2 because you had some temporary spike in uploads? What you're asking for is not possible and will have unintended consequences. Guaranteed not to meet every customer's expectation of how it works.
  • dvfjsdhgfv 7y ago
    > Feels like these services just want people to "gotcha" into spending a bunch of money without simple safety nets. Because it is just like that. Nine years ago Amazon said (about the same issue raised 14 years ago): "We’ve received similar requests from many of our customers and we definitely understand how important this is to your business; We hear you loud and clear. As it stands right now, we are planning to implement some features that are requested on this thread, though we don’t yet have a timeline to share." [0] In other words, they know people need it, but they prefer not to implement it. [0] https://forums.aws.amazon.com/thread.jspa?threadID=58127 https://forums.aws.amazon.com/thread.jspa?threadID=58127
  • Uptrenda 7y ago
    Yeah sign me up for being annoying at this. Several times I'm just like: "why the hell can't I just pay for this thing up front and know what I'm spending." Then there's also both Google Cloud, AWS, etc not letting you spin up certain machines because of "quota limits" which you have to apply to raise. It's like: WTF? do they want your money or not? Idk why it's designed like this but it's a horrible experience.
    • hackbinary 7y ago
      There is significant regulation for prepay of things in many jurisdictions. Eg gift cards, and it is likely that the cloud businesses do not want to enter that mine field. Yes some of them have gift card programs already, but they probably don't want expanded regulations to contend with large sums of money.
    • forty 7y ago
      It's designed to limit the damage when people put their AWS admin credentials on GitHub or in their Android app and someone uses it to mine Bitcoin :)
  • balakk 7y ago
    There are other ways to setup guard-rails in Azure - policies are one such feature: https://docs.microsoft.com/en-us/azure/governance/policy/tutorials/create-and-manage#implement-a-new-custom-policy https://docs.microsoft.com/en-us/azure/governance/policy/tut... They may not necessarily enforce spending limits - but it's possible to restrict provisioning of costly resources, or even whitelist resources that can be provisioned. Almost every Cloud Foundation project nowadays involves setting up these guard-rails.
  • ridaj 7y ago
    Hmm... I'm a big fan of fixed-budget or prepaid services, but between network, storage, VM costs, etc., what should the provider stop serving if you exceed the spend? Create an outage for your whole service? Throttle egress? Start randomly killing VMs?
    • clmckinley 7y ago
      I think for their hard limit on MSDN funds, MS shuts down everything that creates incremental costs. As far as i know they don't delete anything, even things like storage that have an associated cost. That said, i am pretty sure the the TOS for MSDN funds say they are not be used for production systems.
  • jpalomaki 7y ago
    Having reliable hard limits for production accounts can be technically difficult as you need to do billing in real-time and also make decisions on what services to kill once the limit is reached. Do you just stop VMs, do you automatically delete data from storage. Many things could result in loss of production data. There can be also many reasons for the budget overrun. It's not always a user error. It could be issue with the platform itself such as error in billing system or faulty autoscale logic. Or it could be caused by an external event, such as denial-of-service attack. (Not sure how things work with the MSDN subscription credit, but at least you are not supposed to be running production workloads with those)
    • maest 7y ago
      Just because the decisions are difficult to make doesn't mean there's no need for this feature or ways to implement it.
  • Twirrim 7y ago
    At the risk of bringing up the dreaded name here, these kinds of billing shocks were one of the problems Oracle wanted to solve with Oracle Cloud Infrastructure (OCI). So it has been built from the ground up with variable limits and quotas in mind. Every service has them from the outset so that customers can control their maximum expenditure. When they started out building OCI, the major clouds weren't offering this as a key feature. Enterprise companies do not want infinite billing. They want fixed and reliable billing, more than anything else. With on-prem equipment they know a few years in advance what their expenditure is going to be at any time, and will have a budgeted amount over the top of that that they're on-board with. Bring the idea of autoscaling with limits, and they're very happy indeed, particularly with the idea of automatically scaling down. > Azure has, built in, hard price/cost limits but doesn't allow the public to use them. For example if you have MSDN subscription credit you get a hard limit of up to $150/month, but you yourself cannot pick a bespoke limit to use the service more safely. I would be willing to bet that that is something enterprise customers can get access too, particularly if their annual expenditure is high enough under normal operation. Microsoft knows the enterprise market very well, just like Oracle does, and like Amazon doesn't (historically speaking, at least).
  • markus_zhang 7y ago
    Once you set up a subscription service and are selling that service to a huge number of people/companies, rarely that you or one of your salesmen don't want the clients to spend a large sum of money on the service unintentionally and then discount partially as a goodwill. Have seen it in sales of all kinds of services. It's just that they have different tricks.
  • qiller 7y ago
    There are also individual limits on number of cores per account, etc - something we ran into when we needed to quickly scale on Black Friday, and the support too forever to get back to us even with priority A ticket.
• slrainka 7y ago
  Just a couple of months ago, we were blindsided by a massive AWS bill after turning on encryption for logging (an ask from security team). The encryption relied on using KMS, but because it was a serverless setup and each time a lambda was initialized it would grab the KMS keys. Later we found out that invoking that many KMS calls while doesn't cost that much, however does invoke CloudTrail logs which are quite costly. Sometimes it's hard to model something like that. Following this experience KMS team made some changes to how their service/pricing works given its tight coupling to CloudTrail. We also stopped using KMS and simplified the log encryption approach by writing logs directly into an encrypted S3 bucket.
  • sim_card_map 7y ago
    We are using two $3.5 AWS lightsail instances to handle hundreds of thousands of customers :) No hidden or unexpected costs.
    • slrainka 7y ago
      Can you share a bit more about the architecture and stack? And a few more details such as Roughly how many TPS do you serve from this setup? Does it scale well? Are response times consistent?
      • sim_card_map 7y ago
        It's all written in Go + Postgres. Nothing else. Not sure what TPS is. No problem with scaling. Stack Overflow runs on a single machine. Use a fast compiled language and a simple stack. No need in $6k/mo AWS bills which are the fashion these days.
        • lappet 7y ago
          Oh wow, does lightsail come with postgres? Is it like a PaaS on top of AWS? Also TPS means transactions per second
          • 7y ago
• appstorelottery 7y ago
  A few years ago my startup was killed by a AWS mistake that ran overnight. The irony: my AWS expert at the time had made exactly the same provisioning mistake at his previous job - so I figured he'd never make a $80k mistake again. It turns out - his mistake with my startup was even more impressive. More positively - he did help shell out with me to cover the cost & overnight we were out of money. The mistake shocked me so much, and I've since heard so many stories of similar mistakes. The event hit me so hard I went back in time to PHP and shared hosting. Not kidding.
  • jacobra2 7y ago
    What was the mistake?
    • appstorelottery 7y ago
      Running up a shitload of instances for testing and leaving all of them running overnight. Each of these instances continually rendered 4k video data to storage. This kind of test was supposed to be 1000x smaller, running for at most 10-20 seconds at time. He had written his own provisioning system which - according to his report - failed to properly manage instances "weird" edge case. No kidding.
      • appstorelottery 7y ago
        Every morning I would check AWS billing just out of habit. I'm just thankful I did - otherwise everything would have kept running... The lesson for me was don't trust your internally-hacked-together instance management system. The AWS interface to storage and instances is the base truth. And perhaps more importantly - I'm never getting into another startup which has financial risk like that without being a core expert in that risk/tech. I was focused on the business + client code - and had very little clue about the nitty-gritty of AWS. I should have been more involved with the code on that side, or at least the data-flow architecture.
        • seibelj 7y ago
• bane 7y ago
  These basic gaps in cloud hosting providers' tooling have created a rather large cottage industry of companies that exit only to make the tools to fill in the gaps -- like VPC cluster management, security compliance, whatever. The perverse side of it is, it costs to host the third party tooling as well, so cloud providers get more money from you setting up these tools so that you don't burn away all your money. However, they don't get more money from plugging the holes in their own tooling. So they have no incentive to fix it on their own.
  • hinkley 7y ago
    Perverse is the right word here. Perverse incentives are why things are the way they are. This will only change when a large enough competitor defects.
• brassattax 7y ago
  I wonder if reserving all available services on a hacked account is the next DOS attack.
  • zupa-hu 7y ago
    lol sounds effective xD
  • redis_mlc 7y ago
    Already been done for a Defcon talk. Limits were added.
  • TA0x0 7y ago
    Already happening. Instead of reserving, they're mining bitcoin by spinning up tens of thousands of dollars in infrastructure in the cloud.
• thoraway1010 7y ago
  Not that I'd want production infra to be wiped because I went over a spending limit by $1 - but can't you setup a billing alert that goes to an alarm action to terminate all your EC2 instances as soon as the billing alert triggers? Or a lambda function that iterates through your account and deletes everything? I'm not sure why AWS would build something like this though themselves. To stop spending money EVERYTHING must be deleted (s3 / glacier / etc). If someone in accounting loads the wrong budget amount you lose all your data. Amazon's focus seems more on making sure your data is kept and available.
  • maest 7y ago
    Biling alerts are not real time. - Some guy
    • thoraway1010 7y ago
      Good point - I think they are on a 6 hour average delay from what I've seen. An area for improvement it would seem for sure.
  • nucleardog 7y ago
    Yeah this is really the only realistic way for this to work. Every workload is different and it's basically impossible for Amazon to set up a global cost limiting that would work for everyone. Trying to do so would just introduce a lot of new and non-obvious failure modes for a bunch of people. It's like your landlord made a deal that if you couldn't pay your full rent any month, you could lose access to part of the property and he'd clean it out and re-rent it until next month. Except your landlord has no idea what your actual day-to-day life looks like. Some stuff is obviously no good to take away (you need the bathroom), some stuff is seemingly obviously no good (you need the kitchen... unless you always eat out). But what about that spare bedroom? Probably fine if you're just using it to store your recycling. Not fine if you're using it as a home office. You need a understanding of the actual workload in order to figure out where costs could be reduced, and implementing rules or logic around a single workload limits everyone either to using that same thing, or not using the cost limiting.
• seanwilson 7y ago
  Does AWS still not have a way to stop accidentally huge bills? How is this not intentionally negligent on Amazon's part at this stage?
  • opless 7y ago
    If this scenario isn't a good reason for provisioning an internal 'cloud', either on-premise or co-location, I don't know what is. AWS, Azure, and GC are horrendously expensive for what they are. Especially for long-lived services. They do have a use case but 99% of businesses just do not need it. (queue up the downvotes.)
  • Terretta 7y ago
    The fewer system parts running "if" statements in delivering your service, the better. The use case of delivering the service to a consumer of the service happens all the time, the use case of "oh, I didn't understand how this works and foot-gunned" is relatively rare. AWS can eat that cost less expensively than maintaining the "if" statements inline on every request. The most reliable code is code you don't write at all.
    • hinkley 7y ago
      You can run an awful lot of if statements in a microsecond these days.
      • Terretta 7y ago
        At scale, every microsecond matters. But it's less the CPU time, and more the complexity.
        • hinkley 7y ago
          That depends entirely on if the microsecond is in the embarrassingly parallel part of the workload or in the sequential part.
    • dirtydroog 7y ago
• leoedin 7y ago
  I got burned by AWS billing. I played around with some tutorials and before I knew it I had a $100 credit card bill. Even working out what I was paying for proved really tricky. Turning it all off involved crawling through a bunch of opaque control panels. I'm definitely never considering AWS again for a personal project. It's too dangerous. I'm not a company and I don't have corporate level budgets.
  • farisjarrah 7y ago
    I get around this in GCP by having a script that spins up a brand new project whenever I want to play around with stuff. When I am done playing around with things, I have a 2nd script that deletes the project and thus all the associated resources that are in use in that project.
    • scarface74 7y ago
      Why do you need a script? With CloudFormation (and I assume terraform), you create a file containing all of your resources in a “stack” and you delete the stack when you’re done.
• alfianHac 7y ago
  Magic
• Razengan 7y ago
  Takeaways from this and other people’s experiences in the comments (apart from making sure such mistakes don’t happen in the first place): • Providers should let users set hard spending limits. • Providers should offer a channel for investigating and waiving charges for honest mistakes if reported soon enough. • Perhaps you should sign up with throwaway billing details so you can continue on another account till you sort it out. Morally and legally unsound but probably the better alternative to “killing your startup” I guess.
• mcv 7y ago
  A co-worker created a website for a charity and got some free Azure budget for it that would have been plenty for a year. After half a year, though, it was gone because something innocuous that he'd accidentally activated ate up all his budget. Sorry I don't have more detailed data than that, but it certainly alerted me to the fact that you need to be really careful with these cloud services.
• patrec 7y ago
  Which cloud or physical server hosting providers provide either user configurable hard spending limits or the option to (only) pre-pay? The big 3 cloud providers obviously don't, and neither does DO, I believe (you can pre-pay but not as your only payment option). For personal hobby projects taking your chances on customer support goodwill in case you rack up a bill that would bankrupt you due to a wrong click seems kind of insane. So what are good alternatives?
  • why-el 7y ago
    Can't think of any, but managed hosting tends to do better. For instance if you are doing your testing in Ruby, I'd suggest hosting on Heroku while you do all your preliminary work, where it's clear(er) what the cost will be, then move to AWS later when the bank account is, well, beefier. :)
    • patrec 7y ago
      Bonkers! Surely there must be a market for this? Also, I get that the selling point of heroku is kind of AWS for dummies, so you are probably slightly less likely to mess up because of that, but it's still some auto-scale-this, click together-that kinda of service. As far as hosting models are concerned, getting a dedicated box and running the stuff you want seems far safer, no?
      • why-el 7y ago
        Agreed, it's a matter of degree. You can auto-scale up to a certain amount, at least in Heroku, so in practice you can limit it, say up to 8 servers or something, but nothing beats your own box if you can afford the opportunity cost.
• praveenpenumaka 7y ago
  We had an instance when some consultant dev wrote a lambda function which gets triggered when a new image is saved in S3, takes it resizes it and re-save it in same S3. For non-technical guys, that is recursion. We were surprised to see $5000 bill in 10 days.
  • williamdclt 7y ago
    You got lucky, if it saved two different sizes for the each image...
    • bspammer 7y ago
      I'm actually interested how AWS handles exponential cases like this. They must throttle it somehow, otherwise something dumb like this would be causing availability issues everyday.
      • praveenpenumaka 7y ago
        Unfortunately, there is nothing inbuilt in AWS to mitigate these conditions, except for monitoring for "anamolous" behaviour.
        • bspammer 7y ago
          I mean there must be something, because otherwise taking down AWS would be as simple as the above. After 50 steps, you've created a quadrillion images.
          • praveenpenumaka 7y ago
            They surely have very high upper limits on S3 files. Lambda functions have limit on concurrent functions you can run.
• crazygringo 7y ago
  This is hilarious. But serious question: the original tweet is about a single instance that costs over 3 million dollars. Is that genuinely a single physical instance? Like is it even technically possible to build a traditional single physical server of CPU+RAM+disk that costs that much? Or, being a database server, is it some kind of clever abstraction that actually splits it up physically (e.g. relying on the fact that different database threads might be able to get away without sharing memory)?
  • judge2020 7y ago
    I'm not sure exactly what processor AWS uses, but assuming you match the db.r5.24xlarge's 48 cores, the Xeon Platinum 8160 is the closest match for the processor - ~$5k. The ram is likely ddr3 16gb ecc memory, so 768gb at $50/stick (you might be able to find a better price) is $2400. As for disk, the price of the instance doesn't include disk size since it's an EBS-only server. The actual price comes from SQL server enterprise edition. A mysql db.r5.24xlarge multi-az prorated for 3 years is $219,551, which is ~$600 less per month than on-demand pricing. However, sqlserver-ee is 2,782,588. I believe this comes from per-core pricing for sql server. https://docs.google.com/spreadsheets/d/e/2PACX-1vQZT7wl1yvavctIYchaxHAI_BEct1lKRnl3A_Z8D9xBK-u1eqxy__S9w2kB9BucnLq8iZIYb-YGUZiN/pubhtml?gid=0&single=true https://docs.google.com/spreadsheets/d/e/2PACX-1vQZT7wl1yvav... Note - all prices were North Virginia, the original tweet was regarding servers in Bahrain.
    • crazygringo 7y ago
      Oh wow. That makes much more sense then, that it's mostly SQL Server licensing. I was vaguely aware that some server licensing is per-core, but I never realized it could add up to costs at that level. Thanks!
• ertucetin 7y ago
  This is the reason why billing alerts exist. Just use them. You can't make joke of money.
  • hinkley 7y ago
    I know this is common advice, but I expect history will very justifiably tag this kind of talk as victim-blaming. I think fundamentally small and middle-sized companies are building websites improperly (cargo culting companies that can afford million dollar mistakes), which surely exacerbates the situation. But we used to be able to count on a person providing a service not to overcharge you because it was bad for business. When they didn't, it was news.
• igammarays 7y ago
  Existential question about cloud usage: at what point does the risk + cost + knowledge/consultation + DevOps staff required to efficiently manage a cloud provider outweigh the risk + cost + staff of running your own data centre?
• new_here 7y ago
  It’s worth mentioning here that if you find yourself accidentally running up an AWS bill you can get in touch with their customer support, explain the mistake and ask for amnesty. I once accidentally ran up a $2.8k bill and after explaining the situation they added a credit to my account to cancel out the charge at the end of the month. They obviously review it case by case but it’s definitely worth a shot.
• runawaybottle 7y ago
  At what point would a VPS no longer meet your needs? Let’s say a standard startup, not something that will reach TikTok traffic, or something like Slack.
• zengid 7y ago
  I wonder if they have started courses in 'cloud accounting for engineers' at universities yet? Snark aside, it would be useful for business oriented IT degrees.
• londons_explore 7y ago
  Why do none of these cloud services have spend caps? Why not have a simple "budget" setting and have a setting what to discuss when it's exceeded. Options could be "shut down most recently started resources" or "shut down everything but don't delete any data", and "delete everything".
  • maklu 7y ago
    Azure has this in their Cost Management page, available for each subscription. You can set a budget and get notifications about when you are reaching its limit. Not sure about the power it has with re to shutting things down.
    • topkai22 7y ago
      I believe you can hook into apis to shut things down, but it’s not super straight forward. There are many Azure services that can’t be set to zero dollar billing without data loss, so I’m not sure how Azure could deal with those in a unified manner.
• tly_alex 7y ago
  Just quickly glanced over some comments on this thread, found it's a bit funny that most comments are about AWS, mean while the original artichle is about Azure.
• s1k3s 7y ago
  I see a lot of people here saying how they "get burnt" by unexpected increase in their bills, because some guy who was supposed to manage the cloud made a mistake. That's not the real burn, the real burn is when you pay $8K a month on cloud services for apps that could run on a laptop in a basement. And you pay that monthly, for years. And before devops guys slash me with "scaling", "fault tolerance", "redundancy" and what not, remember that your little website isn't a multi million user app and all the issues that you think are fixed by simply moving to the cloud can also be fixed by running your app on a laptop (for free). I used to handle infra for a client a few years ago (the stack was built by another employee before I got there) and it always amazed me why these people pay $10-20K every month to AWS for their app with 500 users a day. Needless to say that AWS was also causing a lot of headaches with their instance management, network maintenance and service unavailability. I know cloud is cool, but business should probably think twice before going for it.
  • RangerScience 7y ago
    You can use the cloud without your tech being deployed like a cloud. Heroku is a good example of this; very easy (and very cheap) to just get something up, and expandable from there. You don't start out needing all the cloud fancy, and you can use "the cloud" just by using the parts you DO need.
  • gitgud 7y ago
    Very true, some companies boast about 10,000 registered users. When in reality there's only 100 daily active users... I think the problem is that companies want to appear bigger than they are. Saying they use AWS Cloud Stuff is a selling point to investors who have a distorted understanding of how scaling works... The cloud service companies are marketed to entice people into buying services they don't need... yet/ever
• 29athrowaway 7y ago
  I burnt $5,000 by putting some servers in the wrong region. Network traffic within the same region was free, but across regions was not.
• anvarik 7y ago
  this part made me laught: > Our next candidate is Azure Databricks. I have no idea what it is, and I don’t even care! All I know is that it’s pretty expensive, and that’s exactly what I need
• ineedasername 7y ago
  Well, not sure if this counts or if it's a cheat, but you could do the following: Find the single most expensive click, and then a consulting company to implement it, and click their button to sign the contract; Still just a single click and Boom: 20% to 200% increase over Azure cost alone.
• ablekh 7y ago
  Could people familiar with the "Cost Management and Billing" functionality on both Azure and AWS, share here relevant coherent thoughts (i.e., mini-review) on feature comparison, pros/cons, etc. for these two platforms? I'm especially interested in this from the perspective of a multi-tenant SaaS provider/vendor.
• moondev 7y ago
  If your startup infrastructure scales "to well" then the new attack is not DoS but a CoS - cost of service attack. Trigger a cloud bill so high it bankrupts them!
• reilly3000 7y ago
  About a year ago I spent $80 making a single query using BigQuery. There was a large public dataset and I did a query that spanned several months of data. My query was something like SELECT fields FROM table.* WHERE date = ... the problem was that WHERE still scans data from ALL date partitioned tables then returns a filtered result. What I should have done is FROM table.date - anyhow I am glad I checked the cost before I hit it with more queries. At $5/TB scanned is remarkably cheap for small data but not for large data. In my case it was 16TB. Still, it was pretty amazing that the query returned data within ~2 seconds. So be careful kids. Public datasets don’t cost anything to store, but your cloud account is going to get pummeled for your exploratory data analysis and if you haven’t set any billing controls or alerts you’re in for a nasty surprise.
• deleted 7y ago
  [deleted]
• croh 7y ago
  Oh boy this why I love digital ocean ! There is never any surprize. Simple and intutive UI. Free alerts, free firewall. you can see all resources on single dashboard. not like aws where you forget to terminate instance from different region and screwed up in one night.
• batoure 7y ago
  I turned on a POC of a platform on AWS last month that stood up 30,000$ worth of AWS spend for it self... now where near this but the ease with which it came online was terrifying
• samstave 7y ago
  The easiest way I can think of is to got commit your AWS keys to a public repo. (This actually happened to me a while back - a new employee created a 201st repo, which our private repo paid limit was 200, so the new repo was automatically made public and he had keys in his code. We had thousands of bitcoin mining instances launched using the keys and cost $75,000 super fast. We caught it really quickly and AWS dropped the cost.