3 ms·

AWS Billing Issue

3 points by zeeshah 4y ago

This AWS Billing is quite overwhelming. Are there any genuine ways to control it?

wackycat 4y ago
This question is too broad as is. A good place to start is looking at your cost breakdown which is available in the console. For example, if it shows your highest spending is on EC2 then you should assess your instance types and use case. If your S3 spending is high, assess what you're storing; if any of it is low-access, assess the storage class you're using and consider glacier. These are just two examples of two services and how you might address high spend; AWS has countless services so it is highly dependent on what you're actually using. So figure that out and then google reducing AWS spend for [specific service name]. Good luck!
- zeeshah 4y ago
  Actually I had an unused instance and it took me quite a lot of time to actually figure out that this was the actual cause. I do check the prices but with so many deep learning projects, it's actually really difficult to figure out. Have you tried any third party AWS cost optimization tool ?
oriettaxx 4y ago
welcome to the club My advice: * hire a professional to have a good look at costs: you'll be "amazed" on how much $$ are wasted: a 15 days contract is more then enough: you can even tell him/her: we pay you exactly the amount of $$ you make us save in a month... I think plenty of professionals will agree on that contract without even have a pre look at your AWS bill hahaha (I would!) I did it recently on my customer bill: more then 45% of pure waste! some examples: * Loki/DynamoDB: uhhh, scary: the defaults for hosting in AWS imply more then 500$ per months https://github.com/grafana/loki/issues/2801 https://github.com/grafana/loki/issues/2801 (so be aware of 'experiments' or quick installations) * EFS (aka NFS): you pay by space... and.. .there is no limits! we had a backup that was not rotating and wasted (aka paid!) for almost 1Tera (this could not happend in the past for a warning 'disk full' coming from your sysadmin) * volumes nobody removes (and useless volumes that proliferates when you have autoscaling on) * Cloudwatch metrics that starts to be cheap then become pretty expensive (be aware that metrics of EC2 instances stays, so you pay for it, for 1 year even if instances are terminated!) * esperiment in AWS kinesis by an intern left there (with a drain of $$) Please note, aws gives you plenty of notices and documentations on everything, there are no trick from their side at all: the problem is the whole "cloud" and pay per use payment structure that made a revolution on who is checking the usage (so the bill), and orgs are just not prepared. You all will agree AWS has all the metrics to see how customers just waste $$$: they of course cannot understand if the data in EFS is useless or not, but on the majority of services they sure can... and I think they also have clear statistics that tell them the $$ that they are making just by having users not taking care: and this amount I think is soooo huge that they can even afford to tell you (in the 'trusted adviser'... that, btw, you only have with the highest cost of support) I hope your post will push users to be more conscious on how they spend their money A final word to all non US customers of AWS: are you aware of the huge amount your country is just giving away? EU wake up!
- zeeshah 4y ago
  Thank you for giving so many valuable suggestions. * My few hired some professionals and they told me that it's really difficult to actually find those, who actually care to reduce your AWS costs. There are few third party tools in the market that gives budget alerts, daily cost of service breakdown etc. when integrated with AWS account? Have you tried any of those and how is your experience?
  - oriettaxx 4y ago
    Yes, I know there exists those services and I have no doubt they can be effective, but we hit the following issues (it is our experience): 1. somehow it is easier to give access to a single professional (a human) instead of an external tool 2. the easiest way to check bills is to have full access to an AWS account, and it is not (due to bureaucracy) an easy task to allow an external tool to have full access (in the EU you have many GDPR implications, which is good of course). 3. at the very easiest level, you may share your screen with your expenses dashboard to a professional via video call without having him allowed to access directly your account This is our experience, but I am sure many others here can give feed-backs (and I am looking forward to read that, too)
  - stefanoco 4y ago
    Totally agree with oriettaxx: nothing beats the supervision of an experienced professional. Automated tools are much like AWS... when automation fails, cost start rising again! Our experience is exactly the same as per your suggestions: we eventually switched to a set of properly configured (by a highly skilled and not so expensive pro) bare metal servers at a well known EU data center, and a few additional VPCs for commodity services that we might want to change more frequently. Was a new life both for money and peace of mind. Each time a receive a bunch of emails from AWS for the billing of a very small service still hosted there my heart starts beating fast. Because they like writing looong messages and duplicates, with plenty of details about your service and bill, and many thanks and everything so difficult to understand. I'm not blaming AWS of course: it's a legit business model, I don't particularly like. Your mileage may vary... but not so much actually unless you're in a highly variable type of business, with real need for constant change of configuration or resources.

AWS Billing Issue

3 points by zeeshah 4y ago

This AWS Billing is quite overwhelming. Are there any genuine ways to control it?

This question is too broad as is. A good place to start is looking at your cost breakdown which is available in the console. For example, if it shows your highest spending is on EC2 then you should assess your instance types and use case. If your S3 spending is high, assess what you're storing; if any of it is low-access, assess the storage class you're using and consider glacier. These are just two examples of two services and how you might address high spend; AWS has countless services so it is highly dependent on what you're actually using. So figure that out and then google reducing AWS spend for [specific service name]. Good luck!

zeeshah 4y ago
Actually I had an unused instance and it took me quite a lot of time to actually figure out that this was the actual cause. I do check the prices but with so many deep learning projects, it's actually really difficult to figure out. Have you tried any third party AWS cost optimization tool ?

oriettaxx 4y ago

welcome to the club My advice: * hire a professional to have a good look at costs: you'll be "amazed" on how much $$ are wasted: a 15 days contract is more then enough: you can even tell him/her: we pay you exactly the amount of $$ you make us save in a month... I think plenty of professionals will agree on that contract without even have a pre look at your AWS bill hahaha (I would!) I did it recently on my customer bill: more then 45% of pure waste! some examples: * Loki/DynamoDB: uhhh, scary: the defaults for hosting in AWS imply more then 500$ per months https://github.com/grafana/loki/issues/2801 https://github.com/grafana/loki/issues/2801 (so be aware of 'experiments' or quick installations) * EFS (aka NFS): you pay by space... and.. .there is no limits! we had a backup that was not rotating and wasted (aka paid!) for almost 1Tera (this could not happend in the past for a warning 'disk full' coming from your sysadmin) * volumes nobody removes (and useless volumes that proliferates when you have autoscaling on) * Cloudwatch metrics that starts to be cheap then become pretty expensive (be aware that metrics of EC2 instances stays, so you pay for it, for 1 year even if instances are terminated!) * esperiment in AWS kinesis by an intern left there (with a drain of $$) Please note, aws gives you plenty of notices and documentations on everything, there are no trick from their side at all: the problem is the whole "cloud" and pay per use payment structure that made a revolution on who is checking the usage (so the bill), and orgs are just not prepared. You all will agree AWS has all the metrics to see how customers just waste $$$: they of course cannot understand if the data in EFS is useless or not, but on the majority of services they sure can... and I think they also have clear statistics that tell them the $$ that they are making just by having users not taking care: and this amount I think is soooo huge that they can even afford to tell you (in the 'trusted adviser'... that, btw, you only have with the highest cost of support) I hope your post will push users to be more conscious on how they spend their money A final word to all non US customers of AWS: are you aware of the huge amount your country is just giving away? EU wake up!

zeeshah 4y ago
Thank you for giving so many valuable suggestions. * My few hired some professionals and they told me that it's really difficult to actually find those, who actually care to reduce your AWS costs. There are few third party tools in the market that gives budget alerts, daily cost of service breakdown etc. when integrated with AWS account? Have you tried any of those and how is your experience?
- oriettaxx 4y ago
  Yes, I know there exists those services and I have no doubt they can be effective, but we hit the following issues (it is our experience): 1. somehow it is easier to give access to a single professional (a human) instead of an external tool 2. the easiest way to check bills is to have full access to an AWS account, and it is not (due to bureaucracy) an easy task to allow an external tool to have full access (in the EU you have many GDPR implications, which is good of course). 3. at the very easiest level, you may share your screen with your expenses dashboard to a professional via video call without having him allowed to access directly your account This is our experience, but I am sure many others here can give feed-backs (and I am looking forward to read that, too)
- stefanoco 4y ago
  Totally agree with oriettaxx: nothing beats the supervision of an experienced professional. Automated tools are much like AWS... when automation fails, cost start rising again! Our experience is exactly the same as per your suggestions: we eventually switched to a set of properly configured (by a highly skilled and not so expensive pro) bare metal servers at a well known EU data center, and a few additional VPCs for commodity services that we might want to change more frequently. Was a new life both for money and peace of mind. Each time a receive a bunch of emails from AWS for the billing of a very small service still hosted there my heart starts beating fast. Because they like writing looong messages and duplicates, with plenty of details about your service and bill, and many thanks and everything so difficult to understand. I'm not blaming AWS of course: it's a legit business model, I don't particularly like. Your mileage may vary... but not so much actually unless you're in a highly variable type of business, with real need for constant change of configuration or resources.