8 ms·

Have you ever used AWS? There's a whole suite of tools they provide around pricing and budgeting. > It's quite a tour de force how Amazon have taken "separatio

by exhaze 6y ago

Have you ever used AWS? There's a whole suite of tools they provide around pricing and budgeting. > It's quite a tour de force how Amazon have taken "separation of concerns", applied it to web services and used it to create complex and difficult to understand or predict pricing to print money. Is applying separation of concerns to web services really that bad? Look, if you're a small company with a simple product, you can just put your stuff on a few EC2 boxes and pay the monthly bill for that. At that size, your infra costs are going to be dwarfed by your other costs of doing business anyway. If you're a big business, you can literally pay people to keep track of this stuff. You've got the extra money, because now you don't employ data center architects, server engineers, etc. AWS is able to "print money" because it brings a lot of value to many businesses.

rumanator 6y ago
> Have you ever used AWS? There's a whole suite of tools they provide around pricing and budgeting. They don't work. To elaborate, they fail to provide the user with information with enough clarity to allow him to form accurate mental models of what is being used,what it costs, and what is the implication in cost of applying some change. Heck, at any given moment it's unexplainably hard to tell exactly which AWS services are you actually using, and being billed for. The only reliable info on cost it the invoice you receive at the end of the month. Even then, the AWS guys felt the need to develop a full blown machine learning service to help customers predict what their next invoice will be.
- exhaze 6y ago
  > Heck, at any given moment it's unexplainably hard to tell exactly which AWS services are you actually using, and being billed for. Did you actually look at the billing page? Here's a screenshot of my billing page: https://imgur.com/a/DQAxgRY https://imgur.com/a/DQAxgRY It meticulously breaks out per-service costs across whatever dimension you want. "EC2-Other" is rolled up in the chart, but it's broken out by each individual item further down in the list.
  - coder543 6y ago
    Isn’t that exactly what the person you replied to said? You’re showing what you’ve been charged for previously. That part works, at a certain level of detail. That doesn’t show what you are being charged for right now, nor does it help you predict what your total costs will be at the end of the month.
    - exhaze 6y ago
      You bring up a good point - it would be a good idea for AWS to set some sensible max budget defaults for new accounts to prevent surprises. Since they don't, if you are literally so concerned with your current costs for this month, you can set a budget and alerts for whatever you want, so that you don't exceed some cost. Also, they show you your current cost for this month, up until the moment, as well as a daily breakdown of your costs.
      - coder543 6y ago
        > You bring up a good point - it would be a good idea for AWS to set some sensible max budget defaults for new accounts to prevent surprises. I didn’t say anything about this at all. It would certainly be a nice feature, and one people have asked for since the very beginning of AWS. > Since they don't, if you are literally so concerned with your current costs for this month, you can set a budget and alerts for whatever you want, so that you don't exceed some cost. Again, this is a strawman. This isn’t a response to anything I said. If you’re trying to understand how much a new deployment on AWS is costing you, and find ways to optimize that, you don’t care about the total budget of the account, or having some auto shutoff. You need detailed, real-time information from Amazon, which I’ve never seen in the console. It’s been several years since I’ve had to be digging around in the billing console, so I have no stake in this discussion and things might have changed. Based on other comments being made around here, I really doubt anything significant has changed. I was just trying to clarify a comment you completely misunderstood.
        exhaze 6y ago
        > If you’re trying to understand how much a new deployment on AWS is costing you, and find ways to optimize that, you don’t care about the total budget of the account, or having some auto shutoff. You need detailed, real-time information from Amazon, which I’ve never seen in the console. No one who is working on this cares about real-time by the second data - the daily cost is enough. > Again, this is a strawman. This isn’t a response to anything I said. OK, what are you trying to say? If my response above did not answer the question, can you give me a one sentence summary of the point?
- StreamBright 6y ago
  If the AWS tools fail to provide you this information than you can just tag everything (we use scope, stage, service, region, owner tags, example: global, dev, hadoop, eu-west-1, data-engineering) and it is trivial to generate a cost report where you can drill down by environments, teams, services, etc.
  - scarface74 6y ago
    Exactly, I’m often fielding questions over on r/aws about how to do X on AWS where X is something simple. I then ask are they trying to solve problem X or are they trying to learn AWS. If it is the former, just use LightSail. Outside of serverless applications, I wouldn’t want the complications of full fledge AWS for a small project and I get paid to know the ins and outs of it.
mherrmann 6y ago
> There's a whole suite of tools they provide around pricing and budgeting. It could be there are such tools. But they are impossible to find, setup or understand for someone who is not already an AWS expert. I recently got a $500 bill from AWS. It was impossible for me to find out which S3 buckets (or even specific files) caused these costs. I looked at this for half an hour and have used AWS before. They just don't tell you how to obtain this information. Or maybe they do, and it is so well hidden that it might as well not exist. This was a horrible user experience. As a customer, I want to know what I'm being charged for. The fact that AWS make it so difficult make me never want to use them again.
- exhaze 6y ago
  It's literally called "Billing" and the "Cost Explorer" is the first item in the navigation when you go to the page. I've attached a screenshot and highlighted it for you. FYI, this month is the first time I've opened the AWS GUI in 4-5 months, and it took me about a minute to find the billing page and drill into our AWS costs. I've provided an image that shows how to get to the cost explorer from "Billing" (you can use the search box available on the AWS account landing page to type in "Billing" in order to access the billing page of AWS). EDIT: sorry, I forgot to include the screenshot link - https://imgur.com/a/tVFoocr https://imgur.com/a/tVFoocr
  - mherrmann 6y ago
    This does not show the costs per bucket. Specifically, I was interested in the costs for data transfer for each of my buckets. As I said, after half an hour of looking into the "Billing" in AWS, the Cost Explorer and googling around I simply could not find a way to obtain this information.
    - helsinkiandrew 6y ago
      If this is an ongoing issue and you can’t get the information from something that is in front of S3 (cloudfront/your application etc) then you can enable bucket logging in the settings (which costs!)
davedx 6y ago
> Have you ever used AWS? There's a whole suite of tools they provide around pricing and budgeting. Yes, I use it extensively. > Is applying separation of concerns to web services really that bad? Look, if you're a small company with a simple product, you can just put your stuff on a few EC2 boxes and pay the monthly bill for that. At that size, your infra costs are going to be dwarfed by your other costs of doing business anyway. If you're a big business, you can literally pay people to keep track of this stuff. You've got the extra money, because now you don't employ data center architects, server engineers, etc. AWS is able to "print money" because it brings a lot of value to many businesses. It's not bad per se. The real problem I often see is that many people within the typical org aren't aware of how these costs can accumulate, especially when you have multiple DTAP environments, microservices, and so on. Another problem is the days of "put your stuff on a few EC2 boxes" seem to have been overtaken by the cloud native, microservices, k8s trends of the last few years. Personally I 100% agree this is the way to go for many (if not most) people -- and for this purpose Digital Ocean is just as good -- but what I see more often is people diving straight into the deep end, because "Is applying separation of concerns to web services really that bad?" I also think even big businesses underestimate the total costs of these all-in cloud infrastructures. You indeed need to pay people to keep track of this stuff -- multiple people. Your cloud costs + your devops/platform personnel costs can get to be a significant % of your total IT costs. Is it really necessary? Just because your business has a big market cap or significant revenue from digital doesn't necessarily mean it needs any of that stuff. Chances are you can run it all on the same couple of EC2 boxes you ran it on when you were still a small company.
- exhaze 6y ago
  I think we are probably 90% in agreement on things, just talking about it in different terms. I think AWS/GCP have both gotten overwhelming in terms of how many different services they provide. Many people fall for it and end up playing AWS service golf (https://news.ycombinator.com/item?id=23274668 https://news.ycombinator.com/item?id=23274668). Maybe a good analogy is to think of AWS as C. It's flexible and powerful, but it's easy to shoot yourself in the foot. That would explain why a lot of these "> magical-tool deploy" services have emerged, promising to abstract away AWS.
sixothree 6y ago
They are awful. I have tried and tried to list the cost by resource (like you can in Azure) and I just can't figure it out. I think I might need to enable some special pricing tools (coincidentally not free) that Cost Explorer don't expose.
wolco 6y ago
From where I stand as someone who isn't using Amazon figuring out what my costs will be is difficult, impossible and would require wasting money on various tests. With something like digitalocean my costs are fixed. I might have misjudged how much I need but can scale up easily or down (with effort). Will Amazon offer fixed pricing? Does Amazom offer fixed pricing?
- anaganisk 6y ago
  Aws is definitely not for hobbyists, or small applications, its a replacement to a datacenter with a lot of variables, not a VM. Your better comparison would be DigitalOcean vs Lightsail (AWS). So yes.
- scarface74 6y ago
  Yes. https://aws.amazon.com/lightsail/ https://aws.amazon.com/lightsail/
Spooky23 6y ago
AWS (or GCP or Azure or ...) value proposition depends on you. For a small/startup, the ability to execute quickly without sinking capital into a product that will likely pivot anyway is a superpower. When you get bigger, stuff gets dicey. Paying a premium for a "X as a Service" made sense when you had no people, but at some point the cost structure may not make sense or your needs are at odds with how the service delivery works. When you work for a big company or government agency, things change again -- you don't hire a database guy for a project, you have a project team that builds stuff, and an ops team that exerts the minimum viable effort to keep your stuff running or an outsourced ops team that does the same, but worse. At that point, you start looking at how much it costs to pay AWS to provide and SLA vs hiring more competent (ie. more $$$) ops people to keep it running. The long and short of it is that AWS is in a position to rake in cash up and down the spectrum -- all they need to do is be competent, which they are very good at.
rudolph9 6y ago
They’re literally entire companies of 100+ employees selling a product that helps you figure out how much money you’re spending on AWS https://www.cloudability.com/ https://www.cloudability.com/

by exhaze 6y ago

rumanator 6y ago

> Have you ever used AWS? There's a whole suite of tools they provide around pricing and budgeting. They don't work. To elaborate, they fail to provide the user with information with enough clarity to allow him to form accurate mental models of what is being used,what it costs, and what is the implication in cost of applying some change. Heck, at any given moment it's unexplainably hard to tell exactly which AWS services are you actually using, and being billed for. The only reliable info on cost it the invoice you receive at the end of the month. Even then, the AWS guys felt the need to develop a full blown machine learning service to help customers predict what their next invoice will be.

exhaze 6y ago
> Heck, at any given moment it's unexplainably hard to tell exactly which AWS services are you actually using, and being billed for. Did you actually look at the billing page? Here's a screenshot of my billing page: https://imgur.com/a/DQAxgRY https://imgur.com/a/DQAxgRY It meticulously breaks out per-service costs across whatever dimension you want. "EC2-Other" is rolled up in the chart, but it's broken out by each individual item further down in the list.
- coder543 6y ago
  Isn’t that exactly what the person you replied to said? You’re showing what you’ve been charged for previously. That part works, at a certain level of detail. That doesn’t show what you are being charged for right now, nor does it help you predict what your total costs will be at the end of the month.
  - exhaze 6y ago
    You bring up a good point - it would be a good idea for AWS to set some sensible max budget defaults for new accounts to prevent surprises. Since they don't, if you are literally so concerned with your current costs for this month, you can set a budget and alerts for whatever you want, so that you don't exceed some cost. Also, they show you your current cost for this month, up until the moment, as well as a daily breakdown of your costs.
    - coder543 6y ago
      > You bring up a good point - it would be a good idea for AWS to set some sensible max budget defaults for new accounts to prevent surprises. I didn’t say anything about this at all. It would certainly be a nice feature, and one people have asked for since the very beginning of AWS. > Since they don't, if you are literally so concerned with your current costs for this month, you can set a budget and alerts for whatever you want, so that you don't exceed some cost. Again, this is a strawman. This isn’t a response to anything I said. If you’re trying to understand how much a new deployment on AWS is costing you, and find ways to optimize that, you don’t care about the total budget of the account, or having some auto shutoff. You need detailed, real-time information from Amazon, which I’ve never seen in the console. It’s been several years since I’ve had to be digging around in the billing console, so I have no stake in this discussion and things might have changed. Based on other comments being made around here, I really doubt anything significant has changed. I was just trying to clarify a comment you completely misunderstood.
      - exhaze 6y ago
        > If you’re trying to understand how much a new deployment on AWS is costing you, and find ways to optimize that, you don’t care about the total budget of the account, or having some auto shutoff. You need detailed, real-time information from Amazon, which I’ve never seen in the console. No one who is working on this cares about real-time by the second data - the daily cost is enough. > Again, this is a strawman. This isn’t a response to anything I said. OK, what are you trying to say? If my response above did not answer the question, can you give me a one sentence summary of the point?
StreamBright 6y ago
If the AWS tools fail to provide you this information than you can just tag everything (we use scope, stage, service, region, owner tags, example: global, dev, hadoop, eu-west-1, data-engineering) and it is trivial to generate a cost report where you can drill down by environments, teams, services, etc.
- scarface74 6y ago
  Exactly, I’m often fielding questions over on r/aws about how to do X on AWS where X is something simple. I then ask are they trying to solve problem X or are they trying to learn AWS. If it is the former, just use LightSail. Outside of serverless applications, I wouldn’t want the complications of full fledge AWS for a small project and I get paid to know the ins and outs of it.

mherrmann 6y ago

> There's a whole suite of tools they provide around pricing and budgeting. It could be there are such tools. But they are impossible to find, setup or understand for someone who is not already an AWS expert. I recently got a $500 bill from AWS. It was impossible for me to find out which S3 buckets (or even specific files) caused these costs. I looked at this for half an hour and have used AWS before. They just don't tell you how to obtain this information. Or maybe they do, and it is so well hidden that it might as well not exist. This was a horrible user experience. As a customer, I want to know what I'm being charged for. The fact that AWS make it so difficult make me never want to use them again.

exhaze 6y ago
It's literally called "Billing" and the "Cost Explorer" is the first item in the navigation when you go to the page. I've attached a screenshot and highlighted it for you. FYI, this month is the first time I've opened the AWS GUI in 4-5 months, and it took me about a minute to find the billing page and drill into our AWS costs. I've provided an image that shows how to get to the cost explorer from "Billing" (you can use the search box available on the AWS account landing page to type in "Billing" in order to access the billing page of AWS). EDIT: sorry, I forgot to include the screenshot link - https://imgur.com/a/tVFoocr https://imgur.com/a/tVFoocr
- mherrmann 6y ago
  This does not show the costs per bucket. Specifically, I was interested in the costs for data transfer for each of my buckets. As I said, after half an hour of looking into the "Billing" in AWS, the Cost Explorer and googling around I simply could not find a way to obtain this information.
  - helsinkiandrew 6y ago
    If this is an ongoing issue and you can’t get the information from something that is in front of S3 (cloudfront/your application etc) then you can enable bucket logging in the settings (which costs!)

davedx 6y ago

> Have you ever used AWS? There's a whole suite of tools they provide around pricing and budgeting. Yes, I use it extensively. > Is applying separation of concerns to web services really that bad? Look, if you're a small company with a simple product, you can just put your stuff on a few EC2 boxes and pay the monthly bill for that. At that size, your infra costs are going to be dwarfed by your other costs of doing business anyway. If you're a big business, you can literally pay people to keep track of this stuff. You've got the extra money, because now you don't employ data center architects, server engineers, etc. AWS is able to "print money" because it brings a lot of value to many businesses. It's not bad per se. The real problem I often see is that many people within the typical org aren't aware of how these costs can accumulate, especially when you have multiple DTAP environments, microservices, and so on. Another problem is the days of "put your stuff on a few EC2 boxes" seem to have been overtaken by the cloud native, microservices, k8s trends of the last few years. Personally I 100% agree this is the way to go for many (if not most) people -- and for this purpose Digital Ocean is just as good -- but what I see more often is people diving straight into the deep end, because "Is applying separation of concerns to web services really that bad?" I also think even big businesses underestimate the total costs of these all-in cloud infrastructures. You indeed need to pay people to keep track of this stuff -- multiple people. Your cloud costs + your devops/platform personnel costs can get to be a significant % of your total IT costs. Is it really necessary? Just because your business has a big market cap or significant revenue from digital doesn't necessarily mean it needs any of that stuff. Chances are you can run it all on the same couple of EC2 boxes you ran it on when you were still a small company.

exhaze 6y ago
I think we are probably 90% in agreement on things, just talking about it in different terms. I think AWS/GCP have both gotten overwhelming in terms of how many different services they provide. Many people fall for it and end up playing AWS service golf (https://news.ycombinator.com/item?id=23274668 https://news.ycombinator.com/item?id=23274668). Maybe a good analogy is to think of AWS as C. It's flexible and powerful, but it's easy to shoot yourself in the foot. That would explain why a lot of these "> magical-tool deploy" services have emerged, promising to abstract away AWS.

sixothree 6y ago

They are awful. I have tried and tried to list the cost by resource (like you can in Azure) and I just can't figure it out. I think I might need to enable some special pricing tools (coincidentally not free) that Cost Explorer don't expose.

wolco 6y ago

From where I stand as someone who isn't using Amazon figuring out what my costs will be is difficult, impossible and would require wasting money on various tests. With something like digitalocean my costs are fixed. I might have misjudged how much I need but can scale up easily or down (with effort). Will Amazon offer fixed pricing? Does Amazom offer fixed pricing?

anaganisk 6y ago
Aws is definitely not for hobbyists, or small applications, its a replacement to a datacenter with a lot of variables, not a VM. Your better comparison would be DigitalOcean vs Lightsail (AWS). So yes.
scarface74 6y ago
Yes. https://aws.amazon.com/lightsail/ https://aws.amazon.com/lightsail/

Spooky23 6y ago

AWS (or GCP or Azure or ...) value proposition depends on you. For a small/startup, the ability to execute quickly without sinking capital into a product that will likely pivot anyway is a superpower. When you get bigger, stuff gets dicey. Paying a premium for a "X as a Service" made sense when you had no people, but at some point the cost structure may not make sense or your needs are at odds with how the service delivery works. When you work for a big company or government agency, things change again -- you don't hire a database guy for a project, you have a project team that builds stuff, and an ops team that exerts the minimum viable effort to keep your stuff running or an outsourced ops team that does the same, but worse. At that point, you start looking at how much it costs to pay AWS to provide and SLA vs hiring more competent (ie. more $$$) ops people to keep it running. The long and short of it is that AWS is in a position to rake in cash up and down the spectrum -- all they need to do is be competent, which they are very good at.

rudolph9 6y ago

They’re literally entire companies of 100+ employees selling a product that helps you figure out how much money you’re spending on AWS https://www.cloudability.com/ https://www.cloudability.com/