6 ms·

I would say the main story here is that AWS overcharges on traffic to such a ridiculous degree. My SaaS regularly consumes 50+ TB in traffic, but I pay €150 in

I would say the main story here is that AWS overcharges on traffic to such a ridiculous degree. My SaaS regularly consumes 50+ TB in traffic, but I pay €150 in monthly fees for that. For someone coming from my perspective, that would be a huge and unpleasant surprise to be billed $2600 for a service that I know costs $180 elsewhere. Edit: especially so because people keep repeating the mantra that one should use the cloud to save money by not paying for what you don't use. Obviously, my 3 Xeon servers with 64 GB RAM each are way overpowered for sending out some GB of static files, but I wanted to have a bit of redundancy. But with my setup, there should be plenty of obvious inefficiencies for "the cloud" to eliminate => It feels like cloud should be cheaper than my dedicated servers. But it's not, and that is the unpleasant surprise.

Malcx 6y ago
Out of interest who are your servers with? That seems a good price and might be something I'm interested in...
- thg 6y ago
  Not the OP, but it could be Hetzner. They're in that price range, with that kind of hardware. I myself have been a happy customer there for almost 20 years now.
  - fxtentacle 6y ago
    OP here, yes I use the hetzner marketplace for used dedicated servers. I have found that using hardware that's 6+ months old actually decreased failure rate. Plus it's cheaper and they waive the setup fee.
    - Malcx 6y ago
      The marketplace has some really good deals on it! Thanks for sharing.
  - dvfjsdhgfv 6y ago
    Of of the good aspects of Hetzner that they also offer desktop CPUs which are often more powerful than the so-called enterprise CPUs (see https://jan.rychter.com/enblog/cloud-server-cpu-performance-comparison-2019-12-12 https://jan.rychter.com/enblog/cloud-server-cpu-performance-...).
    - chucky_z 6y ago
      I've gotten servers from Hivelocity in the past, they like(d) to offer servers with the "enterprise" equivalent of these CPUs, I'm not sure if Intel still sells them, but there used to be a "xeon i3" basically, usually 4 core CPUs clocked to the moon, that supported ECC. They were really good!
_jjkk 6y ago
Of course the cost of data egress for AWS does not add up to nearly what they charge for it. But it's totally unfair to say they're "overcharging". The pricing is set up to encourage using the service properly. For example, did you know you get unlimited free (very fast) transfer between s3 <-> ec2? You are free to use S3 however you want, but the pricing is set up such that people use a proper cdn backed by s3, do as much as they can between ec2 <-> s3, etc. instead of making s3 the backbone of their public site. If you want to use it in a way it's not intended it will cost you more $$ which is how it should be.
- ooobit2 6y ago
  No one prices to "encourage proper use." They price for a number of variables. And I do agree that arguing they're overcharging is unfair; however, SU-pricing standards include triggers for excessive use as part of the customer guarantee. If Amazon has no problem automating the product, they can automate some billing features, like notifications and bandwidth restrictions when a customer is nearing excessive use based on the population use of a particular service. Amazon's systems should have notified at least twice and required direct confirmation from the customer for their continued use. We don't just make business decisions like this based off our self-interest. In this case, a failure to be resourceful on Amazon's part gave the customer one of two options: track everything manually minute-by-minute, or be surprised by an excessive bill.
  - deleted 6y ago
    [deleted]
  - alFReD-NSH 6y ago
    AWS does have budget notifications that you can set up with a few clicks. It doesn't come by default because AWS doesn't know what's the right amount for those notifications. I already get enough emails from AWS for every account that I make. I can actually think of possible/real workloads that would burn 2700 usd out of blue in two days and those customers would not be happy if AWS blocks their account because AWS thinks they did something wrong. AWS Support quite often happily refunds these amounts and you don't need to post to Twitter. I've seen them refund a lot more. Why haven't they done something to prevent this mistakes? They already refund these things, it's just those refunds is a drop in bucket compared to services that enterprise customers are paying for, and AWS engineers are busy building for them. AWS gives you a lot of power compared to what you can do on your average hosting provider, but sadly there's also a lot of room to shoot yourself in the foot if you don't know what you are doing.
siscia 6y ago
Always thought that a great statup idea would be to interact with dedicated server provider and offer on top of those cheap servers AWS like services. The most common services like S3, Kafka, RDS (pg and mysql), Redis should be enough to cover most use cases. With k8s and a dedicated smart team, it would be a possible adventure. Moreover it can work great also on-premises. Several old medium business have their own physical infrastructure and they are not yet ready to move to the cloud.
- kazen44 6y ago
  i don't know about the US, but in europe a ton of these smaller, local parties exist. (usually called something like managed infrastructure or thereabouts).
- ignoramous 6y ago
  > Always thought that a great statup idea would be to interact with dedicated server provider and offer on top of those cheap servers AWS like services. Which is what AWS is anyway...? Also, I don't think AWS' data-centers exist in isolation: https://www.datacenterdynamics.com/en/news/wikileaks-publishes-list-aws-data-center-locations-colo-providers/ https://www.datacenterdynamics.com/en/news/wikileaks-publish... Definitely not the Cloudfront PoPs which ought to be leased, surely? Also see: https://www.tatacommunications.com/solutions/cloud/infra-dc/ https://www.tatacommunications.com/solutions/cloud/infra-dc/ And: https://www.vapor.io/vapor-io-partners-with-cloudflare-on-nationwide-deployment/ https://www.vapor.io/vapor-io-partners-with-cloudflare-on-na...
- alFReD-NSH 6y ago
  There's a actually a lot of companies doing that. Google 'aws managed services'
  - bigiain 6y ago
    Some of those companies are even more eyewateringly expensive than the story in the article. I worked on a project a few years back where the client were paying $10k/month as a "managed service" fee, on top of about $4k/month worth of platform at full on-demand prices. I showed the client how they could have it all running on reserved instances for Prod and spot instances for dev/staging for under $2k/month - but no, somebody had signed up for $14k+ per month just to have someone to blame/shout at 24x7 if something went wrong. (And that company was ~85% likely to call me and blame it on the app before they even bothered looking to see if the platform was working...)
freakz 6y ago
Who is your provider?
ufmace 6y ago
IMO, it's not so much that they overcharge as that the service was created and priced for a different purpose. It's possible to serve huge files to tens of thousands of people, but it's not really what it's meant for. Naturally, the pricing isn't really optimized for that, and a service that's designed for that can be an order of magnitude cheaper. Seems to me S3 is more designed for highly reliable data storage. If you wanna store a bunch of data in a way that's highly reliable and resilient to hardware failures and also not have to worry about managing RAIDs and clusters of servers for the data size, S3 is just the thing, and probably priced pretty fairly. It's also a capable and flexible service. It's possible to use it for things it isn't really designed for and have it behave pretty well. Well enough that you can mostly ignore that the other thing isn't really what it's designed around. The "mostly" is important, though. If you start hitting any extremes while using it in a different way, then you certainly can run into some pathological cases in the pricing structure and pay way too much for something that would have been cheaper in a service dedicated to that.
- fxtentacle 6y ago
  S3 is much cheaper than the egress traffic. As such, this isn't so much about S3 because you'd be paying the same price for that traffic if you were using EC2 instead of S3. Looking purely at the traffic price, I believe the word choice "overcharging" is warranted if you can buy the exact same product elsewhere for 90% less.
- raiyu 6y ago
  They over charge. Their cost is less than .01 cents per gb and look at their published pricing for customers. And I’m including networking equipment as well as head count in my cost, not just the cost of pipes.
  - cortesoft 6y ago
    They charge so much for it as a disincentive to use it for CDN purposes. It is not architected to act as a CDN for people, and if they don't overcharge for egress, people will use it as a CDN because it is easy.
imtringued 6y ago
This is part of AWS's business strategy. Overcharge to an absurd extent (90% margins) and then if a surprise bill happens you can just refund it for 10% of the cost. This cost is a rounding error if it allows you to maintain your margins. It's worth it even if only a single person is willing to pay the regular rate.

Malcx 6y ago

Out of interest who are your servers with? That seems a good price and might be something I'm interested in...

thg 6y ago
Not the OP, but it could be Hetzner. They're in that price range, with that kind of hardware. I myself have been a happy customer there for almost 20 years now.
- fxtentacle 6y ago
  OP here, yes I use the hetzner marketplace for used dedicated servers. I have found that using hardware that's 6+ months old actually decreased failure rate. Plus it's cheaper and they waive the setup fee.
  - Malcx 6y ago
    The marketplace has some really good deals on it! Thanks for sharing.
- dvfjsdhgfv 6y ago
  Of of the good aspects of Hetzner that they also offer desktop CPUs which are often more powerful than the so-called enterprise CPUs (see https://jan.rychter.com/enblog/cloud-server-cpu-performance-comparison-2019-12-12 https://jan.rychter.com/enblog/cloud-server-cpu-performance-...).
  - chucky_z 6y ago
    I've gotten servers from Hivelocity in the past, they like(d) to offer servers with the "enterprise" equivalent of these CPUs, I'm not sure if Intel still sells them, but there used to be a "xeon i3" basically, usually 4 core CPUs clocked to the moon, that supported ECC. They were really good!

_jjkk 6y ago

Of course the cost of data egress for AWS does not add up to nearly what they charge for it. But it's totally unfair to say they're "overcharging". The pricing is set up to encourage using the service properly. For example, did you know you get unlimited free (very fast) transfer between s3 <-> ec2? You are free to use S3 however you want, but the pricing is set up such that people use a proper cdn backed by s3, do as much as they can between ec2 <-> s3, etc. instead of making s3 the backbone of their public site. If you want to use it in a way it's not intended it will cost you more $$ which is how it should be.

ooobit2 6y ago
No one prices to "encourage proper use." They price for a number of variables. And I do agree that arguing they're overcharging is unfair; however, SU-pricing standards include triggers for excessive use as part of the customer guarantee. If Amazon has no problem automating the product, they can automate some billing features, like notifications and bandwidth restrictions when a customer is nearing excessive use based on the population use of a particular service. Amazon's systems should have notified at least twice and required direct confirmation from the customer for their continued use. We don't just make business decisions like this based off our self-interest. In this case, a failure to be resourceful on Amazon's part gave the customer one of two options: track everything manually minute-by-minute, or be surprised by an excessive bill.
- deleted 6y ago
  [deleted]
- alFReD-NSH 6y ago
  AWS does have budget notifications that you can set up with a few clicks. It doesn't come by default because AWS doesn't know what's the right amount for those notifications. I already get enough emails from AWS for every account that I make. I can actually think of possible/real workloads that would burn 2700 usd out of blue in two days and those customers would not be happy if AWS blocks their account because AWS thinks they did something wrong. AWS Support quite often happily refunds these amounts and you don't need to post to Twitter. I've seen them refund a lot more. Why haven't they done something to prevent this mistakes? They already refund these things, it's just those refunds is a drop in bucket compared to services that enterprise customers are paying for, and AWS engineers are busy building for them. AWS gives you a lot of power compared to what you can do on your average hosting provider, but sadly there's also a lot of room to shoot yourself in the foot if you don't know what you are doing.

siscia 6y ago

Always thought that a great statup idea would be to interact with dedicated server provider and offer on top of those cheap servers AWS like services. The most common services like S3, Kafka, RDS (pg and mysql), Redis should be enough to cover most use cases. With k8s and a dedicated smart team, it would be a possible adventure. Moreover it can work great also on-premises. Several old medium business have their own physical infrastructure and they are not yet ready to move to the cloud.

kazen44 6y ago
i don't know about the US, but in europe a ton of these smaller, local parties exist. (usually called something like managed infrastructure or thereabouts).
ignoramous 6y ago
> Always thought that a great statup idea would be to interact with dedicated server provider and offer on top of those cheap servers AWS like services. Which is what AWS is anyway...? Also, I don't think AWS' data-centers exist in isolation: https://www.datacenterdynamics.com/en/news/wikileaks-publishes-list-aws-data-center-locations-colo-providers/ https://www.datacenterdynamics.com/en/news/wikileaks-publish... Definitely not the Cloudfront PoPs which ought to be leased, surely? Also see: https://www.tatacommunications.com/solutions/cloud/infra-dc/ https://www.tatacommunications.com/solutions/cloud/infra-dc/ And: https://www.vapor.io/vapor-io-partners-with-cloudflare-on-nationwide-deployment/ https://www.vapor.io/vapor-io-partners-with-cloudflare-on-na...
alFReD-NSH 6y ago
There's a actually a lot of companies doing that. Google 'aws managed services'
- bigiain 6y ago
  Some of those companies are even more eyewateringly expensive than the story in the article. I worked on a project a few years back where the client were paying $10k/month as a "managed service" fee, on top of about $4k/month worth of platform at full on-demand prices. I showed the client how they could have it all running on reserved instances for Prod and spot instances for dev/staging for under $2k/month - but no, somebody had signed up for $14k+ per month just to have someone to blame/shout at 24x7 if something went wrong. (And that company was ~85% likely to call me and blame it on the app before they even bothered looking to see if the platform was working...)

freakz 6y ago

Who is your provider?

ufmace 6y ago

IMO, it's not so much that they overcharge as that the service was created and priced for a different purpose. It's possible to serve huge files to tens of thousands of people, but it's not really what it's meant for. Naturally, the pricing isn't really optimized for that, and a service that's designed for that can be an order of magnitude cheaper. Seems to me S3 is more designed for highly reliable data storage. If you wanna store a bunch of data in a way that's highly reliable and resilient to hardware failures and also not have to worry about managing RAIDs and clusters of servers for the data size, S3 is just the thing, and probably priced pretty fairly. It's also a capable and flexible service. It's possible to use it for things it isn't really designed for and have it behave pretty well. Well enough that you can mostly ignore that the other thing isn't really what it's designed around. The "mostly" is important, though. If you start hitting any extremes while using it in a different way, then you certainly can run into some pathological cases in the pricing structure and pay way too much for something that would have been cheaper in a service dedicated to that.

fxtentacle 6y ago
S3 is much cheaper than the egress traffic. As such, this isn't so much about S3 because you'd be paying the same price for that traffic if you were using EC2 instead of S3. Looking purely at the traffic price, I believe the word choice "overcharging" is warranted if you can buy the exact same product elsewhere for 90% less.
raiyu 6y ago
They over charge. Their cost is less than .01 cents per gb and look at their published pricing for customers. And I’m including networking equipment as well as head count in my cost, not just the cost of pipes.
- cortesoft 6y ago
  They charge so much for it as a disincentive to use it for CDN purposes. It is not architected to act as a CDN for people, and if they don't overcharge for egress, people will use it as a CDN because it is easy.

imtringued 6y ago

This is part of AWS's business strategy. Overcharge to an absurd extent (90% margins) and then if a surprise bill happens you can just refund it for 10% of the cost. This cost is a rounding error if it allows you to maintain your margins. It's worth it even if only a single person is willing to pay the regular rate.