10 ms·

AWS gives you all the things you'd need to scale, without heavy up-front costs. There's a natural path from small instances -> bigger instances -> load balancer

AWS gives you all the things you'd need to scale, without heavy up-front costs. There's a natural path from small instances -> bigger instances -> load balancers/ELB -> reserved instances (or spot if it fits your workload). For a smaller company, any savings you'd get from owned servers would be offset by much higher dev ops costs. Plus, as mentioned, you get a vast menu of services to choose from, all of which are managed. Just in terms of databases alone, you could go with RDS (raw MySql/PG), Aurora, Dynamo, Elasticache/Redis, DocumentDB (Mongo) and more. Plus managed backups, failover, multi-AZ, security, and connectivity built-in. If your team is filled with devops engineers, then, sure, go with the non-AWS route. But it's a lifesaver for agile companies who can't afford a lot of infrastructure work and need easy levers to scale, and can afford a modest markup.

tw04 6y ago
>can afford a modest markup. I'm with you until that statement. AWS is nothing approaching "modest" in their markup. 20% minimum and typically much higher if you know how to negotiate when purchasing your on-prem gear. And if you happen to be a shop that sweats assets for 7+ years that number starts being measured in the hundreds of percentage points more expensive.
- whatshisface 6y ago
  >And if you happen to be a shop that sweats assets for 7+ years that number starts being measured in the hundreds of percentage points more expensive. If you include the growth in electric efficiency over time, it can be bad to run old servers after a certain point.
- nixgeek 6y ago
  But again, is that comparable? Seems like apples versus oranges. What’s the connectivity into your 7+ year old asset cost? What about the space, power and cooling to provide it a home? What happens when your environment gets attacked by a volumetric DDOS, is that 100Mbps circuit going to get overwhelmed? I’m not arguing AWS is cheaper, simply that there are lots of things around the physical server which also have costs and complexities.
  - tw04 6y ago
    >What’s the connectivity into your 7+ year old asset cost? What about the space, power and cooling to provide it a home? 10Gbe ethernet was standard in 2013/2014. Space power and cooling is company dependent. If they're tiny, a closet in the building or a colocation facility, if they're larger, a room. If they're huge, a datacenter. >What happens when your environment gets attacked by a volumetric DDOS, is that 100Mbps circuit going to get overwhelmed? I can't recall the last time a customer got hit by a "volumetric DDoS". End-user facing systems sit behind cloudflare. Internal apps are... internal. Who cares if the WAN connection is getting hit by DDoS when your users are on campus? In post-covid world, they've got multiple tunnels with SD-WAN. That's ignoring that fact that Centurylink or Comcast or Telia are pretty good at mitigating DDoS in 2020. Just because they don't post about the attacks they endure on a blog doesn't mean they're inept. https://business.comcast.com/help-and-support/ethernet/comcast-business-ethernet-threat-management-service/ https://business.comcast.com/help-and-support/ethernet/comca... https://assets.centurylink.com/is/content/centurylink/centurylink-ddos-mitigation-service-data-sheet-1 https://assets.centurylink.com/is/content/centurylink/centur... >I’m not arguing AWS is cheaper, simply that there are lots of things around the physical server which also have costs and complexities. Sure, and they're all complexities which we solved 30+ years ago.
    - throwaway894345 6y ago
      > Sure, and they're all complexities which we solved 30+ years ago. The argument isn't that they are solved or not, but the cost of implementing and maintaining them vs AWS.
      - kikimora 6y ago
        I never say a whop running on AWS and not having a person managing it. Said person always has enough skill to replicate same setup on a dedicated hardware/VPS. I don't say such shops does not exists, it just I never saw them. To me AWS killer features are RDS and ability to provision 100 servers in a few minutes. Most of the time scaling is not needed and RDS alone cannot justify the cost of AWS.
- drchopchop 6y ago
  For a lot of companies, 20% is an acceptable markup to pay, in return for not having to deal with hardware failures, delays due to ordering new hardware, not always having instant failovers, over-purchasing for capacity/scale, the extra human dev ops costs, etc. If you're well-positioned to handle that, then great, AWS may not be cost-effective for you.
- nemothekid 6y ago
  >20% minimum and typically much higher if you know how to negotiate when purchasing your on-prem gear. A 20% premium on a $10k/mo server bill is an incredibly small premium to pay versus a single FTE.
  - tw04 6y ago
    I have yet to move a single customer to Amazon that fired the FTE who was managing on-prem infrastructure and chalked it up to a cost-savings move. . The idea of cutting headcount is a fallacy. In small shops the guy that was managing on-prem infrastructure is now doing that (because it turns out you still need switches and phones and routers and firewalls even if you host things in AWS) as well as managing the AWS infrastructure. In large shops you're typically replacing "cheap" headcount (the guy who racked servers) with someone significantly more expensive (a team of cloud architects).
    - x0x0 6y ago
      We're a small company who avoided the hire. And we don't bother with a firewall at our office -- the only things on the office network are laptops, phones, printers, and sonos. Basically, if you model senior eng time as $5k/week -- not even counting the opportunity cost -- you'll understand why AWS. While I definitely think AWS is very expensive for the servers, they are not overall expensive. Again, setting up CloudWatch in an afternoon or dumping tons of heterogenous logs into S3 and querying them with Athena or spinning up full environment stacks (front end pool, back end pool, load balancers, databases, vpcs, vpns, etc) is an enormous cost savings vs finding, installing, configuring, and maintaining batches of software to do the same. edit: pay an eng $180k. Fully loaded, you're looking at $220k plus. Add in either recruiting time or a recruiting fee that probably runs us $2k/mo amortized over the first year on the low end. Add in the fact that the first 6 weeks weren't super productive, and also absorbed tons of time from other SE to get this person up to speed. Add in PM time bringing this person up to speed on our product. Yup, $5k/week.
      - hobby-coder-guy 6y ago
        $5k per week?
- GordonS 6y ago
  And on bandwidth their markup tends towards infinity - same on Azure and GCP. As a example, 20TB of egress bandwidth will cost you around $1,750 on Azure, almost $90/TB! Your typical server/VPS provider gives you that kind of bandwidth allowance for free, because it costs them close to nothing. I have a feeling almost all complaints about cloud costs would disappear forever if they'd only stop gouging customers on bandwidth. The likes of Hetzner and OVH are starting to offer more cloudy features, like load balancers and block storage. My hope is that eventually they become viable alternatives to the big 3, and they are finally forced to reduce their ludicrous bandwidth prices.
  - hyperdimension 6y ago
    The term 'roach motel' comes to mind when I see their network traffic rates. "Come on in; there's no commitment. Pay as you go, and did I mention our ingress rates are free! Bring us your data and give the cloud a try. It's the future, you know..." "Oh you're leaving and want your data back now?"
hw 6y ago
> AWS gives you all the things you'd need to scale, without heavy up-front costs A startup doesn't need AWS right off the bat. Planning for scale right from the beginning is a way to quickly bleed $. Of course, if you have VC money, why not spend that cash right? Where I work we've started non-AWS and have continued non-AWS. We don't have a team of devops engineers, but rather a team where there are a engineers who _can_ do devops. I dread the day we need to move to AWS, but it's much easier moving to AWS than off it.
- renewiltord 6y ago
  Yeah but that's what capex based stuff (like buying and colocating) is: planning for scale. With AWS you think you need a robust DB and shit and you've made a decision that can be undone in minutes. You think you want Elasticache? It's a fifteen minute operation. In that time your devops guy isn't going to have even downloaded the binary and figured out the docs.
  - HelloNurse 6y ago
    > With AWS you think you need a robust DB and shit and you've made a decision that can be undone in minutes. If it can be undone in minutes, it isn't much of a decision. A service that can be enabled or canceled on a whim is unnecessary. Realistically, analyzing the guarantees offered by a cloud platform, your corresponding requirements, and how everything is supposed to work is going to take days, and actually developing and testing disaster recovery procedures is going to take even longer.
  - dx034 6y ago
    As I wrote further up, there's not just colocation and AWS. Dedicated servers are a great fit for many start ups and need no capex. You just rent hardware monthly like you lease most of your office equipment. Much cheaper than AWS, no hardware expertise needed and even (manual) scaling over time works quite well, it's easy to add servers within a day or so. Sure, you'll always have idle capacity, but this way you could use it. With AWS, Amazon runs that idle capacity and charges you for it.
- drchopchop 6y ago
wazoox 6y ago
Then you remember that for years, Stack Overflow ran out of a couple of well administered servers. YAGNI. KISS. People forget the basics because "infrastructure astronautics" is fun, and it probably helps make a beautiful resume, too.
- throwaway894345 6y ago
  I would guess SO had a relatively small number of employees and their servers were running a pretty straight-forward CRUD app and a database. Comparing that to the heterogeneous workloads that large organizations deal with is a bit silly. No doubt there's still a lot of fat to trim in those large organizations, but trimming that fat is rarely their largest opportunity (much to the chagrin of those of us who like simple, elegant systems).
  - ksec 6y ago
    It is the World top 50 site by Alexa. Comparatively speaking even compare them to the same CRUD app and DB, most are using 5 to 10x more servers with 1/2 to 1/5 of the traffic.
    - throwaway894345 6y ago
      > Comparatively speaking even compare them to the same CRUD app and DB, most are using 5 to 10x more servers with 1/2 to 1/5 of the traffic. No doubt, but how large are those other CRUD app organizations? Do they have a staff that is 20x the size all trying to coordinate deployments on the same handful of servers? What are their opportunity costs? Is trimming down server expenses really their most valuable opportunity? No doubt that SO has a great Ops capability, but it's not the only variable at play.
    - notyourwork 6y ago
      It is a large site by traffic measure but I would guess the traffic is heavily read only. Managing workloads with more data mutation introduces different complexities which mean you can't just cache everything and accept the TTL for writes based on cache invalidation. edit: To be clear, not saying SO isn't an achievement, but its one type of use case that yields a really simple tech stack.
thomaslord 6y ago
> If your team is filled with devops engineers, then, sure, go with the non-AWS route. This seems backwards to me - running a simple VPS on something like OVH/DigitalOcean/Linode is a matter of creating the instance, setting the size, and setting up your server software. Super simple, and likely about the same complexity as setting up a dev environment. Setting the same thing up on AWS requires slogging through the documentation of their many services, deciding what makes the most sense for you (EC2? ECS? EKS? Lambda? Fargate?), and then configuring more options with less-friendly UI than any other provider I've seen. If you can fully wrap your head around all those services, create a stack that makes sense, and maintain it long-term, you're probably well on your way to being qualified for a DevOps engineer job. I can spin up a quick DigitalOcean droplet for a new project in less than half the time it'd take me to set up an EC2 instance, configure the exact amount of storage I need, and pull up 2 additional browser windows to determine the specs and pricing of different instance sizes in different regions.
- mwcampbell 6y ago
  > running a simple VPS on something like OVH/DigitalOcean/Linode is a matter of creating the instance, setting the size, and setting up your server software. Super simple, and likely about the same complexity as setting up a dev environment. Until that server goes down.
  - rcxdude 6y ago
    I'd say the risks of a server going down are about the same as an AWS setup going haywire at that level.
  - eli 6y ago
    Sure but sometimes that’s ok. Everything goes down sometimes. You need to figure out how much effort you’re willing to expend for each extra “9”. I wonder what percentage of all sites hosted at AWS are actually ready for a zone to fail.
  - lossolo 6y ago
    # uptime 01:43:36 up 666 days, 11:42, 1 user, load average: 0.00, 0.03, 0.01 From OVH VPS. Funny thing it's 666 today. YMMV # uptime 01:47:03 up 640 days, 6:53, 2 users, load average: 2.39, 2.62, 2.73 From hetzner dedicated server. YMMV # uptime 01:48:11 up 482 days, 20 min, 2 users, load average: 1.53, 2.10, 2.78 From leaseweb dedicated server. YMMV
suifbwish 6y ago
As someone who has built out over 30 different accounts in AWS for various clients, this is definitely not entirely accurate. Elasticache is a caching mechanism not a database. Anyone who thinks they are the same has not used them extensively. Also RDS and Aurora both have had their super privileges removed so you might have to possibly overhaul some of your code base just to make it work. I’m sensing some uninformed fan base propaganda here

tw04 6y ago

>can afford a modest markup. I'm with you until that statement. AWS is nothing approaching "modest" in their markup. 20% minimum and typically much higher if you know how to negotiate when purchasing your on-prem gear. And if you happen to be a shop that sweats assets for 7+ years that number starts being measured in the hundreds of percentage points more expensive.

whatshisface 6y ago
>And if you happen to be a shop that sweats assets for 7+ years that number starts being measured in the hundreds of percentage points more expensive. If you include the growth in electric efficiency over time, it can be bad to run old servers after a certain point.
nixgeek 6y ago
But again, is that comparable? Seems like apples versus oranges. What’s the connectivity into your 7+ year old asset cost? What about the space, power and cooling to provide it a home? What happens when your environment gets attacked by a volumetric DDOS, is that 100Mbps circuit going to get overwhelmed? I’m not arguing AWS is cheaper, simply that there are lots of things around the physical server which also have costs and complexities.
- tw04 6y ago
  >What’s the connectivity into your 7+ year old asset cost? What about the space, power and cooling to provide it a home? 10Gbe ethernet was standard in 2013/2014. Space power and cooling is company dependent. If they're tiny, a closet in the building or a colocation facility, if they're larger, a room. If they're huge, a datacenter. >What happens when your environment gets attacked by a volumetric DDOS, is that 100Mbps circuit going to get overwhelmed? I can't recall the last time a customer got hit by a "volumetric DDoS". End-user facing systems sit behind cloudflare. Internal apps are... internal. Who cares if the WAN connection is getting hit by DDoS when your users are on campus? In post-covid world, they've got multiple tunnels with SD-WAN. That's ignoring that fact that Centurylink or Comcast or Telia are pretty good at mitigating DDoS in 2020. Just because they don't post about the attacks they endure on a blog doesn't mean they're inept. https://business.comcast.com/help-and-support/ethernet/comcast-business-ethernet-threat-management-service/ https://business.comcast.com/help-and-support/ethernet/comca... https://assets.centurylink.com/is/content/centurylink/centurylink-ddos-mitigation-service-data-sheet-1 https://assets.centurylink.com/is/content/centurylink/centur... >I’m not arguing AWS is cheaper, simply that there are lots of things around the physical server which also have costs and complexities. Sure, and they're all complexities which we solved 30+ years ago.
  - throwaway894345 6y ago
    > Sure, and they're all complexities which we solved 30+ years ago. The argument isn't that they are solved or not, but the cost of implementing and maintaining them vs AWS.
    - kikimora 6y ago
      I never say a whop running on AWS and not having a person managing it. Said person always has enough skill to replicate same setup on a dedicated hardware/VPS. I don't say such shops does not exists, it just I never saw them. To me AWS killer features are RDS and ability to provision 100 servers in a few minutes. Most of the time scaling is not needed and RDS alone cannot justify the cost of AWS.
drchopchop 6y ago
For a lot of companies, 20% is an acceptable markup to pay, in return for not having to deal with hardware failures, delays due to ordering new hardware, not always having instant failovers, over-purchasing for capacity/scale, the extra human dev ops costs, etc. If you're well-positioned to handle that, then great, AWS may not be cost-effective for you.
nemothekid 6y ago
>20% minimum and typically much higher if you know how to negotiate when purchasing your on-prem gear. A 20% premium on a $10k/mo server bill is an incredibly small premium to pay versus a single FTE.
- tw04 6y ago
  I have yet to move a single customer to Amazon that fired the FTE who was managing on-prem infrastructure and chalked it up to a cost-savings move. . The idea of cutting headcount is a fallacy. In small shops the guy that was managing on-prem infrastructure is now doing that (because it turns out you still need switches and phones and routers and firewalls even if you host things in AWS) as well as managing the AWS infrastructure. In large shops you're typically replacing "cheap" headcount (the guy who racked servers) with someone significantly more expensive (a team of cloud architects).
  - x0x0 6y ago
    We're a small company who avoided the hire. And we don't bother with a firewall at our office -- the only things on the office network are laptops, phones, printers, and sonos. Basically, if you model senior eng time as $5k/week -- not even counting the opportunity cost -- you'll understand why AWS. While I definitely think AWS is very expensive for the servers, they are not overall expensive. Again, setting up CloudWatch in an afternoon or dumping tons of heterogenous logs into S3 and querying them with Athena or spinning up full environment stacks (front end pool, back end pool, load balancers, databases, vpcs, vpns, etc) is an enormous cost savings vs finding, installing, configuring, and maintaining batches of software to do the same. edit: pay an eng $180k. Fully loaded, you're looking at $220k plus. Add in either recruiting time or a recruiting fee that probably runs us $2k/mo amortized over the first year on the low end. Add in the fact that the first 6 weeks weren't super productive, and also absorbed tons of time from other SE to get this person up to speed. Add in PM time bringing this person up to speed on our product. Yup, $5k/week.
    - hobby-coder-guy 6y ago
      $5k per week?
GordonS 6y ago
And on bandwidth their markup tends towards infinity - same on Azure and GCP. As a example, 20TB of egress bandwidth will cost you around $1,750 on Azure, almost $90/TB! Your typical server/VPS provider gives you that kind of bandwidth allowance for free, because it costs them close to nothing. I have a feeling almost all complaints about cloud costs would disappear forever if they'd only stop gouging customers on bandwidth. The likes of Hetzner and OVH are starting to offer more cloudy features, like load balancers and block storage. My hope is that eventually they become viable alternatives to the big 3, and they are finally forced to reduce their ludicrous bandwidth prices.
- hyperdimension 6y ago
  The term 'roach motel' comes to mind when I see their network traffic rates. "Come on in; there's no commitment. Pay as you go, and did I mention our ingress rates are free! Bring us your data and give the cloud a try. It's the future, you know..." "Oh you're leaving and want your data back now?"

hw 6y ago

> AWS gives you all the things you'd need to scale, without heavy up-front costs A startup doesn't need AWS right off the bat. Planning for scale right from the beginning is a way to quickly bleed $. Of course, if you have VC money, why not spend that cash right? Where I work we've started non-AWS and have continued non-AWS. We don't have a team of devops engineers, but rather a team where there are a engineers who _can_ do devops. I dread the day we need to move to AWS, but it's much easier moving to AWS than off it.

renewiltord 6y ago
Yeah but that's what capex based stuff (like buying and colocating) is: planning for scale. With AWS you think you need a robust DB and shit and you've made a decision that can be undone in minutes. You think you want Elasticache? It's a fifteen minute operation. In that time your devops guy isn't going to have even downloaded the binary and figured out the docs.
- HelloNurse 6y ago
  > With AWS you think you need a robust DB and shit and you've made a decision that can be undone in minutes. If it can be undone in minutes, it isn't much of a decision. A service that can be enabled or canceled on a whim is unnecessary. Realistically, analyzing the guarantees offered by a cloud platform, your corresponding requirements, and how everything is supposed to work is going to take days, and actually developing and testing disaster recovery procedures is going to take even longer.
- dx034 6y ago
  As I wrote further up, there's not just colocation and AWS. Dedicated servers are a great fit for many start ups and need no capex. You just rent hardware monthly like you lease most of your office equipment. Much cheaper than AWS, no hardware expertise needed and even (manual) scaling over time works quite well, it's easy to add servers within a day or so. Sure, you'll always have idle capacity, but this way you could use it. With AWS, Amazon runs that idle capacity and charges you for it.
drchopchop 6y ago

wazoox 6y ago

Then you remember that for years, Stack Overflow ran out of a couple of well administered servers. YAGNI. KISS. People forget the basics because "infrastructure astronautics" is fun, and it probably helps make a beautiful resume, too.

throwaway894345 6y ago
I would guess SO had a relatively small number of employees and their servers were running a pretty straight-forward CRUD app and a database. Comparing that to the heterogeneous workloads that large organizations deal with is a bit silly. No doubt there's still a lot of fat to trim in those large organizations, but trimming that fat is rarely their largest opportunity (much to the chagrin of those of us who like simple, elegant systems).
- ksec 6y ago
  It is the World top 50 site by Alexa. Comparatively speaking even compare them to the same CRUD app and DB, most are using 5 to 10x more servers with 1/2 to 1/5 of the traffic.
  - throwaway894345 6y ago
    > Comparatively speaking even compare them to the same CRUD app and DB, most are using 5 to 10x more servers with 1/2 to 1/5 of the traffic. No doubt, but how large are those other CRUD app organizations? Do they have a staff that is 20x the size all trying to coordinate deployments on the same handful of servers? What are their opportunity costs? Is trimming down server expenses really their most valuable opportunity? No doubt that SO has a great Ops capability, but it's not the only variable at play.
  - notyourwork 6y ago
    It is a large site by traffic measure but I would guess the traffic is heavily read only. Managing workloads with more data mutation introduces different complexities which mean you can't just cache everything and accept the TTL for writes based on cache invalidation. edit: To be clear, not saying SO isn't an achievement, but its one type of use case that yields a really simple tech stack.

thomaslord 6y ago

> If your team is filled with devops engineers, then, sure, go with the non-AWS route. This seems backwards to me - running a simple VPS on something like OVH/DigitalOcean/Linode is a matter of creating the instance, setting the size, and setting up your server software. Super simple, and likely about the same complexity as setting up a dev environment. Setting the same thing up on AWS requires slogging through the documentation of their many services, deciding what makes the most sense for you (EC2? ECS? EKS? Lambda? Fargate?), and then configuring more options with less-friendly UI than any other provider I've seen. If you can fully wrap your head around all those services, create a stack that makes sense, and maintain it long-term, you're probably well on your way to being qualified for a DevOps engineer job. I can spin up a quick DigitalOcean droplet for a new project in less than half the time it'd take me to set up an EC2 instance, configure the exact amount of storage I need, and pull up 2 additional browser windows to determine the specs and pricing of different instance sizes in different regions.

mwcampbell 6y ago
> running a simple VPS on something like OVH/DigitalOcean/Linode is a matter of creating the instance, setting the size, and setting up your server software. Super simple, and likely about the same complexity as setting up a dev environment. Until that server goes down.
- rcxdude 6y ago
  I'd say the risks of a server going down are about the same as an AWS setup going haywire at that level.
- eli 6y ago
  Sure but sometimes that’s ok. Everything goes down sometimes. You need to figure out how much effort you’re willing to expend for each extra “9”. I wonder what percentage of all sites hosted at AWS are actually ready for a zone to fail.
- lossolo 6y ago
  # uptime 01:43:36 up 666 days, 11:42, 1 user, load average: 0.00, 0.03, 0.01 From OVH VPS. Funny thing it's 666 today. YMMV # uptime 01:47:03 up 640 days, 6:53, 2 users, load average: 2.39, 2.62, 2.73 From hetzner dedicated server. YMMV # uptime 01:48:11 up 482 days, 20 min, 2 users, load average: 1.53, 2.10, 2.78 From leaseweb dedicated server. YMMV

suifbwish 6y ago

As someone who has built out over 30 different accounts in AWS for various clients, this is definitely not entirely accurate. Elasticache is a caching mechanism not a database. Anyone who thinks they are the same has not used them extensively. Also RDS and Aurora both have had their super privileges removed so you might have to possibly overhaul some of your code base just to make it work. I’m sensing some uninformed fan base propaganda here