6 ms·

>AWS isn't a scam on companies too poor to buy servers. Do the math and see how using AWS just doesn't make sense at the scale most companies operate.

mavhc 5y ago
Most companies have 1 server
- qayxc 5y ago
  Iron is cheap nowadays. It's the upkeep that kills you. This "1 server" is equivalent to at least two part-time admins to ensure smooth operations. No biggie if said server just hosts a static website, big deal if runs mission critical processes. That's the part too many "self-hosting" advocates conveniently leave out. A correct TCO calculation must include this cost (and proper risk assessment) and only then can a comparison be made.
  - owl57 5y ago
    And, if instead of a dedicated server you rent some combination of AWS services, you suddenly don't need admin or need a cheaper one?
    - iso1210 5y ago
      You just need an extra developer who's an expert with various AWS services to make sure you're using them correctly However as all developers are AWS experts (at least according to their resume), you have to pay that price regardless
    - qayxc 5y ago
      With services in particular you indeed don't need one. No need to ensure the software is up-to-date, no worries about hardware outages, no need to monitor the system, etc. etc. The person working with a rented service infrastructure can focus on what makes them money: the process. With self-owned hardware you need contingency plans, setup, maintenance, redundant ISPs, UPS, extra electrical setup (many offices automatically turn off power to all outlets after office hours to safe electricity and reduce the risk of fires), and much more depending on the required tasks and their importance. If you want to know what happens if you cheap out on that because "it just works" anyway once it's been set up and operated by an underqualified and overworked person, look no further than the recent ransomware incidents. Some of them could've been easily avoided had the IT infrastructure been up to snuff. Way too many businesses and public entities still treat IT as a necessary evil that can be handled casually, not realising their dependence on it. The specifics differ on a case-to-case basis, but in general it's a terrible idea for a small non-IT business to operate their own IT infrastructure. The solution doesn't have to be "cloud" or SaaS either. Hiring a competent 3rd party to deal with the specific implementation and operation for you is just as good even said party is "Bob's Business IT Services" and not AWS or Microsoft Azure.
      - deathanatos 5y ago
        > With services in particular you indeed don't need one. > No need to ensure the software is up-to-date, no worries about hardware outages, no need to monitor the system, etc. etc. > The person working with a rented service infrastructure can focus on what makes them money: the process. No, that's just not even remotely true. VMs, managed services all require updates that are pushed to the end user of the service. Downtimes happen, and dealing with those are pushed to the end user of the service by cloud providers far too often. I have countless support tickets at all levels of the cloud stack of back-and-forth on issues, many as simple as "how about serving 200s, not 500s?" I wish managed services were a rainbows and unicorns world of "I give you cash, you give me magic computer thing that scales and never goes down" … but that just isn't the case today for many cloud services. (I think about the one service I'd be willing to pin such a medal on is S3. It comes remarkably close, but even there, I've had an incident where we had an outage due to S3 that took much back & forth with support to finally get an engineer on their side to look at it.)
        qayxc 5y ago
        I think we're talking about very different things here. When I say "No need to ensure the software is up-to-date, no worries about hardware outages, no need to monitor the system, etc. etc." I don't mean these things don't need to happen at some point. The idea is that it's not you, as in the customer, who's responsible for this. If I book a Redis-DB of a given size, I don't have to care about its software version, proxies, what OS it's running on, how big the VM is, what the specs are, and so forth. That's what I'm getting at. It's far higher level than individual VMs and it doesn't mean downtimes don't happen either. It simply means it's not you, the customer, who has to troubleshoot and fix them.
        deathanatos 5y ago
        > If I book a Redis-DB of a given size, I don't have to care about its software version, proxies, what OS it's running on, how big the VM is, what the specs are, and so forth. That's what I'm getting at. And what I'm getting at is that you do. While there might exist managed services that don't require this, many managed services nonetheless do. E.g., AWS RDS regularly has updated versions, and it is somewhat up to the customer to schedule them & apply them, and downtime happens during that. RDS is relatively pain-free, and is higher on my list of "would use again" for that reason. But take something like Azure AKS: it's a "managed" service, but upgrades are very much not pain free. I have to manually schedule them, and manually apply them, and deal with fallout (which is often, IME). > It simply means it's not you, the customer, who has to troubleshoot and fix them. Again, I disagree. When S3 had its outage, I had to debug why certain requests weren't happening, I had to capture sufficient request IDs to prove to AWS that S3 was in an outage for us, and only then did we get the proper support, and eventually, attention from engineering. My company had to pay an engineer to do that. Sure, I'm not literally debugging S3 actual, but that doesn't mean an engineer isn't needed on the consuming side. Same story with an example with ACR where we were receiving 500s: it took not only capturing 5xx requests & their time of occurrence, it took multiple back and forths to convince Azure that the 5xxs were on their side. (Both of these incidents also demonstrated a real fundamental lack of alerting on the side of the companies managing these services. Our monitors on their services very clearly indicated outage. S3's excuse was that it was "just" our bucket. But the point is that they have no insight into the actual service the end user is perceiving even after the customer raises the support ticket.) Just recently I was SSHed onto an AKS VM to try to help Azure understand why packets weren't making it from their Virtual network into AKS. Data, from one Azure service to another! (And we didn't figure it out, either; we ended up changing what we were doing anyways, and so the investigation became moot.) And this demonstrates another issue I've hit, a bunch, is that interactions between managed services are also impossible to debug: you end up with two separate services both claiming it isn't them that is out of spec, but nonetheless the end result doesn't work, and nobody on either side has the competence to actually debug the problem. If you want the downtime to end, it becomes you, the customer that has to debug & troubleshoot these things. And that is a cost that must be accounted for when weighing managed services vs. unmanaged ones.
    - WJW 5y ago
      You don't exactly need to be an expert in object storage to stick a file into S3 and download it again some time later, just as 99% of SQL queries don't need a full blown DBA to write efficiently. (In fact, most queries are so simple that most shops have automated them with ORMs and the like) It is always the 1% of difficult stuff that hits you, but it is very rarely the correct call to optimize for that 1% instead of for the 99% of uses.
      - owl57 5y ago
        Yes? Neither you need to be an expert in file systems to stick a file on a server. And, yes, 99% of the time you don't need a DBA, just install Postgres on said server and go ahead with ORM.
        qayxc 5y ago
        Unless that file is to be read by 1000 users simultaneously - that's where the fun begins ;)
        owl57 5y ago
        Agree, when you have thousands of simultaneous users, it could be time to think what the hell are you doing. But the number sounds a bit off… I have fuzzy memories of some fun around that many concurrent requests with Apache 1.3 — that is, one process per user. Doesn't the fun begin quite a bit higher with just any more modern software?
        qayxc 5y ago
        > Doesn't the fun begin quite a bit higher with just any more modern software? You tell me, I'm not a sysadmin:) I can only attest to the fact that without any kind of RAID or suitable hardware setup (RAM for caching, fast networking etc.), even a dozen people accessing the same file can be very problematic (provided the file is large enough). As for the use case: digital content creation comes to mind. Sharing assets globally especially in times like these when more people are working remotely is such use case.
    - avereveard 5y ago
      a RDS instance is way less labor intensive that properly roll your own backed up, automatically updating server that can be snapshotted at your will and can be upgraded and restored with no downtime, I've spent about six day in five years of continuous operations to keep it running, and that'd probably be two dozen postgres upgrades I'd had to roll and test manually if I went the unmanaged route Now it isn't cheap, but the five year ownership cost difference compared to iron doesn't cover a month of my salary, so.
  - bsenftner 5y ago
    Maintaining a server is one of the big scary nothings that prevents many from learning the reality of how basic and elementary it actually is. Giving/having the process developers semi-knowledgeable about devOps helps significantly. But it is not hard, and not hard to learn from zero.
    - qayxc 5y ago
      We're talking about businesses here, not your private amusement, though. As a business you need to ask yourself: and who pays for the time it takes to educate at least two employees on this "basic and elementary matter"? If it really were so "basic and elementary", how come even critical infrastructure regularly falls victim to the most primitive of ransomware attacks? Misconfigured email and network access settings, vulnerable, out of date software stacks, non-existing contingency plans, lack of monitoring, no escalation strategy, etc. etc. Billions are lost each year due to this. It's very easy to underestimate the actual complexity of the task in its fullest whilst overestimating one's own abilities because "it's not hard to setup NGinx and service stack XY on a random x64 box". Anecdote from personal experience: tasked an applicant for a DevOps position with comparing two systems we could choose from for a backend service. The systems came in the form of installation packages and docker-images respectively. Having had no experience with Docker at the time, I thought it'd safe my team a great deal of time to have the "new guy", who knew Docker inside out, deal with it. A week later we had two results: 1) a usable comparison of the two systems and 2) an e-mail from the cloud service who informed us that the machines we rented were performing DDoS attacks on a foreign company... New guy had forgotten to properly secure the test machines he had set up and it took hackers all of 72 hours to find and hijack them. Luckily it only took a few clicks to wipe the offending boxes, but it served as a warning for what happens if you take the whole "big scary nothings" too lightly.
      - bsenftner 5y ago
        It certainly helps if one begins with experienced engineers. I am not talking about personal projects, I am discussing funded startups. If your team has not run servers on their own in the past, get a different team. It is not as if there is not a giant population of grey beards with this knowledge seeking work... I accept your points, but far too many inexperienced devOps are given keys that should not have them. From your Docker story, it sounds like poor decision making on your behalf. Just because a developer knows Docker does not mean they know how to expose that securely to the public - far too many devs believe that's not their job.
berkes 5y ago
Are you saying that if I did the math, I'd move my company to a more expensive heroku (I did), onto another, cheaper VPS provider (I did, Linode, DO, rackspace even, for other companies) or onto self hosted&maintained hardware (this hasn't been cost effective for me for decades)?
- turminal 5y ago
  There's a point between being too small to use up a single beefy server and being big enough to go cloud scale, where colocating a few servers and maintaining them makes sense financially in most scenarios. For companies smaller than that renting a cheaper VPS probably makes more sense.
- bsenftner 5y ago
  If you're any form of hi-compute, the cloud is insanely expensive and a no-brainer for colocation, even for as little as one server. If your service requires more than the average CRUD serving of media, the cloud quickly becomes far far too expensive.
  - whoknew1122 5y ago
    It's all fun and games until that co-lo goes down and with it your entire business. Speaking from someone who moved a SaaS operation from co-lo to AWS. Everyone (especially the C-suite) was much happier.
    - adflux 5y ago
      Always fun to see people compare bare metal costs to actual running cost of cloud infrastructure. For some reason such advocates always think the system administrators work for free, the network will always work, the machine will never break, the machine will not need to be replicated etc etc etc
      - berkes 5y ago
        To drive home an argument a co-owner of an old hosting company I had, calculated the costs for replacing a single HDD. Everything from buying the HDD, to his hours to the parking ticket at the data-center. He convinced me and the point was clear: a single failing HDD would costs us more than renting similar computing power for over 6 years. We had two such failing HDDs in three years prior.
adflux 5y ago
>AWS just doesn't make sense at the scale most companies operate You are delusional if you think "most companies" operate at a scale at which they can support their own datacentres
- turminal 5y ago
  I am not delusional and colocation is a thing.

mavhc 5y ago

Most companies have 1 server

qayxc 5y ago
Iron is cheap nowadays. It's the upkeep that kills you. This "1 server" is equivalent to at least two part-time admins to ensure smooth operations. No biggie if said server just hosts a static website, big deal if runs mission critical processes. That's the part too many "self-hosting" advocates conveniently leave out. A correct TCO calculation must include this cost (and proper risk assessment) and only then can a comparison be made.
- owl57 5y ago
  And, if instead of a dedicated server you rent some combination of AWS services, you suddenly don't need admin or need a cheaper one?
  - iso1210 5y ago
    You just need an extra developer who's an expert with various AWS services to make sure you're using them correctly However as all developers are AWS experts (at least according to their resume), you have to pay that price regardless
  - qayxc 5y ago
    With services in particular you indeed don't need one. No need to ensure the software is up-to-date, no worries about hardware outages, no need to monitor the system, etc. etc. The person working with a rented service infrastructure can focus on what makes them money: the process. With self-owned hardware you need contingency plans, setup, maintenance, redundant ISPs, UPS, extra electrical setup (many offices automatically turn off power to all outlets after office hours to safe electricity and reduce the risk of fires), and much more depending on the required tasks and their importance. If you want to know what happens if you cheap out on that because "it just works" anyway once it's been set up and operated by an underqualified and overworked person, look no further than the recent ransomware incidents. Some of them could've been easily avoided had the IT infrastructure been up to snuff. Way too many businesses and public entities still treat IT as a necessary evil that can be handled casually, not realising their dependence on it. The specifics differ on a case-to-case basis, but in general it's a terrible idea for a small non-IT business to operate their own IT infrastructure. The solution doesn't have to be "cloud" or SaaS either. Hiring a competent 3rd party to deal with the specific implementation and operation for you is just as good even said party is "Bob's Business IT Services" and not AWS or Microsoft Azure.
    - deathanatos 5y ago
      > With services in particular you indeed don't need one. > No need to ensure the software is up-to-date, no worries about hardware outages, no need to monitor the system, etc. etc. > The person working with a rented service infrastructure can focus on what makes them money: the process. No, that's just not even remotely true. VMs, managed services all require updates that are pushed to the end user of the service. Downtimes happen, and dealing with those are pushed to the end user of the service by cloud providers far too often. I have countless support tickets at all levels of the cloud stack of back-and-forth on issues, many as simple as "how about serving 200s, not 500s?" I wish managed services were a rainbows and unicorns world of "I give you cash, you give me magic computer thing that scales and never goes down" … but that just isn't the case today for many cloud services. (I think about the one service I'd be willing to pin such a medal on is S3. It comes remarkably close, but even there, I've had an incident where we had an outage due to S3 that took much back & forth with support to finally get an engineer on their side to look at it.)
      - qayxc 5y ago
        I think we're talking about very different things here. When I say "No need to ensure the software is up-to-date, no worries about hardware outages, no need to monitor the system, etc. etc." I don't mean these things don't need to happen at some point. The idea is that it's not you, as in the customer, who's responsible for this. If I book a Redis-DB of a given size, I don't have to care about its software version, proxies, what OS it's running on, how big the VM is, what the specs are, and so forth. That's what I'm getting at. It's far higher level than individual VMs and it doesn't mean downtimes don't happen either. It simply means it's not you, the customer, who has to troubleshoot and fix them.
        deathanatos 5y ago
        > If I book a Redis-DB of a given size, I don't have to care about its software version, proxies, what OS it's running on, how big the VM is, what the specs are, and so forth. That's what I'm getting at. And what I'm getting at is that you do. While there might exist managed services that don't require this, many managed services nonetheless do. E.g., AWS RDS regularly has updated versions, and it is somewhat up to the customer to schedule them & apply them, and downtime happens during that. RDS is relatively pain-free, and is higher on my list of "would use again" for that reason. But take something like Azure AKS: it's a "managed" service, but upgrades are very much not pain free. I have to manually schedule them, and manually apply them, and deal with fallout (which is often, IME). > It simply means it's not you, the customer, who has to troubleshoot and fix them. Again, I disagree. When S3 had its outage, I had to debug why certain requests weren't happening, I had to capture sufficient request IDs to prove to AWS that S3 was in an outage for us, and only then did we get the proper support, and eventually, attention from engineering. My company had to pay an engineer to do that. Sure, I'm not literally debugging S3 actual, but that doesn't mean an engineer isn't needed on the consuming side. Same story with an example with ACR where we were receiving 500s: it took not only capturing 5xx requests & their time of occurrence, it took multiple back and forths to convince Azure that the 5xxs were on their side. (Both of these incidents also demonstrated a real fundamental lack of alerting on the side of the companies managing these services. Our monitors on their services very clearly indicated outage. S3's excuse was that it was "just" our bucket. But the point is that they have no insight into the actual service the end user is perceiving even after the customer raises the support ticket.) Just recently I was SSHed onto an AKS VM to try to help Azure understand why packets weren't making it from their Virtual network into AKS. Data, from one Azure service to another! (And we didn't figure it out, either; we ended up changing what we were doing anyways, and so the investigation became moot.) And this demonstrates another issue I've hit, a bunch, is that interactions between managed services are also impossible to debug: you end up with two separate services both claiming it isn't them that is out of spec, but nonetheless the end result doesn't work, and nobody on either side has the competence to actually debug the problem. If you want the downtime to end, it becomes you, the customer that has to debug & troubleshoot these things. And that is a cost that must be accounted for when weighing managed services vs. unmanaged ones.
  - WJW 5y ago
    You don't exactly need to be an expert in object storage to stick a file into S3 and download it again some time later, just as 99% of SQL queries don't need a full blown DBA to write efficiently. (In fact, most queries are so simple that most shops have automated them with ORMs and the like) It is always the 1% of difficult stuff that hits you, but it is very rarely the correct call to optimize for that 1% instead of for the 99% of uses.
    - owl57 5y ago
      Yes? Neither you need to be an expert in file systems to stick a file on a server. And, yes, 99% of the time you don't need a DBA, just install Postgres on said server and go ahead with ORM.
      - qayxc 5y ago
        Unless that file is to be read by 1000 users simultaneously - that's where the fun begins ;)
        owl57 5y ago
        Agree, when you have thousands of simultaneous users, it could be time to think what the hell are you doing. But the number sounds a bit off… I have fuzzy memories of some fun around that many concurrent requests with Apache 1.3 — that is, one process per user. Doesn't the fun begin quite a bit higher with just any more modern software?
        qayxc 5y ago
        > Doesn't the fun begin quite a bit higher with just any more modern software? You tell me, I'm not a sysadmin:) I can only attest to the fact that without any kind of RAID or suitable hardware setup (RAM for caching, fast networking etc.), even a dozen people accessing the same file can be very problematic (provided the file is large enough). As for the use case: digital content creation comes to mind. Sharing assets globally especially in times like these when more people are working remotely is such use case.
  - avereveard 5y ago
    a RDS instance is way less labor intensive that properly roll your own backed up, automatically updating server that can be snapshotted at your will and can be upgraded and restored with no downtime, I've spent about six day in five years of continuous operations to keep it running, and that'd probably be two dozen postgres upgrades I'd had to roll and test manually if I went the unmanaged route Now it isn't cheap, but the five year ownership cost difference compared to iron doesn't cover a month of my salary, so.
- bsenftner 5y ago
  Maintaining a server is one of the big scary nothings that prevents many from learning the reality of how basic and elementary it actually is. Giving/having the process developers semi-knowledgeable about devOps helps significantly. But it is not hard, and not hard to learn from zero.
  - qayxc 5y ago
    We're talking about businesses here, not your private amusement, though. As a business you need to ask yourself: and who pays for the time it takes to educate at least two employees on this "basic and elementary matter"? If it really were so "basic and elementary", how come even critical infrastructure regularly falls victim to the most primitive of ransomware attacks? Misconfigured email and network access settings, vulnerable, out of date software stacks, non-existing contingency plans, lack of monitoring, no escalation strategy, etc. etc. Billions are lost each year due to this. It's very easy to underestimate the actual complexity of the task in its fullest whilst overestimating one's own abilities because "it's not hard to setup NGinx and service stack XY on a random x64 box". Anecdote from personal experience: tasked an applicant for a DevOps position with comparing two systems we could choose from for a backend service. The systems came in the form of installation packages and docker-images respectively. Having had no experience with Docker at the time, I thought it'd safe my team a great deal of time to have the "new guy", who knew Docker inside out, deal with it. A week later we had two results: 1) a usable comparison of the two systems and 2) an e-mail from the cloud service who informed us that the machines we rented were performing DDoS attacks on a foreign company... New guy had forgotten to properly secure the test machines he had set up and it took hackers all of 72 hours to find and hijack them. Luckily it only took a few clicks to wipe the offending boxes, but it served as a warning for what happens if you take the whole "big scary nothings" too lightly.
    - bsenftner 5y ago
      It certainly helps if one begins with experienced engineers. I am not talking about personal projects, I am discussing funded startups. If your team has not run servers on their own in the past, get a different team. It is not as if there is not a giant population of grey beards with this knowledge seeking work... I accept your points, but far too many inexperienced devOps are given keys that should not have them. From your Docker story, it sounds like poor decision making on your behalf. Just because a developer knows Docker does not mean they know how to expose that securely to the public - far too many devs believe that's not their job.

berkes 5y ago

Are you saying that if I did the math, I'd move my company to a more expensive heroku (I did), onto another, cheaper VPS provider (I did, Linode, DO, rackspace even, for other companies) or onto self hosted&maintained hardware (this hasn't been cost effective for me for decades)?

turminal 5y ago
There's a point between being too small to use up a single beefy server and being big enough to go cloud scale, where colocating a few servers and maintaining them makes sense financially in most scenarios. For companies smaller than that renting a cheaper VPS probably makes more sense.
bsenftner 5y ago
If you're any form of hi-compute, the cloud is insanely expensive and a no-brainer for colocation, even for as little as one server. If your service requires more than the average CRUD serving of media, the cloud quickly becomes far far too expensive.
- whoknew1122 5y ago
  It's all fun and games until that co-lo goes down and with it your entire business. Speaking from someone who moved a SaaS operation from co-lo to AWS. Everyone (especially the C-suite) was much happier.
  - adflux 5y ago
    Always fun to see people compare bare metal costs to actual running cost of cloud infrastructure. For some reason such advocates always think the system administrators work for free, the network will always work, the machine will never break, the machine will not need to be replicated etc etc etc
    - berkes 5y ago
      To drive home an argument a co-owner of an old hosting company I had, calculated the costs for replacing a single HDD. Everything from buying the HDD, to his hours to the parking ticket at the data-center. He convinced me and the point was clear: a single failing HDD would costs us more than renting similar computing power for over 6 years. We had two such failing HDDs in three years prior.

adflux 5y ago

>AWS just doesn't make sense at the scale most companies operate You are delusional if you think "most companies" operate at a scale at which they can support their own datacentres

turminal 5y ago
I am not delusional and colocation is a thing.