4 ms·

I think people need to make a conscious tradeoff between ~5x AWS cost compared to rented server vs. OPS costs. Some startups (A) I work with have basically no

I think people need to make a conscious tradeoff between ~5x AWS cost compared to rented server vs. OPS costs. Some startups (A) I work with have basically no OPS costs beyond setup, integrating Docker deployements and getting automatic backup working. Most simple technology just works and devs easily can do operations. The largest pain point still is VPN. Machines today are very very fast and load of many startups is very low. This are mostly simple marketplace startups etc. without any rocket science aka "web ui frontend to database". They often have <10 servers and are still over provisioned mostly due to HA requirements. Some startups (B) I work with have high OPS costs due to demanding technology needs for throughput, load peaks, amount of data with innovative technology at their core, aka "rocket science". I have not seen any correlation between AWS usage and A/B types. From my experience with startups the only way to successfully use AWS is deep integration and using lots of services. If you use AWS and do everything on your own people are doing it wrong.

pizza234 9y ago
I managed a metal to AWS transition, and 5x definitely doesn't match the costs I experienced (in this specific case, it was around the lines of 1.2x). I don't mean this case to be universal, in particular, I think cloud services force applications to have a particularly good/modular design (which is a cost in itself) - where, with metal, as you wrote, you can relatively cheaply overprovision. I think the analysis you're making overlooks some important characteristics of the infrastructure engineering aspect. Some typical network/infrastructure elements, in particular firewalling, load balancing, and network management don't necessarily belong to the "rocket science" type of application; they are easy to overlook in "type A" services, ending with a "kind-of-HA-but-not-really" infrastructure, which is ok, but it makes the comparison cloud <> metal not really meaningful, as in the cloud, those features are baked in ("almost" for free). I'm very skeptical for example, that the 5x figure includes hardware for the above network equipment and management. To summarize, it's perfectly fine not to have an "advanced" infrastructure but it must be highlighted that such conditions make a direct comparison incorrect.
- vidarh 9y ago
  I think the "5x" (though my experience is closer to 2x-3x) tends to come in if you already do the good/modular design and are set up to scale into hybrid setups as needed to handle traffic spikes. In those instances you don't overprovision, or overprovision maybe 20%, knowing that extending your virtualized setup into a part on premises or manage hosting setup, part public cloud is fast and seamless. E.g. I had a setup that spanned on-demand instances, rented managed servers, racks in two separate colos and racks on premises. We expanded resources whenever it was cost effective at the time. Generally the colos won out, with on-demand instances handling traffic spikes, and managed servers primarily used for locations we did not have staff. When your infrastructure is designed so that adding a new one of any of those is just a matter of assigning IP space to the new satellite network and deploy the first instances - whatever they're physically on - your utilisation of all the resources can be far higher. E.g. in this setup we have instances where we move containers seamlessly between the UK, New Zealand and Germany currently depending on load, available resources, and which instances need low latency (Germany vs. UK makes a roughly 8ms latency difference despite going over an encrypted VPN connection, so we've even had times where client traffic hits load-balancers in the UK while the web servers were temporarily in Germany because it happened to be cheaper to expand there for a while (and contrary to with AWS, our bandwidth costs in both locations are trivial). If you're comparing to "lets throw a bunch of servers somewhere", then, yes, AWS probably won't be that much more expensive, and presumably that is a big part of why so many people gets caught out by AWS costs once they start scaling up.
- _Codemonkeyism 9y ago
  Disclaimer: My experience is with growing startups younger than 5 years. So everything I say needs to be seen in that context. From what did you transition to what? 20% sounds like a very small markup, from my experience and from other people experiences documented on the web it's much much larger (at 10x?) than your experience. So I would be rather interested in more details as many people ask me about Amazon transitioning, and 20% markup would be killer. "Some typical network/infrastructure elements, in particular firewalling, load balancing, and network management don't necessarily belong to the "rocket science" type of application;" I surely do not know your demands, but firewalling, load balanicng etc. looks rather easy to me today for everyone except Google, Amazon, LinkedIn, AirBnB and 99% of startups are not one of these. "I'm very skeptical for example, that the 5x figure includes hardware for the above network equipment and management." Not sure what you are using, the hardware would be around $40 per month for that kind of network architecture (FW,HAProxy,Nginx,...). In my last job we had large NetScalers which where much more powerful then HAProxy/Nginx on a rented server, and I assume AWS is as powerful, but for most of my clients this would be huge overkill.

pizza234 9y ago

I managed a metal to AWS transition, and 5x definitely doesn't match the costs I experienced (in this specific case, it was around the lines of 1.2x). I don't mean this case to be universal, in particular, I think cloud services force applications to have a particularly good/modular design (which is a cost in itself) - where, with metal, as you wrote, you can relatively cheaply overprovision. I think the analysis you're making overlooks some important characteristics of the infrastructure engineering aspect. Some typical network/infrastructure elements, in particular firewalling, load balancing, and network management don't necessarily belong to the "rocket science" type of application; they are easy to overlook in "type A" services, ending with a "kind-of-HA-but-not-really" infrastructure, which is ok, but it makes the comparison cloud <> metal not really meaningful, as in the cloud, those features are baked in ("almost" for free). I'm very skeptical for example, that the 5x figure includes hardware for the above network equipment and management. To summarize, it's perfectly fine not to have an "advanced" infrastructure but it must be highlighted that such conditions make a direct comparison incorrect.

vidarh 9y ago
I think the "5x" (though my experience is closer to 2x-3x) tends to come in if you already do the good/modular design and are set up to scale into hybrid setups as needed to handle traffic spikes. In those instances you don't overprovision, or overprovision maybe 20%, knowing that extending your virtualized setup into a part on premises or manage hosting setup, part public cloud is fast and seamless. E.g. I had a setup that spanned on-demand instances, rented managed servers, racks in two separate colos and racks on premises. We expanded resources whenever it was cost effective at the time. Generally the colos won out, with on-demand instances handling traffic spikes, and managed servers primarily used for locations we did not have staff. When your infrastructure is designed so that adding a new one of any of those is just a matter of assigning IP space to the new satellite network and deploy the first instances - whatever they're physically on - your utilisation of all the resources can be far higher. E.g. in this setup we have instances where we move containers seamlessly between the UK, New Zealand and Germany currently depending on load, available resources, and which instances need low latency (Germany vs. UK makes a roughly 8ms latency difference despite going over an encrypted VPN connection, so we've even had times where client traffic hits load-balancers in the UK while the web servers were temporarily in Germany because it happened to be cheaper to expand there for a while (and contrary to with AWS, our bandwidth costs in both locations are trivial). If you're comparing to "lets throw a bunch of servers somewhere", then, yes, AWS probably won't be that much more expensive, and presumably that is a big part of why so many people gets caught out by AWS costs once they start scaling up.
_Codemonkeyism 9y ago
Disclaimer: My experience is with growing startups younger than 5 years. So everything I say needs to be seen in that context. From what did you transition to what? 20% sounds like a very small markup, from my experience and from other people experiences documented on the web it's much much larger (at 10x?) than your experience. So I would be rather interested in more details as many people ask me about Amazon transitioning, and 20% markup would be killer. "Some typical network/infrastructure elements, in particular firewalling, load balancing, and network management don't necessarily belong to the "rocket science" type of application;" I surely do not know your demands, but firewalling, load balanicng etc. looks rather easy to me today for everyone except Google, Amazon, LinkedIn, AirBnB and 99% of startups are not one of these. "I'm very skeptical for example, that the 5x figure includes hardware for the above network equipment and management." Not sure what you are using, the hardware would be around $40 per month for that kind of network architecture (FW,HAProxy,Nginx,...). In my last job we had large NetScalers which where much more powerful then HAProxy/Nginx on a rented server, and I assume AWS is as powerful, but for most of my clients this would be huge overkill.