5 ms·

Does anyone know what the break-even point is for building/managing your own infrastructure, vs. using cloud services? I'm also curious - what's the rough time

Does anyone know what the break-even point is for building/managing your own infrastructure, vs. using cloud services? I'm also curious - what's the rough time/cost of moving to your own infrastructure? I'm sure it depends on lots of things, but looking for a ballpark. 1 year + 10 engineers? 5 years + 100 engineers?

bcrosby95 6y ago
Part of the problem is at smaller companies, managing your own infrastructure is just a part time job. So you have to outsource it to someone you trust, find someone in your company that wants to moonlight as a sysadmin for 10-20 minutes/day, or pay a full time employee to not do that much. How much time you have to spend is generally related to how many servers you have. In my experience, if you have automation setup correctly (1-2 weeks) you can average around 5 hours per month to manage about 30 servers.
gravypod 6y ago
Depends on what your use case is. For GPU compute, in my experience, it's pretty much 2 months of continuous usage.
hn_throwaway_99 6y ago
I mean, the more important question for me is, is running your own infrastructure something that should be a core competency. For Dropbox, the answer seems like an obvious yes. I mean, virtually their entire product is around data storage. I would be shocked if they COULDN'T save money to build a storage system tailored to their needs.
- baskire 6y ago
  It’s also a lot of cold storage, low iops, moderate throughput. Not high performance compute with random high iops. Of course it’s cheaper. Compare the cost of an enterprise NAS vs external hdd from bestbuy. It’s a similar comparison here
ebiester 6y ago
There are no shortcuts here. You really have to understand your use cases.
cashewchoo 6y ago
My take is that there's only a very specific kind of company where on-prem vs cloud is primarily a cost-driven decision. "Cloud" providers are so much more than just the hardware. There are probably a lot of AWS customers who could absolutely save lots of money if they took all their cloud servers and magically made them physical on-prem servers. But then they might hit a situation like: "We need to create a whole new prod-sized cluster for load testing but only for a day". Do you rack all those servers, do the test, then take them off and bin them? Or what? I'm tempted to go on and on with similar examples but imo the biggest deal with cloud providers is that they can operate at such an insane infra scale that it lets you treat extremely large quantities of servers like an abstraction instead of a physical metal box that needs to be bottle-fed and rocked to sleep at night lest it get cranky. I think at fairly small scales it's easy to build in-house systems that let you treat servers like abstract units as long as you're not trying to 2+x your infra dynamically. But cloud solutions let you do that because your scale <<< the cloud providers' scale even at pretty large values of scale for any one company.
jokethrowaway 6y ago
It depends hugely on your cost of staffing. Also, there is not only aws vs on prem. There are solutions in the middle where you don't have all of the AWS services but price is much lower. I've seen very average businesses spending 1mln per year in AWS for, really not much. IMHO most mid businesses would be better off renting servers and having some staff maintaining them.
toast0 6y ago
There's a bunch of steps along the way between AWS[1] and managing your own infrastructure. Virtual private servers, bare metal hosting, cabinet, rack, or cage in a colo, and then operating your own datacenters. The more you buy into the AWS services and not just EC2 instances, the harder it is to move out. In other parts of the thread, people are saying you need to have a good team to run your own infra, but you also need to have a good team to run on other people's infra, and debug their bugs without visibility, so that you can guide their techs to fixing your issues, so I don't think that using other people's services absolves you of having a good team. Really the issue isn't age of company or number of engineers. It's the number of servers you need, and how stable that is. If you can't predict your server count 3 months out, you need to host with someone who has stock on hand to buffer your growth. If your server count is small, you get better geographic redundancy picking up an instance here and there from around the world from a single vendor; colo space is available everywhere, of course, but you would likely be dealing with different vendors in each locality. If you can take advantage of growing and shrinking your deployment throughout the day in response to load, and there's a dramatic difference between peak and trough, it makes a lot of sense to be somewhere that you pay by the hour, instead of by the month or have to buy for the peak and let it idle. [1] Or Google Cloud, or Azure, or Oracle Cloud or whoever.

bcrosby95 6y ago

Part of the problem is at smaller companies, managing your own infrastructure is just a part time job. So you have to outsource it to someone you trust, find someone in your company that wants to moonlight as a sysadmin for 10-20 minutes/day, or pay a full time employee to not do that much. How much time you have to spend is generally related to how many servers you have. In my experience, if you have automation setup correctly (1-2 weeks) you can average around 5 hours per month to manage about 30 servers.

gravypod 6y ago

Depends on what your use case is. For GPU compute, in my experience, it's pretty much 2 months of continuous usage.

hn_throwaway_99 6y ago

I mean, the more important question for me is, is running your own infrastructure something that should be a core competency. For Dropbox, the answer seems like an obvious yes. I mean, virtually their entire product is around data storage. I would be shocked if they COULDN'T save money to build a storage system tailored to their needs.

baskire 6y ago
It’s also a lot of cold storage, low iops, moderate throughput. Not high performance compute with random high iops. Of course it’s cheaper. Compare the cost of an enterprise NAS vs external hdd from bestbuy. It’s a similar comparison here

ebiester 6y ago

There are no shortcuts here. You really have to understand your use cases.

cashewchoo 6y ago

My take is that there's only a very specific kind of company where on-prem vs cloud is primarily a cost-driven decision. "Cloud" providers are so much more than just the hardware. There are probably a lot of AWS customers who could absolutely save lots of money if they took all their cloud servers and magically made them physical on-prem servers. But then they might hit a situation like: "We need to create a whole new prod-sized cluster for load testing but only for a day". Do you rack all those servers, do the test, then take them off and bin them? Or what? I'm tempted to go on and on with similar examples but imo the biggest deal with cloud providers is that they can operate at such an insane infra scale that it lets you treat extremely large quantities of servers like an abstraction instead of a physical metal box that needs to be bottle-fed and rocked to sleep at night lest it get cranky. I think at fairly small scales it's easy to build in-house systems that let you treat servers like abstract units as long as you're not trying to 2+x your infra dynamically. But cloud solutions let you do that because your scale <<< the cloud providers' scale even at pretty large values of scale for any one company.

jokethrowaway 6y ago

It depends hugely on your cost of staffing. Also, there is not only aws vs on prem. There are solutions in the middle where you don't have all of the AWS services but price is much lower. I've seen very average businesses spending 1mln per year in AWS for, really not much. IMHO most mid businesses would be better off renting servers and having some staff maintaining them.

toast0 6y ago

There's a bunch of steps along the way between AWS[1] and managing your own infrastructure. Virtual private servers, bare metal hosting, cabinet, rack, or cage in a colo, and then operating your own datacenters. The more you buy into the AWS services and not just EC2 instances, the harder it is to move out. In other parts of the thread, people are saying you need to have a good team to run your own infra, but you also need to have a good team to run on other people's infra, and debug their bugs without visibility, so that you can guide their techs to fixing your issues, so I don't think that using other people's services absolves you of having a good team. Really the issue isn't age of company or number of engineers. It's the number of servers you need, and how stable that is. If you can't predict your server count 3 months out, you need to host with someone who has stock on hand to buffer your growth. If your server count is small, you get better geographic redundancy picking up an instance here and there from around the world from a single vendor; colo space is available everywhere, of course, but you would likely be dealing with different vendors in each locality. If you can take advantage of growing and shrinking your deployment throughout the day in response to load, and there's a dramatic difference between peak and trough, it makes a lot of sense to be somewhere that you pay by the hour, instead of by the month or have to buy for the peak and let it idle. [1] Or Google Cloud, or Azure, or Oracle Cloud or whoever.