5 ms·

The things you mention are ostensibly true, yet still they don't make sense to me. It might make sense when you're a startup that's growing, but when your SSD c

by tinco 5y ago

The things you mention are ostensibly true, yet still they don't make sense to me. It might make sense when you're a startup that's growing, but when your SSD costs are so large you can save millions on them, then the numbers just don't add up. In my experience, doing things in the cloud is about as expensive per 12-18 months as buying the hardware up front is. That's super interesting for a fast growing startup that could go bust any minute and wants to spend every second of their time on growing, expanding and marketing. But when you're spending so much on AWS you can save millions just by reducing filesystem overhead by 20%, it should have stopped making sense a while ago. $2 million should get you a team of 10 sysadmins and devops engineers. Sure automation would be more difficult, but you'd have the manpower to achieve it. Isn't that what running a business is about? Flexibility, when you're growing quickly it's nice that you can provision new hardware instantly, but AWS is so expensive you could continuously over provision your hardware by 50% and still always be ahead of the AWS price curve. And as I said, you could fully swap out your hardware every 18 months and be at the same price basically. You could even hire a merchant to offload your old hardware and recuperate 50% of those costs. And I'm not saying to throw AWS overboard altogether, that you have your core business outside of AWS's datacenter doesn't preclude you from buying into RDS, Kinesis, S3. Is AWS just cutting you more financial slack than we're getting as a tiny company? Or am I underestimating the costs of getting that sysadmin team on board?

zerd 5y ago
There's also opportunity cost. What if you instead of hiring a team of 10 people (probably need more, managers, distributed geographically etc), you hire more developers, marketing, sales and increase revenue. So instead of saving $2M, you make $3M more revenue. Making up numbers of course, but companies can estimate this. If that's the case it makes sense to keep paying AWS, until the ratio changes, and then they can cut costs.
gwright 5y ago
> In my experience, doing things in the cloud is about as expensive per 12-18 months as buying the hardware up front is. It seems like you are glossing over the other costs: staff to implement and manage, development time and maintenance for automation to re-implement everything that AWS includes, data center costs (not clear if you were thinking of hardware ownership only or data-center also). I'm not saying you didn't think about those things just saying that they can't be ignored in these types of comparisons.
- aflag 5y ago
  You don’t need to reimplement everything aws provides. AWS provides services on demand for a huge costumer base. The sysadmin/devops team you set up needs to solve only your particular problem. You can often get a better, easier to use and maintain system this way. The downside is that you have to pay for that extra team, but if only 20% of your data already costs millions, your scale is big enough that you’ll likely save money by hiring a sys admin team.
  - lazide 5y ago
    Sometimes, sometimes not. A lot of common ‘prod bricks’ (S3, Managed kubernetes, etc) get used because they are convenient, and while they could be implemented in some other, more bespoke way, it’s rarer and rarer that it actually pencils out as a net win. You also have to deal with the complexity of managing your own version of it, which is non trivial over the full lifecycle of something. If it is your core business to provide that thing? Sometimes or even often worth it. Otherwise, often not.
    - manquer 5y ago
      Unless you are literally Amazon or big Co, spending ten of millions ( if 20% is millions then at-least 5-10 million ) on just SSDs alone ! should make it pretty much a core business problem to solve? My sense that in the last decade startups have not lost the skills to do Co-Location setups as they did in 2000s and think it is more complex than actually is. Co-Lo hardware management is hard yes, but if it not even worth doing 10+ million /year budgets we would never have SaaS companies pre-cloud at all.
      - lazide 5y ago
        Which is the core business problem exactly that you think they should be solving?
        manquer 5y ago
        A core business problem, the core business problem may or may not be the same.[1] At this scale infrastructure cost is a critical problem that can afford to have dedicated teams/vertical trying to solve for. Hypothetically if you are spending $50 Million+ / Year on the cloud a dedicated team of even 10 senior engineers to setup your co-location with your hardware to consider migration of your costliest and also least cloud native components would maybe cost $2-5M more. With attractive debt financing that is readily available these days you can easily amortize your purchase expenses over the 2-3 year hardware lifecycle and realize savings, there is not much justification not to also pursue this along with all other features you are also pursuing. The cost is a very low investment compared to your costs with potential for very high saving ROI, so even if the chance of success is low you should give it a shot. i.e. If you can save say $10 Million on the $50 Million, your $2 Million investment needs only 20% probability of success to have expected value in the green. [1] I don't think there should be only one core (the) business problem for a startup, there are always few critical problems startups have to solve for at any given stage.
PragmaticPulp 5y ago
> In my experience, doing things in the cloud is about as expensive per 12-18 months as buying the hardware up front is. The fallacy is comparing hardware costs to services cost. The hardware is the cheap part. When you run your own system, you have to develop the entire system up front and maintain it on the backend. The hardware is cheap by comparison to the salaries and development costs you pay. > $2 million should get you a team of 10 sysadmins and devops engineers. Probably double that once you add in fully-loaded costs as well as the compensation for ~2 managers to manage them.
- aflag 5y ago
  I think he compared hardware + team salary vs aws. After a certain size that starts tipping over to the side of having things on prem. Running things on prem is nothing scary. You just need people with the skill set needed to do it. But when you’re spending millions in infrastructure, that’s hardly a problem.
- mbreese 5y ago
  But if you’re paying X in OpEx to AWS, at some point Y in CapEx (hardware) and Z for OpEx (your people) becomes more compelling. What I believe the comment above is arguing is that if you are saving millions for 20% savings on SSD costs, X >> Y + Z. Yes, you have to manage the hardware, and Y doesn’t automatically go to zero for year two, but the convenience of the cloud isn’t always cost effective. Don’t get caught up with the details. The 2 million figure doesn’t matter as much. It’s finding that inflection point and making the better business decision.
- starfallg 5y ago
  >When you run your own system, you have to develop the entire system up front and maintain it on the backend. The hardware is cheap by comparison to the salaries and development costs you pay The development costs are OTC that are amortised during the life of the solution. Whereas in xAAS, they are MRC. The longer your tech refresh cycle, the cheaper it is. It's inherent to the pricing model. Also when you get down to the cruz if it, these solutions (like openstack or vSphere), are software platform that provides similar features. There's not much development costs, it's just software licensing and PS. In terms of operations, it's not like you can get rid of sysadmins, they just morphed into DevOps. >Probably double that once you add in fully-loaded costs as well as the compensation for ~2 managers to manage them. You might as well add all sorts of additional costs such as egress charges on exit and cloud consultancy.
stef25 5y ago
> $2 million should get you a team of 10 sysadmins and devops engineers Managing staff vs managing AWS ... I know what I'd choose (without really knowing the numbers)

by tinco 5y ago

zerd 5y ago

There's also opportunity cost. What if you instead of hiring a team of 10 people (probably need more, managers, distributed geographically etc), you hire more developers, marketing, sales and increase revenue. So instead of saving $2M, you make $3M more revenue. Making up numbers of course, but companies can estimate this. If that's the case it makes sense to keep paying AWS, until the ratio changes, and then they can cut costs.

gwright 5y ago

> In my experience, doing things in the cloud is about as expensive per 12-18 months as buying the hardware up front is. It seems like you are glossing over the other costs: staff to implement and manage, development time and maintenance for automation to re-implement everything that AWS includes, data center costs (not clear if you were thinking of hardware ownership only or data-center also). I'm not saying you didn't think about those things just saying that they can't be ignored in these types of comparisons.

aflag 5y ago
You don’t need to reimplement everything aws provides. AWS provides services on demand for a huge costumer base. The sysadmin/devops team you set up needs to solve only your particular problem. You can often get a better, easier to use and maintain system this way. The downside is that you have to pay for that extra team, but if only 20% of your data already costs millions, your scale is big enough that you’ll likely save money by hiring a sys admin team.
- lazide 5y ago
  Sometimes, sometimes not. A lot of common ‘prod bricks’ (S3, Managed kubernetes, etc) get used because they are convenient, and while they could be implemented in some other, more bespoke way, it’s rarer and rarer that it actually pencils out as a net win. You also have to deal with the complexity of managing your own version of it, which is non trivial over the full lifecycle of something. If it is your core business to provide that thing? Sometimes or even often worth it. Otherwise, often not.
  - manquer 5y ago
    Unless you are literally Amazon or big Co, spending ten of millions ( if 20% is millions then at-least 5-10 million ) on just SSDs alone ! should make it pretty much a core business problem to solve? My sense that in the last decade startups have not lost the skills to do Co-Location setups as they did in 2000s and think it is more complex than actually is. Co-Lo hardware management is hard yes, but if it not even worth doing 10+ million /year budgets we would never have SaaS companies pre-cloud at all.
    - lazide 5y ago
      Which is the core business problem exactly that you think they should be solving?
      - manquer 5y ago
        A core business problem, the core business problem may or may not be the same.[1] At this scale infrastructure cost is a critical problem that can afford to have dedicated teams/vertical trying to solve for. Hypothetically if you are spending $50 Million+ / Year on the cloud a dedicated team of even 10 senior engineers to setup your co-location with your hardware to consider migration of your costliest and also least cloud native components would maybe cost $2-5M more. With attractive debt financing that is readily available these days you can easily amortize your purchase expenses over the 2-3 year hardware lifecycle and realize savings, there is not much justification not to also pursue this along with all other features you are also pursuing. The cost is a very low investment compared to your costs with potential for very high saving ROI, so even if the chance of success is low you should give it a shot. i.e. If you can save say $10 Million on the $50 Million, your $2 Million investment needs only 20% probability of success to have expected value in the green. [1] I don't think there should be only one core (the) business problem for a startup, there are always few critical problems startups have to solve for at any given stage.

PragmaticPulp 5y ago

> In my experience, doing things in the cloud is about as expensive per 12-18 months as buying the hardware up front is. The fallacy is comparing hardware costs to services cost. The hardware is the cheap part. When you run your own system, you have to develop the entire system up front and maintain it on the backend. The hardware is cheap by comparison to the salaries and development costs you pay. > $2 million should get you a team of 10 sysadmins and devops engineers. Probably double that once you add in fully-loaded costs as well as the compensation for ~2 managers to manage them.

aflag 5y ago
I think he compared hardware + team salary vs aws. After a certain size that starts tipping over to the side of having things on prem. Running things on prem is nothing scary. You just need people with the skill set needed to do it. But when you’re spending millions in infrastructure, that’s hardly a problem.
mbreese 5y ago
But if you’re paying X in OpEx to AWS, at some point Y in CapEx (hardware) and Z for OpEx (your people) becomes more compelling. What I believe the comment above is arguing is that if you are saving millions for 20% savings on SSD costs, X >> Y + Z. Yes, you have to manage the hardware, and Y doesn’t automatically go to zero for year two, but the convenience of the cloud isn’t always cost effective. Don’t get caught up with the details. The 2 million figure doesn’t matter as much. It’s finding that inflection point and making the better business decision.
starfallg 5y ago
>When you run your own system, you have to develop the entire system up front and maintain it on the backend. The hardware is cheap by comparison to the salaries and development costs you pay The development costs are OTC that are amortised during the life of the solution. Whereas in xAAS, they are MRC. The longer your tech refresh cycle, the cheaper it is. It's inherent to the pricing model. Also when you get down to the cruz if it, these solutions (like openstack or vSphere), are software platform that provides similar features. There's not much development costs, it's just software licensing and PS. In terms of operations, it's not like you can get rid of sysadmins, they just morphed into DevOps. >Probably double that once you add in fully-loaded costs as well as the compensation for ~2 managers to manage them. You might as well add all sorts of additional costs such as egress charges on exit and cloud consultancy.

stef25 5y ago

> $2 million should get you a team of 10 sysadmins and devops engineers Managing staff vs managing AWS ... I know what I'd choose (without really knowing the numbers)