6 ms·

Why I’ll never own another server

63 points by twampss 17y ago

IgorPartola 17y ago
Admittedly I never read any of these cost comparisons, but if I were to write one, especially if my own money were on the line, I would sure as hell include maintenance costs and vendor expenses. Otherwise it's like buying a car and not taking into account the cost of fuel.
- borism 17y ago
  Why would you buy a car if you care about the cost of fuel?
pierrefar 17y ago
One of the biggest reasons I use SimpleDB is because I don't want to admin a database. I just don't have the time to deal. The extra $ I pay for SimpleDB is the value of my time I save, and, frankly, my time is worth more than the SimpleDB premium.
fierarul 17y ago
The title is about owning a server but all the points he makes are about owning and managing a whole Data Center by yourself. Of course that complicates matters ! What I don't understand is why wouldn't they take for example collocating their servers in multiple, geographically distributed, Data Centers. Also, this doesn't need to be black and white. One could use "the cloud" to offload peak traffic/computation but if you have a predictable usage pattern, it becomes economical to also own hardware. And I think this is the core of the matter: if you have an unpredictable scenario, it's better to rent. And being that startups and new projects are by definition rather volatile, it makes sense not to invest money into hardware. Not to mention that unless the company / project takes off, there is no money to actually buy hardware upfront. There is also no time to manage that hardware.
- wmf 17y ago
  All his complaints about managing a "datacenter" also apply to colo.
- imajes 17y ago
  As a startup, owning hardware (or, sometimes, leasing) tends to eat up on your balance sheet really quickly as a cap ex (capital expenditure). You're immediately spending money and taking runway off the table and turning it into illiquid assets. So sure, raw hardware you own in a rack somewhere is probably cheaper over 18 months. But then you're still stuck with it after 18 months (whereas you can just swap out for a larger sized ec2 in the space of a few minutes, whenever you want, without penalty). Then again, unless you're buying a whole cage/closet you're probably paying the top whack fee anyhow, since data centers almost always deal with negotiating on the cage/closet level - anything smaller is typically not worth it to them. So you're paying full rate for your 2u box, and sitting on hardware that's depreciating. Plus, to get into the good datacenters, the ones that have provable disaster recovery, multi homed bandwidth, power and cooling , you will need to prove your creditworthiness and such. But that's OK - you'll stake all you got to get a couple cheap servers hosted somewhere that won't go down (and if it did, be trivial to heal) Finally: to the point about predictable usage lowering costs: amazon already does this, with reserved instances. You give them a heads up about what you're going to need over the coming year, they'll give you a discount. If distributed, fail tolerant and instant deployable are three things that describe your startup, relying on your hardware to always work, that your sysop friend is always around and that you're on top of kernel bugs seems like a pretty fast way to fail. On the other hand, offload all this to a company who's doing it anyhow, and you might find that you get to extend your runway, grow with your customers (who says it is bad to deploy more servers based on customer orders??) and get to focus on more interesting things, such as where to find the next beer. :)
  - fierarul 17y ago
    > If distributed, fail tolerant and instant deployable are three things that describe your startup, relying on your hardware to always work, that your sysop friend is always around and that you're on top of kernel bugs seems like a pretty fast way to fail. Indeed, if you are a typical startup, then you either don't have the capital, the time or the knowledge to manage your own hardware. But at some point you should move into a hardware-owner category at least because it's the foundation on whatever your are building and you might want some control over that.
benologist 17y ago
It'd be nice if the comparison compared leased hardware as well, pretty sure that's most-relevant for most startups.
- mmastrac 17y ago
  When we started DotSpots in 2007, we leased physical servers from iWeb. It was far cheaper than EC2, but you lose of ton of agility when you can't pave a server and rebuild it from the command line. We switched to EC2 and starting paying a bit more, but we've been able to expand capacity much more nimbly than before. The only thing I'd recommend running outside of EC2 would be memcache, but that's only if you're really pinching pennies. If you're an early stage startup and you're not running virtual machines, you might be doing something wrong (IMHO).
  - WALoeIII 17y ago
    Why would you run memcache outside of your VLAN though, doesn't that obviate most of the point (speed) by adding so much network latency?
    - imajes 17y ago
      that's what i thought, however amazon's ec2 structure doesn't seem to be the most impressive for memcache throughput...
    - mmastrac 17y ago
      Yeah, it adds latency, so it won't solve every problem. It does scale a lot better than something that ends up thrashing the disk, though.
  - benologist 17y ago
    Would you recommend it in a scenario where you're serving a significant volume of requests per second?
    - mmastrac 17y ago
moserware 17y ago
Similiar ideas came up in the Mix10 talk "Cloud Computing Economies of Scale": http://live.visitmix.com/MIX10/Sessions/EX01 http://live.visitmix.com/MIX10/Sessions/EX01
kylemathews 17y ago
The funny thing is is I've never owned a server, and from the look of things, probably never will. Granted I've only been running my company for a ~6 months or so but the idea of owning a server feels pretty archaic.
deleted 17y ago
[deleted]
erlanger 17y ago
He mentions all of the overhead of running your own data center, but fails to provide a balanced analysis by omitting the significant expense of running an AWS deployment properly. Just as AWS solves some problems of dedicated servers, it introduces its own challenges which one would not have to consider when running dedicated servers.
- rbranson 17y ago
  This is relatively trivial compared to running your own physical server setup (colo or datacenter). Think about where we're at now with EC2: EBS-backed servers are about as easy to manage as a regular VPS, plus they can be regularly snapshotted (every few minutes if you want to) and moved to another datacenter in very little time in the event of an outage.
- Mc_Big_G 17y ago
  Right, like the fact that an instance can have only one ip address. Virtual hosting for multiple websites with SSL goes right out the window. There are a couple solutions to this problem, but they're either impractical or just plain unacceptable for production boxes. All platform choices have these kinds of gotchas and/or extra costs for things that are just free if you own the server. I use AWS every chance I get, but sometimes a project requirement forces you to go dedicated.
  - WALoeIII 17y ago
    This is easily done with Elastic Load Balancers, simply proxy to different ports for SSL and instruct your webserver to listen there. The huge upside is you can automate this just as well as any other piece.
    - andrewvc 17y ago
      except you don't have X-Forwarded-For with ELB, which can be a problem, since if you need SSL, there's a good chance you probably also need to be logging IPs for security purposes. Of course, you can just setup your own load balancer on EC2 with HAProxy and stunnel, but that's a PITA.
dpifke 17y ago
Am I the only one who reads "I'll never own another server" and thinks about all the folks who said "I'll never hire another programmer" back when sites like Elance first started getting popular? Outsourcing is great and all, but it's hardly the panacea people make it out to be.
- kunley 17y ago
  Yes you are probably the only one. You can't automate hiring & managing coders, while you can automate purchasing & maintaining infrastructure.
  - dpifke 17y ago
    That's a fairly insulting way to look at the art of system administration, database administration, and the like. Anyone who's ever been any good at those jobs would disagree that you can automate them any more than you can automate software development. It makes sense to outsource those functions when you can't afford to hire someone who knows what they're doing and when what you're doing exactly fits the cookie-cutter mold of what the vendor is offering, and when you're willing to trust an outside vendor with core business functionality (or when it's not core to your business). But a skilled sysadmin or DBA will absolutely add value that you won't get from outsourcing.
    - kunley 17y ago
      You know what.. I am supposedly skilled sysadmin, middleware admin and a DBA, so let's assume I know what I mean. And I dare to say that for many cases many tasks usually attributed to "us" can be automatized (and who's writing such layer? of course an admin, not a programmer; a casual programmer doesn't have mindset needed). I earn money on the large projects (telco industry mostly) where people like us are absolutely needed and without us the things would turn into chaos & hell. Still you don't need an admin to build an infractructure for a small app when you have service providers like Heroku. Then admin can be given another work to do (esp. coding, as most of admins make excellent coders, not necessarily vice-versa). You may think these are quite strong opinions, inconvenient for some people maybe, but I live in the industry for 15 years or so and that's what I see all around.

63 points by twampss 17y ago

Admittedly I never read any of these cost comparisons, but if I were to write one, especially if my own money were on the line, I would sure as hell include maintenance costs and vendor expenses. Otherwise it's like buying a car and not taking into account the cost of fuel.

borism 17y ago
Why would you buy a car if you care about the cost of fuel?

pierrefar 17y ago

One of the biggest reasons I use SimpleDB is because I don't want to admin a database. I just don't have the time to deal. The extra $ I pay for SimpleDB is the value of my time I save, and, frankly, my time is worth more than the SimpleDB premium.

fierarul 17y ago

The title is about owning a server but all the points he makes are about owning and managing a whole Data Center by yourself. Of course that complicates matters ! What I don't understand is why wouldn't they take for example collocating their servers in multiple, geographically distributed, Data Centers. Also, this doesn't need to be black and white. One could use "the cloud" to offload peak traffic/computation but if you have a predictable usage pattern, it becomes economical to also own hardware. And I think this is the core of the matter: if you have an unpredictable scenario, it's better to rent. And being that startups and new projects are by definition rather volatile, it makes sense not to invest money into hardware. Not to mention that unless the company / project takes off, there is no money to actually buy hardware upfront. There is also no time to manage that hardware.

wmf 17y ago
All his complaints about managing a "datacenter" also apply to colo.
imajes 17y ago
As a startup, owning hardware (or, sometimes, leasing) tends to eat up on your balance sheet really quickly as a cap ex (capital expenditure). You're immediately spending money and taking runway off the table and turning it into illiquid assets. So sure, raw hardware you own in a rack somewhere is probably cheaper over 18 months. But then you're still stuck with it after 18 months (whereas you can just swap out for a larger sized ec2 in the space of a few minutes, whenever you want, without penalty). Then again, unless you're buying a whole cage/closet you're probably paying the top whack fee anyhow, since data centers almost always deal with negotiating on the cage/closet level - anything smaller is typically not worth it to them. So you're paying full rate for your 2u box, and sitting on hardware that's depreciating. Plus, to get into the good datacenters, the ones that have provable disaster recovery, multi homed bandwidth, power and cooling , you will need to prove your creditworthiness and such. But that's OK - you'll stake all you got to get a couple cheap servers hosted somewhere that won't go down (and if it did, be trivial to heal) Finally: to the point about predictable usage lowering costs: amazon already does this, with reserved instances. You give them a heads up about what you're going to need over the coming year, they'll give you a discount. If distributed, fail tolerant and instant deployable are three things that describe your startup, relying on your hardware to always work, that your sysop friend is always around and that you're on top of kernel bugs seems like a pretty fast way to fail. On the other hand, offload all this to a company who's doing it anyhow, and you might find that you get to extend your runway, grow with your customers (who says it is bad to deploy more servers based on customer orders??) and get to focus on more interesting things, such as where to find the next beer. :)
- fierarul 17y ago
  > If distributed, fail tolerant and instant deployable are three things that describe your startup, relying on your hardware to always work, that your sysop friend is always around and that you're on top of kernel bugs seems like a pretty fast way to fail. Indeed, if you are a typical startup, then you either don't have the capital, the time or the knowledge to manage your own hardware. But at some point you should move into a hardware-owner category at least because it's the foundation on whatever your are building and you might want some control over that.

benologist 17y ago

It'd be nice if the comparison compared leased hardware as well, pretty sure that's most-relevant for most startups.

mmastrac 17y ago
When we started DotSpots in 2007, we leased physical servers from iWeb. It was far cheaper than EC2, but you lose of ton of agility when you can't pave a server and rebuild it from the command line. We switched to EC2 and starting paying a bit more, but we've been able to expand capacity much more nimbly than before. The only thing I'd recommend running outside of EC2 would be memcache, but that's only if you're really pinching pennies. If you're an early stage startup and you're not running virtual machines, you might be doing something wrong (IMHO).
- WALoeIII 17y ago
  Why would you run memcache outside of your VLAN though, doesn't that obviate most of the point (speed) by adding so much network latency?
  - imajes 17y ago
    that's what i thought, however amazon's ec2 structure doesn't seem to be the most impressive for memcache throughput...
  - mmastrac 17y ago
    Yeah, it adds latency, so it won't solve every problem. It does scale a lot better than something that ends up thrashing the disk, though.
- benologist 17y ago
  Would you recommend it in a scenario where you're serving a significant volume of requests per second?
  - mmastrac 17y ago

moserware 17y ago

Similiar ideas came up in the Mix10 talk "Cloud Computing Economies of Scale": http://live.visitmix.com/MIX10/Sessions/EX01 http://live.visitmix.com/MIX10/Sessions/EX01

kylemathews 17y ago

The funny thing is is I've never owned a server, and from the look of things, probably never will. Granted I've only been running my company for a ~6 months or so but the idea of owning a server feels pretty archaic.

deleted 17y ago

[deleted]

erlanger 17y ago

He mentions all of the overhead of running your own data center, but fails to provide a balanced analysis by omitting the significant expense of running an AWS deployment properly. Just as AWS solves some problems of dedicated servers, it introduces its own challenges which one would not have to consider when running dedicated servers.

rbranson 17y ago
This is relatively trivial compared to running your own physical server setup (colo or datacenter). Think about where we're at now with EC2: EBS-backed servers are about as easy to manage as a regular VPS, plus they can be regularly snapshotted (every few minutes if you want to) and moved to another datacenter in very little time in the event of an outage.
Mc_Big_G 17y ago
Right, like the fact that an instance can have only one ip address. Virtual hosting for multiple websites with SSL goes right out the window. There are a couple solutions to this problem, but they're either impractical or just plain unacceptable for production boxes. All platform choices have these kinds of gotchas and/or extra costs for things that are just free if you own the server. I use AWS every chance I get, but sometimes a project requirement forces you to go dedicated.
- WALoeIII 17y ago
  This is easily done with Elastic Load Balancers, simply proxy to different ports for SSL and instruct your webserver to listen there. The huge upside is you can automate this just as well as any other piece.
  - andrewvc 17y ago
    except you don't have X-Forwarded-For with ELB, which can be a problem, since if you need SSL, there's a good chance you probably also need to be logging IPs for security purposes. Of course, you can just setup your own load balancer on EC2 with HAProxy and stunnel, but that's a PITA.

dpifke 17y ago

Am I the only one who reads "I'll never own another server" and thinks about all the folks who said "I'll never hire another programmer" back when sites like Elance first started getting popular? Outsourcing is great and all, but it's hardly the panacea people make it out to be.

kunley 17y ago
Yes you are probably the only one. You can't automate hiring & managing coders, while you can automate purchasing & maintaining infrastructure.
- dpifke 17y ago
  That's a fairly insulting way to look at the art of system administration, database administration, and the like. Anyone who's ever been any good at those jobs would disagree that you can automate them any more than you can automate software development. It makes sense to outsource those functions when you can't afford to hire someone who knows what they're doing and when what you're doing exactly fits the cookie-cutter mold of what the vendor is offering, and when you're willing to trust an outside vendor with core business functionality (or when it's not core to your business). But a skilled sysadmin or DBA will absolutely add value that you won't get from outsourcing.
  - kunley 17y ago
    You know what.. I am supposedly skilled sysadmin, middleware admin and a DBA, so let's assume I know what I mean. And I dare to say that for many cases many tasks usually attributed to "us" can be automatized (and who's writing such layer? of course an admin, not a programmer; a casual programmer doesn't have mindset needed). I earn money on the large projects (telco industry mostly) where people like us are absolutely needed and without us the things would turn into chaos & hell. Still you don't need an admin to build an infractructure for a small app when you have service providers like Heroku. Then admin can be given another work to do (esp. coding, as most of admins make excellent coders, not necessarily vice-versa). You may think these are quite strong opinions, inconvenient for some people maybe, but I live in the industry for 15 years or so and that's what I see all around.