4 ms·

Hosting your own servers doesn't require foregoing virtualization and containers. This is a critical point. In every recent hardware project where we've deploy

Hosting your own servers doesn't require foregoing virtualization and containers. This is a critical point. In every recent hardware project where we've deployed big, beefy servers to colocation, the first step was deciding on the virtualization platform and the automation tools to manage it, as the flexibility, scaling, and reliability concerns are principal now. I have to hope that no one thinks of in terms of a physical box that is the database server, etc, any more. And worth noting that on a couple of medium-range Dell servers you literally can spin up and down hundreds of virtual machines, obviously depending upon size and scale. Even on a recent low end OVH server I requisitioned, the first thing I did was configure libvirt. The base image becomes almost irrelevant.

matlock 13y ago
But even in that case there is a single point of failure in the host. That's all manageable, but it is effort, that could otherwise be put into developing the application
- corresation 13y ago
  I would never advocate that someone build a production stack with a single physical server -- I've generally used three servers in concert in such situations.
  - matlock 13y ago
    But then you are probably getting close in pricing to running the stack on AWS, as you don't need to have that much reserved infrastructure when you can easily replace it when there are problems with the server
    - druiid 13y ago
      If you have the requirements for more than a few VM's on AWS, then the pricing for physical is going to be cheaper by far, even with a three-stack of physical servers. Let's put a basic Poweredge R620 as an example. For about $10k/server (and I'm going for buying hardware to kind of do a worst-case-scenario), you can get 384GB of memory per server, times three that's over 1T of memory in your cluster. Given the EC2 Small instance type of 1.7GB/VM that would be over 600 VM's you could run on that kind of stack. Pricing for a half cabinet with power and bandwidth (you can find places with un-metered gigabit connectivity) will run you no more than about $2500/month (Much less if you are good at negotiating). So you have an outlay of around $40k for total hardware costs and recurring monthly of $2500/month. Amortizing this over expected lifespan of three years for the servers (worst case scenario), you can estimate per month costs somewhere in the range of $3600/month. With EC2 to reach parity with those 600 'Small' instances would be an initial deposit of $57,600 (for three year term) and a recurring monthly of $11,826 for a total 'recurring' of $13,426/month. Now, I should point out that these are obviously not apples-to-apples if you have: Huge CPU requirements on those 600 instances, or need lots of disk. Your up-front hardware costs increase (perhaps substantially) once those come into play. More I'm just trying to compare a bare instance scenario. As per one of my comments elsewhere in the thread really it all comes down to workloads and doing a real business analysis.
    - bluedino 13y ago
      Probably not - you can stick quite a few virtual servers on a $6,000 phyiscal server and not have to worry about costs for usage, or noisy neighbors.
      - matlock 13y ago
        But that's already an investment that can get you quite some resources on AWS for a while.
spongle 13y ago
That doesn't quite work for us. All our front ends/service and infrastructure nodes are virtual but databases are all dedicated. This is because no virtualization tech likes handling large nodes (FCAL, 48 cores, 256Gb RAM). We lost 20% throughput by sticking our SQL Server primary and replicas on ESX versus physical when running our trials. That's a big hit when you run 150million queries a day.

matlock 13y ago

But even in that case there is a single point of failure in the host. That's all manageable, but it is effort, that could otherwise be put into developing the application

corresation 13y ago
I would never advocate that someone build a production stack with a single physical server -- I've generally used three servers in concert in such situations.
- matlock 13y ago
  But then you are probably getting close in pricing to running the stack on AWS, as you don't need to have that much reserved infrastructure when you can easily replace it when there are problems with the server
  - druiid 13y ago
    If you have the requirements for more than a few VM's on AWS, then the pricing for physical is going to be cheaper by far, even with a three-stack of physical servers. Let's put a basic Poweredge R620 as an example. For about $10k/server (and I'm going for buying hardware to kind of do a worst-case-scenario), you can get 384GB of memory per server, times three that's over 1T of memory in your cluster. Given the EC2 Small instance type of 1.7GB/VM that would be over 600 VM's you could run on that kind of stack. Pricing for a half cabinet with power and bandwidth (you can find places with un-metered gigabit connectivity) will run you no more than about $2500/month (Much less if you are good at negotiating). So you have an outlay of around $40k for total hardware costs and recurring monthly of $2500/month. Amortizing this over expected lifespan of three years for the servers (worst case scenario), you can estimate per month costs somewhere in the range of $3600/month. With EC2 to reach parity with those 600 'Small' instances would be an initial deposit of $57,600 (for three year term) and a recurring monthly of $11,826 for a total 'recurring' of $13,426/month. Now, I should point out that these are obviously not apples-to-apples if you have: Huge CPU requirements on those 600 instances, or need lots of disk. Your up-front hardware costs increase (perhaps substantially) once those come into play. More I'm just trying to compare a bare instance scenario. As per one of my comments elsewhere in the thread really it all comes down to workloads and doing a real business analysis.
  - bluedino 13y ago
    Probably not - you can stick quite a few virtual servers on a $6,000 phyiscal server and not have to worry about costs for usage, or noisy neighbors.
    - matlock 13y ago
      But that's already an investment that can get you quite some resources on AWS for a while.

spongle 13y ago

That doesn't quite work for us. All our front ends/service and infrastructure nodes are virtual but databases are all dedicated. This is because no virtualization tech likes handling large nodes (FCAL, 48 cores, 256Gb RAM). We lost 20% throughput by sticking our SQL Server primary and replicas on ESX versus physical when running our trials. That's a big hit when you run 150million queries a day.