49 ms·
We maintain 30 bare metal servers at a colo center, and between me (primarily a developer) and the CTO we spend maybe 1 day per month "managing" them. The last
by bcrosby95 7y ago
We maintain 30 bare metal servers at a colo center, and between me (primarily a developer) and the CTO we spend maybe 1 day per month "managing" them. The last time we had a hardware failure was months ago. The last hardware emergency was years ago.
Servers run on electricity, not sysadmin powered hamster wheels.
- djmobley 7y agoEverything in one colo facility with no geographic redundancy?
- opless 7y agoMost use cases there is just no need.
- krab 7y agoSome colo companies, even the small ones, offer multiple datacenters. You then have to either use public IPs for the inter-service traffic, maintain a VPN or contract it from them.
- pepemon 7y agoYour suggested solutions aren't reliable at all. AWS/GCP/Azure backbones work absolutely differently.
- qaq 7y agoIf you look beyond marketing a single top tier DC has better uptime than AWS AZ. You are way more likely to be bitten by AWS control layer issues than by DC in a good location dropping.
- foobarian 7y agoWhat about software updates and reboots?
- krab 7y agoThere is the same issue with a cloud provider unless you run immutable deployments. Then, you need to invest in the tooling to produce the immutable images.
- krab 7y agoYes, the maintenance is cheap. The changes are more costly. We run a dozen bare metal servers and I see the difference what it takes to spin up a new VM vs. set up a new physical server. There's planning, OS installation (we use preseed images but we weren't able to automate everything), sometimes the redundant network setup doesn't play well with what the switches expect (so you need to call the datacenter). Still, it works out in favor of the bare metal servers. But I'm looking forward to a bit bigger scale to justify a MaaS tool to avoid this gruntwork.
- andreimackenzie 7y agoI completely agree that some types of changes are much more expensive with a bare metal architecture than with cloud. 6 years ago, I worked for a company in the mobile space. This was around the time of the Candy Crush boom, and our traffic and processing/storage needs doubled roughly every six months. Our primary data center was rented space co-located near our urban office. For a while, our sysadmins could simply drive over and rack more servers. We reached a point where our cages were full, and the data center was not willing rent us adjacent space. We were now looking at a very large project to stand up additional capacity elsewhere to augment what we had (with pretty serious implications on the architecture of the whole system beyond the hardware) or move the whole operation to a larger space. This problem ended up hamstringing the business for many months, as many of our decisions were affected by concern about hitting the scale ceiling. We also devoted significant engineering/sysadmin resources to dealing with this problem instead of building new features to grow the business. If the company had chosen a cloud provider or even VPS, it would have been less critical to try to guess how much capacity we'd need a few years down the road to avoid the physical ceiling we dealt with.
- krab 7y agoYes, the cloud premium is also a kind of insurance - you know you'll probably be able to double your capacity anytime you need it.
- hosh 7y agoYeah, "CAPEX always increases, OPEX always decreases".
- GrinningFool 7y agoDoes that time include security updates for the OS and installed services? I would assume so, but 8 (16?) hours a month seems lower than I'd expect given the frequency of vulnerability discovery and security patches.
- deleted 7y ago[deleted]
- bcrosby95 7y agoYes.
- GrinningFool 7y agoThanks!