7 ms·

The raw performance benefits of bare metal vs cloud are incredible, but why does that necessarily mean building & maintaining your own hardware when you can lea

The raw performance benefits of bare metal vs cloud are incredible, but why does that necessarily mean building & maintaining your own hardware when you can lease (or work out whatever financing you want, but still let the hosting company maintain a lot of the responsibility for HW)? And besides financing, taking on all the HW maint? I'm not sure your needs are so unique as to require custom hardware. You're talking about only 64 nodes right now. Your storage and IOPS requirements are not huge. A lot of mid-size hosting companies will give you fantastic service at the 10-1000 servers range. If I were you I'd talk to someone like https://www.m5hosting.com/ https://www.m5hosting.com/ (note: happy dedicated server customer for many years -- and I'm sure there's similar scale operations on east coast if that's really what you need) who have experience running rock solid hosting for ~100s of dedicated servers per customer. I suspect you may just be able to get your 5-10X cost/month improvement (and bare metal performance gains) without having to take on the financing and hardware bits yourself.

rhizome 10y ago
To be sure, I've seen benchmarks (can look up later) that found a 10x+ performance improvement over and above the cost savings, leading to I guess a 100x price/performance improvement.
BJanecke 10y ago
Right now that makes sense yes. What about 5 years from now when it doesn't? This way they get to start building a team and culture to support that kind of infrastructure and wean the baby teeth on nice soft furniture.
- twic 10y ago
  I wonder if they could go with a hosting provider, hire a sysadmin, then second them to the hosting provider to learn how it's done, in return for a credit against their costs. In fact, i wonder if that's a business model for hosting providers. Kind of a sysadmin incubator.
  - BJanecke 10y ago
    Except the really good -> exceptional talent just ends up on staff at the operations incubator and the rest go to the world at large :'(
fabian2k 10y ago
When I looked at how expensive a rented dedicated server was compared to AWS I expected a big difference, but I was still suprised how cheap you can get dedicated servers today. Not worrying about actually owning the hardware is nice, and you still get vastly more hardware for the same price as in the cloud. There are many advantages to AWS and similar services, but if you can't really take advantage of all the goodies because e.g. you also need to provide a local version of your software (which is the case for Gitlab, as far as I understand), renting dedicated servers is an order of magnitude cheaper.
sytse 10y ago
We looked at providers such as Softlayer but while they guarantee the performance of the servers they typically can't guaranty network latency. Since we're doing this to reduce latency https://about.gitlab.com/2016/11/10/why-choose-bare-metal/ https://about.gitlab.com/2016/11/10/why-choose-bare-metal/ this is essential to us. We'll be glad to look into alternatives that manage the servers and network for us although the argument in https://news.ycombinator.com/item?id=13153455 https://news.ycombinator.com/item?id=13153455 that this is the time to build a team that can handle this makes sense to me too.
- pyrox420 10y ago
  Been a softlayer customer for 4 years now. Their network is pretty awesome. When hosted in the same datacenter it's sub-ms response time always. If there is an issue they get right on it. You can even ask them to host the stuff in the same rack to get even better response time.
  - aberoham 10y ago
    Same experience here, going on seven or eight years. They offer a great solution between cloud native and 100% owned/leased/caged bare metal.
  - sytse 10y ago
    Cool, I was not aware you can ask for same rack hosting.
  - krenoten 10y ago
    in the past they've spread servers into different rooms that had interconnects that challenged a team I was on to spend extra time writing compression code. is this better now?
  - sulam 10y ago
    We are also a SL customer -- 4-figures of hosts with them. We have had networking problems in the past (latency and loss far higher than I would expect to see in a well-provisioned DC) and talked to them about it. It ended up being contention with another customer, it got fixed, and our network performance has been great since. I would encourage you to look at what you can get without trying to do your own colo. You're not at the scale where you should be thinking about that.

rhizome 10y ago

To be sure, I've seen benchmarks (can look up later) that found a 10x+ performance improvement over and above the cost savings, leading to I guess a 100x price/performance improvement.

BJanecke 10y ago

Right now that makes sense yes. What about 5 years from now when it doesn't? This way they get to start building a team and culture to support that kind of infrastructure and wean the baby teeth on nice soft furniture.

twic 10y ago
I wonder if they could go with a hosting provider, hire a sysadmin, then second them to the hosting provider to learn how it's done, in return for a credit against their costs. In fact, i wonder if that's a business model for hosting providers. Kind of a sysadmin incubator.
- BJanecke 10y ago
  Except the really good -> exceptional talent just ends up on staff at the operations incubator and the rest go to the world at large :'(

fabian2k 10y ago

When I looked at how expensive a rented dedicated server was compared to AWS I expected a big difference, but I was still suprised how cheap you can get dedicated servers today. Not worrying about actually owning the hardware is nice, and you still get vastly more hardware for the same price as in the cloud. There are many advantages to AWS and similar services, but if you can't really take advantage of all the goodies because e.g. you also need to provide a local version of your software (which is the case for Gitlab, as far as I understand), renting dedicated servers is an order of magnitude cheaper.

sytse 10y ago

We looked at providers such as Softlayer but while they guarantee the performance of the servers they typically can't guaranty network latency. Since we're doing this to reduce latency https://about.gitlab.com/2016/11/10/why-choose-bare-metal/ https://about.gitlab.com/2016/11/10/why-choose-bare-metal/ this is essential to us. We'll be glad to look into alternatives that manage the servers and network for us although the argument in https://news.ycombinator.com/item?id=13153455 https://news.ycombinator.com/item?id=13153455 that this is the time to build a team that can handle this makes sense to me too.

pyrox420 10y ago
Been a softlayer customer for 4 years now. Their network is pretty awesome. When hosted in the same datacenter it's sub-ms response time always. If there is an issue they get right on it. You can even ask them to host the stuff in the same rack to get even better response time.
- aberoham 10y ago
  Same experience here, going on seven or eight years. They offer a great solution between cloud native and 100% owned/leased/caged bare metal.
- sytse 10y ago
  Cool, I was not aware you can ask for same rack hosting.
- krenoten 10y ago
  in the past they've spread servers into different rooms that had interconnects that challenged a team I was on to spend extra time writing compression code. is this better now?
- sulam 10y ago
  We are also a SL customer -- 4-figures of hosts with them. We have had networking problems in the past (latency and loss far higher than I would expect to see in a well-provisioned DC) and talked to them about it. It ended up being contention with another customer, it got fixed, and our network performance has been great since. I would encourage you to look at what you can get without trying to do your own colo. You're not at the scale where you should be thinking about that.