5 ms·

Here's my anecdotal, one-data-point experience from moving a giant EC2 environment to datacenter: 1. Your operational overhead will increase _a lot_. Be ready

Here's my anecdotal, one-data-point experience from moving a giant EC2 environment to datacenter: 1. Your operational overhead will increase _a lot_. Be ready to hire on a lot of ops staff if you expect them to do anything but put out fires. And as you grow you'll need experts, people like network engineers. 2. Any weirdness you experienced with AWS infrastructure will be replaced with weirdness in your own environment, except now you're on the line to troubleshoot and fix it yourself. 3. Operation staff will immediately start guarding the food bowl as resources become finite. Server provision waits start to seem like breadlines. Power is consolidated with Those Whom You Must Ask. 4. Your cost will decrease, sometimes significantly so. 5. You'll have more hardware flexibility to run your app just the way you want to (Stack Overflow's mega databases come to mind[0]). In the end I think this type of transition is for stable companies that don't mind or even prefer strong divisions of labor (coders who code, sysadmins who sysadmin, testers who test), but it's not for startups or companies that hope to move with any kind of strong velocity. [0] http://highscalability.com/blog/2014/7/21/stackoverflow-update-560m-pageviews-a-month-25-servers-and-i.html http://highscalability.com/blog/2014/7/21/stackoverflow-upda...

chx 12y ago
> Your operational overhead will increase _a lot_. Be ready to hire on a lot of ops staff if you expect them to do anything but put out fires. And as you grow you'll need experts, people like network engineers. Why? What are you talking about? You are hiring servers you are no colo'ing them. Networking them is not your problem. Your responsibility still starts from a root prompt just there's no VM layer between that and the physical server.
- kovyrin 12y ago
  Yes, it really does look like @stephen-mw is talking about a migration to an owned/colocated hardware. And that's a very different kind of hairy mess I'd prefer to stay away from as long as humanly possible.
  - chx 12y ago
    OMG, colo. I've been the architect of a Top 500 website acquired by a Top 100 and even for that we just rented a few servers (you'd be surprised how few -- a dozen or so). I do not really know how big you need to be make colo a sensible choice but quite probably much, much bigger than most think of. We ran the numbers and they weren't pretty. And for cloud: we purchased video conversion as a service, that one ran in the cloud. I can see how that makes sense.
- merb 12y ago
  and if you have more than one? or more than one dc? somebody needs to connect it, or you will need a VPN. it's not that easy without Clouds if you need connected servers. We switched to AWS since connecting servers in a dc isn't as cheap as people think of.
  - Negitivefrags 12y ago
    This article was talking about renting servers from Softlayer. All servers on your account in Softlayer have a private VLAN that connects them, even between data centers. Private networks for your servers are a feature of many server hosts.
  - kovyrin 12y ago
    We get free inter-DC connectivity for free with Softlayer. There really is no difference for us between connecting to a server within a DC or between datacenters (aside from added latency, of cource) – the same private address space, the same access rules (we see all of our machines in a private backend network from any datacenter).
  - nasalgoat 12y ago
    Every enterprise router in existence has built-in tunnel support. Even the open source ones like PFSense.
  - corford 12y ago
    I host all my stuff with Hetzner (who are at the cheap end of the market compared to Softlayer) and even they provide pre-configured VLANS and private switches. For intra-DC you can just setup some redundant openvpn links or pay the dc to configure a hardware tunnel between both sites.
twblalock 12y ago
Operation staff will immediately start guarding the food bowl as resources become finite. Server provision waits start to seem like breadlines. Power is consolidated with Those Whom You Must Ask. This is the worst part about moving out of the cloud, especially since cloud computing has moved a lot of ops and deployment responsibility to developers.
- rodgerd 12y ago
  I would say an absolute pre-req for moving from Amazon to own servers would be tooling like VCAC or OpenShift to give people a nice self-service experience.
  - wmf 12y ago
    Then you get "out of capacity" errors even sooner. You could have a pretty bad experience with an internal cloud that was self-service but wasn't run like a service on the business side, so the fixed annual budget was exhausted immediately.
Negitivefrags 12y ago
Contrast that to my experience with renting from Softlayer: 1) The operational overhead is the same as Amazon. Actually I think it's less because everything is so predictable. Every machine we get has identical performance to any other. We still don't have to care about the exact same list of things that EC2 provided but we also don't have to care about weird cloud issues. 2) See above answer but also Softlayer are happy to provide actual support for any issues in a timely manner. In general everything has been much more reliable that you actually have to think less about High Availabilty technologies that make your stack more complex. In the years I've been using them we have only had a few hard disk failures that were replaced in around an hour and we just failed over to a replicated slave manually for around 2-3 minutes of downtime. 3) Resources are no more finite than EC2. The only difference is that the provisioning time is 1 hour rather than 1 minute. That has still been perfectly fine to respond to unexpected load events. 4) Absolutely 5) Also great. People often compare Cloud Hosting with Colo. What they should compare cloud hosting with is renting servers. You get 95% of the benefits of cloud with none of the drawbacks.
- snewman 12y ago
  Can you walk me through a concrete example of cost savings? Every time I look at something like SoftLayer, it doesn't seem to offer much of a savings compared to a corresponding EC2 reserved instance. Sure, you can do way better than EC2 on-demand instances, but the reasonable comparison is to a reserved instance.
  - zaroth 12y ago
    I think a big part of it is that dedicated metal and "reserved instances" are not apples-to-apples. You want absolute control, consistency, and customization of the build, you can buy/lease metal and co-locate it, or you can rent it from someone like SoftLayer, and you still don't ever have to walk into a data center.
  - Sidis 12y ago
    Inter-DC network traffic is not rated/charged for SoftLayer. The key cost differentation is network charges when contrasting SoftLayer other services.

chx 12y ago

> Your operational overhead will increase _a lot_. Be ready to hire on a lot of ops staff if you expect them to do anything but put out fires. And as you grow you'll need experts, people like network engineers. Why? What are you talking about? You are hiring servers you are no colo'ing them. Networking them is not your problem. Your responsibility still starts from a root prompt just there's no VM layer between that and the physical server.

kovyrin 12y ago
Yes, it really does look like @stephen-mw is talking about a migration to an owned/colocated hardware. And that's a very different kind of hairy mess I'd prefer to stay away from as long as humanly possible.
- chx 12y ago
  OMG, colo. I've been the architect of a Top 500 website acquired by a Top 100 and even for that we just rented a few servers (you'd be surprised how few -- a dozen or so). I do not really know how big you need to be make colo a sensible choice but quite probably much, much bigger than most think of. We ran the numbers and they weren't pretty. And for cloud: we purchased video conversion as a service, that one ran in the cloud. I can see how that makes sense.
merb 12y ago
and if you have more than one? or more than one dc? somebody needs to connect it, or you will need a VPN. it's not that easy without Clouds if you need connected servers. We switched to AWS since connecting servers in a dc isn't as cheap as people think of.
- Negitivefrags 12y ago
  This article was talking about renting servers from Softlayer. All servers on your account in Softlayer have a private VLAN that connects them, even between data centers. Private networks for your servers are a feature of many server hosts.
- kovyrin 12y ago
  We get free inter-DC connectivity for free with Softlayer. There really is no difference for us between connecting to a server within a DC or between datacenters (aside from added latency, of cource) – the same private address space, the same access rules (we see all of our machines in a private backend network from any datacenter).
- nasalgoat 12y ago
  Every enterprise router in existence has built-in tunnel support. Even the open source ones like PFSense.
- corford 12y ago
  I host all my stuff with Hetzner (who are at the cheap end of the market compared to Softlayer) and even they provide pre-configured VLANS and private switches. For intra-DC you can just setup some redundant openvpn links or pay the dc to configure a hardware tunnel between both sites.

twblalock 12y ago

Operation staff will immediately start guarding the food bowl as resources become finite. Server provision waits start to seem like breadlines. Power is consolidated with Those Whom You Must Ask. This is the worst part about moving out of the cloud, especially since cloud computing has moved a lot of ops and deployment responsibility to developers.

rodgerd 12y ago
I would say an absolute pre-req for moving from Amazon to own servers would be tooling like VCAC or OpenShift to give people a nice self-service experience.
- wmf 12y ago
  Then you get "out of capacity" errors even sooner. You could have a pretty bad experience with an internal cloud that was self-service but wasn't run like a service on the business side, so the fixed annual budget was exhausted immediately.

Negitivefrags 12y ago

Contrast that to my experience with renting from Softlayer: 1) The operational overhead is the same as Amazon. Actually I think it's less because everything is so predictable. Every machine we get has identical performance to any other. We still don't have to care about the exact same list of things that EC2 provided but we also don't have to care about weird cloud issues. 2) See above answer but also Softlayer are happy to provide actual support for any issues in a timely manner. In general everything has been much more reliable that you actually have to think less about High Availabilty technologies that make your stack more complex. In the years I've been using them we have only had a few hard disk failures that were replaced in around an hour and we just failed over to a replicated slave manually for around 2-3 minutes of downtime. 3) Resources are no more finite than EC2. The only difference is that the provisioning time is 1 hour rather than 1 minute. That has still been perfectly fine to respond to unexpected load events. 4) Absolutely 5) Also great. People often compare Cloud Hosting with Colo. What they should compare cloud hosting with is renting servers. You get 95% of the benefits of cloud with none of the drawbacks.

snewman 12y ago
Can you walk me through a concrete example of cost savings? Every time I look at something like SoftLayer, it doesn't seem to offer much of a savings compared to a corresponding EC2 reserved instance. Sure, you can do way better than EC2 on-demand instances, but the reasonable comparison is to a reserved instance.
- zaroth 12y ago
  I think a big part of it is that dedicated metal and "reserved instances" are not apples-to-apples. You want absolute control, consistency, and customization of the build, you can buy/lease metal and co-locate it, or you can rent it from someone like SoftLayer, and you still don't ever have to walk into a data center.
- Sidis 12y ago
  Inter-DC network traffic is not rated/charged for SoftLayer. The key cost differentation is network charges when contrasting SoftLayer other services.