4 ms·
I was involved in a long term project/service that moved from private bare metal to private vmware to private openstack to private vastly newer vmware to AWS.
by VLM 4y ago
I was involved in a long term project/service that moved from private bare metal to private vmware to private openstack to private vastly newer vmware to AWS.
You'll probably laugh but the biggest problem we had was security, moving from one thing to another is possible but the "in between" needs to be as secure as normal operation so you write a LOT more security group style access lists than you'd think. If your process involves five middleware servers there are a large combination of partial moves possible which requires security review of each set of access lists.
The second biggest problem we had with "always be moving" was latency. "Sure, we can move the mysql server today and the front end servers next month". Then insert a surprise 30 ms latency that nobody ever expected and suddenly round trips go from immeasurable and unimportant to being a pretty big idea while parts have been moved. Also funny watching front end people who did "SELECT *" and filter in their frontend because its a 10G connection find things are a little slower when the db is far away on the internet.
The third biggest problem we had was documentation. "Hey its Tuesday so does anyone know where the 'ohio' database is today?" Anybody can move stuff, bigger question is can everyone find it, LOL? Whatever you use, a wiki or some more elaborate project planner, everyone needs to be able to find where "stuff" currently is. How many people need to know where things are and what's the labor cost of moving something?
The forth biggest problem which has kind of gone away with Docker is the old fashioned version bump. "Well, we're moving it anyway, and it would take extra work to either upgrade the old xyz-server from 3.99 to 4.00 or install 3.99 on the new cloud, so what could possibly go wrong?" Turns out to be a lot, mostly performance tuning related although occasionally "ha ha we deprecated that feature two years ago so we removed it last week because nobody would notice". So try not to merge an upgrade process with a cloud conversion process if at all possible.
The fifth biggest problem we had was budget. The bean counters liked how the electric bill was about constant and the bare metal servers were a fraction of HVAC and lighting although you'd have to replace a HDD every couple YEARS. Suddenly "operations" became bean-countable with the cloud and different clouds count different beans so suddenly someone else is driving "how we're going to save money" because overtime and weekend labor is "free" if its salaried but god help anyone provably "wasting" 25 cents on the AWS bill, and the best way to get a promotion is to force someone elses team to work saturday and sunday unpaid salaried to save fifty cents of AWS budget. The lock-in is your internal procedures will depend on some cloud providers crazy arbitrary billing ideas, they're not all on the same page. The endusers will never accept any explanation that's truthful along the lines of "sorry we can't change the font on that webpage because of AWS" which in enormous technical detail would be an entire page of tech and billing nonsense.
I think really the meta-observation of cloud lockins is the front line salaried employees don't like putting in 30+ hours of unpaid maint window overtime just to lower the cloud bill by $5 so some manager can get a promotion. "Technically possible to move and reduce the bill" doesn't save money if employees quit or essentially rebel.