4 ms·

I was working on large scale deployments with multiple datacenters, multiple databases, message passing networks and running multiple customer-facing products.

I was working on large scale deployments with multiple datacenters, multiple databases, message passing networks and running multiple customer-facing products. Load balancers and HA setups were already in use. LVS existed. VMs were popular. "CI/CD" existed, without that name. > 2005 your average box ... Speak for yourself.

_vertigo 4y ago
- terpans 4y ago
  Indeed all of those things existed and they weren’t in widespread use because 99% of organization did not need them. And it hasn't changed. > 2 sysadmins were absolutely not managing “large scale deployments with multiple data centers” Please don't make things up because I was there. > But today it’s actually super plausible that 2 or 3 Devops with modern tooling can manage all of that. Not really.
  - _vertigo 4y ago
    - deleted 4y ago
      [deleted]
    - dijit 4y ago
      I was there too and some things were better, some thing were worse. The problem is that we’re being sold a lie and it’s hard to swallow. “Cloud will save you on staffing costs” - no, it just means you have specialists working on proprietary solutions, just like IBM mainframes or storage appliances or load-balancers/traffic-shapers of yore. “This technology will make rollouts easier” - until it breaks down and is not easy to put together again, you’re at the mercy of your upstream and you had better hope you keep a really solid update cadence and not introduce something you can’t consume later. “Layers on layers means things are composable” - sometimes, but you need more people to know more things to put it together properly. Running everything from QuickStarts is doomed to fail, but I see so much of it. Our config management tools back in the day were crummy as hell, cfengine being the only notable one (which was awful), our monitoring systems required a lot more handholding and truthfully: people were not happy to talk through requirements; which was probably why “devs run production” was appealing. these things have gotten better, but nearly everything else has definitely gotten worse.
raylad 4y ago
And I had all this in place by 1999. Including the multiple databases, the caches, the load balancing, edge processing nodes all over the world, etc. Sure we built a bunch of it ourselves, and we had 4 sysadmins, but please don't tell me that this wasn't already going on over 20 years ago.
hinkley 4y ago
The future is already here, it’s just unevenly distributed. By 2004 we were doing perforce and CI. No Continuous Deployment because we were building on-prem software, but by 2008 I was doing svn and Continuous Delivery (similar problem, not hosting). We would have been if it had been appropriate. To be fair though, I think a lot of my coworkers suspect i’m talking out of my ass because they’ve been doing this stuff for 5-8 years and think everyone else has too. As tempting as it sometimes is to launch into a lecture., it wouldn’t really help. Everybody knows what they know, and they’re going to know it until they watch it catch on fire and then they will either want to hear about alternatives or they’ll share ideas for fixing it, not realizing they’re repeating something you told them two years ago. Hey that’s a great idea, wish I’d thought of it.

_vertigo 4y ago

terpans 4y ago
Indeed all of those things existed and they weren’t in widespread use because 99% of organization did not need them. And it hasn't changed. > 2 sysadmins were absolutely not managing “large scale deployments with multiple data centers” Please don't make things up because I was there. > But today it’s actually super plausible that 2 or 3 Devops with modern tooling can manage all of that. Not really.
- _vertigo 4y ago
  - deleted 4y ago
    [deleted]
  - dijit 4y ago
    I was there too and some things were better, some thing were worse. The problem is that we’re being sold a lie and it’s hard to swallow. “Cloud will save you on staffing costs” - no, it just means you have specialists working on proprietary solutions, just like IBM mainframes or storage appliances or load-balancers/traffic-shapers of yore. “This technology will make rollouts easier” - until it breaks down and is not easy to put together again, you’re at the mercy of your upstream and you had better hope you keep a really solid update cadence and not introduce something you can’t consume later. “Layers on layers means things are composable” - sometimes, but you need more people to know more things to put it together properly. Running everything from QuickStarts is doomed to fail, but I see so much of it. Our config management tools back in the day were crummy as hell, cfengine being the only notable one (which was awful), our monitoring systems required a lot more handholding and truthfully: people were not happy to talk through requirements; which was probably why “devs run production” was appealing. these things have gotten better, but nearly everything else has definitely gotten worse.

raylad 4y ago

And I had all this in place by 1999. Including the multiple databases, the caches, the load balancing, edge processing nodes all over the world, etc. Sure we built a bunch of it ourselves, and we had 4 sysadmins, but please don't tell me that this wasn't already going on over 20 years ago.

hinkley 4y ago

The future is already here, it’s just unevenly distributed. By 2004 we were doing perforce and CI. No Continuous Deployment because we were building on-prem software, but by 2008 I was doing svn and Continuous Delivery (similar problem, not hosting). We would have been if it had been appropriate. To be fair though, I think a lot of my coworkers suspect i’m talking out of my ass because they’ve been doing this stuff for 5-8 years and think everyone else has too. As tempting as it sometimes is to launch into a lecture., it wouldn’t really help. Everybody knows what they know, and they’re going to know it until they watch it catch on fire and then they will either want to hear about alternatives or they’ll share ideas for fixing it, not realizing they’re repeating something you told them two years ago. Hey that’s a great idea, wish I’d thought of it.