3 ms·

Sounds like a pretty good shed! Like a lot of pithy commentary on the cloud, this ignores the fact the practical alternative to a shed in Virginia for most busi

Sounds like a pretty good shed! Like a lot of pithy commentary on the cloud, this ignores the fact the practical alternative to a shed in Virginia for most businesses is a shelf in the supply closet. "Oops, Jim Bob tripped over the power cord, guess we won't get any emails until the IT guy shows up" - this used to be a routine experience.

darkwater 1y ago
With a gazillion of shelves, closets, Jims and cables. So if Fortnite's Jim trips on a wire, Canva's Jim is quitely sipping coffee at his desk.
noir_lord 1y ago
Once had a site wide outage (biggish manufacturing company) of the internet and backup servers because one of the women wanted to plug her hair straighteners in for the xmas party. In a surprise to literally no one that happening on the last friday before xmas break got my "We need to secure the main comms cabinet" (which had the backup server and main ingress for WAN and was in a separate building on other side of site) item that I'd been asking about for months to the top of the list. Still one of my favourite "outages" because I got to my desk, turned PC on, no network, walked across the landing into the main office, opened comms cabinet, plugged it back in and was "resolved" before the MD got to my desk.
gspencley 1y ago
> "Oops, Jim Bob tripped over the power cord, guess we won't get any emails until the IT guy shows up" - this used to be a routine experience. You're not entirely wrong, but you're being hyperbolic too. I'm actually curious how old you are / how long you've worked in tech, because I started out pre-cloud and things weren't nearly as bad or as limited as you suggest. First, on-prem servers are not the only alternative to "cloud." Many businesses, including the ones I worked for, did co-location. The companies owned their own bare metal servers, but would rent a rack in a data centre, and certain things - like the network admin - was entirely outsourced to the data centre / hosting company. You could also rent managed bare metal servers (you still can). This means that you can pretty much outsource your entire IT department, but you're still not doing cloud services. Meaning you've got bare metal servers, someone you're paying at the hosting company is handling security updates and troubleshooting. You don't get things like auto-scaling or serverless or other cloud features, but you also don't have to worry about Jim tripping over the power cable either. There's also still virtual servers. Which is basically a VM running on a server that hosts multiple clients. All of this is to say that the alternative is not "cloud" or "box in a closet." The alternative is "cloud" and a ton of different server options: owned, rented, co-located, on-prem, dedicated, virtual, managed v un-managed (outsource IT vs admin your own) and the list goes on and on.
- maccard 1y ago
  We run a subset of our CI workload on on-prem workstations because the cost/performance ratio of consumer hardware is so much higher than servers. 1TB NVMe drive, with a 7950x/i9, 64GB RAM and gigabit networking is < $1000. It actually completes our CI job faster than AWS restarts a gpu instance. 100% of our failure rates with this machine have been "carpet cleaners unplugged the machine" in 2 years. Last year we had nobody in the office (due to carpet cleaning). This year we sent someone in straight after the cleaning to fix it.
- Spivak 1y ago
  But is the distinction meaningful? The alternative to a shed in Virginia is a different shed in Montana? I mean sure there are a lot of different sheds out there but they're all still sheds. They're all shared responsibility models where the line is drawn in different areas, some outages will be because of your fuckup, some will be theirs. Not saying as an industry we shouldn't diversify a little but it doesn't fundamentally change the relationship each company has to their hosting provider.
- SpicyLemonZest 1y ago
  I've never managed IT professionally myself (pre-cloud or otherwise), so a lot of my information comes from family members who do, but my impression is that bare metal rental and colo centers weren't realistic options for any but the most technically sophisticated organizations. I know schools, stores, even research centers who went straight from on-prem to managed cloud with no real consideration for anything in between.
  - gspencley 1y ago
    My first paid position as a software developer was for a small, dot-com startup in Windsor, ON Canada. We co-located in Detroit - which meant border crossings (though this was pre-911 so crossing as a Canadian citizen was easier) - and had just a couple of servers on a rack. We were software engineers and had people who knew what they were doing. So yeah we were technically proficient. But I'm not sure I'd call that company among "the most technically sophisticated of organizations". We were tiny. In fact, when I first started working there, we were working out of a house with a workforce of like 25 people max. When that company went under during the dot-com crash, I started my first business shortly after. It was 2003, I was 21 years-old and this business allowed me to work from home and feed my family until I re-entered the job market in 2018. For 15 years, I was a one-person organization, and because my business operated "free" adult-entertainment websites, bandwidth was my most significant expense. For that reason, even when Cloud became a thing (which it wasn't in 2003), I never migrated because of the bandwidth costs alone. Cloudflare was a major game changer but even it didn't exist when I first started out. There were CDNs like Akamai but they were crazy expensive and out of my league. So at its peak, I had about 12 bare metal servers around the world (all rented from the same hosting company - original called Server Matrix it then became Softlayer and then was bought by IBM and went to shit and is now IBM Cloud). I admin'd those on top of writing and maintaining all of the code and running the business independently with occasional help from my wife. I am obviously very technically competent. I'm a Principal Software Engineer today. But technically sophisticated? There wasn't much sophistication about it. I did bare metal servers because it was the only cost-effective way to run my business. It was attainable and it worked. And it worked in a way that Cloud couldn't when Cloud came on the scene - so I never went Cloud with that operation just due to cost alone.
franz_vlkshp 1y ago
on the other hand, that's a small price to pay to having total control and physical access to your own infrastructure. if the sysadmin did his job properly, an incident like that shouldn't require anything else but to plug the server back in and hit the power switch. but then if he did his job properly, no one but IT should be tripping on power cables to begin with.
- hdgvhicv 1y ago
  My home infrastructure is immune from the “unplug” problem by being hosted on two different 10 year old £30 raspberry pis in different rooms. But apparently that’s too hard for the average.
jacobsenscott 1y ago
Largely mitigated by twist lock sockets.

darkwater 1y ago

With a gazillion of shelves, closets, Jims and cables. So if Fortnite's Jim trips on a wire, Canva's Jim is quitely sipping coffee at his desk.

noir_lord 1y ago

Once had a site wide outage (biggish manufacturing company) of the internet and backup servers because one of the women wanted to plug her hair straighteners in for the xmas party. In a surprise to literally no one that happening on the last friday before xmas break got my "We need to secure the main comms cabinet" (which had the backup server and main ingress for WAN and was in a separate building on other side of site) item that I'd been asking about for months to the top of the list. Still one of my favourite "outages" because I got to my desk, turned PC on, no network, walked across the landing into the main office, opened comms cabinet, plugged it back in and was "resolved" before the MD got to my desk.

gspencley 1y ago

> "Oops, Jim Bob tripped over the power cord, guess we won't get any emails until the IT guy shows up" - this used to be a routine experience. You're not entirely wrong, but you're being hyperbolic too. I'm actually curious how old you are / how long you've worked in tech, because I started out pre-cloud and things weren't nearly as bad or as limited as you suggest. First, on-prem servers are not the only alternative to "cloud." Many businesses, including the ones I worked for, did co-location. The companies owned their own bare metal servers, but would rent a rack in a data centre, and certain things - like the network admin - was entirely outsourced to the data centre / hosting company. You could also rent managed bare metal servers (you still can). This means that you can pretty much outsource your entire IT department, but you're still not doing cloud services. Meaning you've got bare metal servers, someone you're paying at the hosting company is handling security updates and troubleshooting. You don't get things like auto-scaling or serverless or other cloud features, but you also don't have to worry about Jim tripping over the power cable either. There's also still virtual servers. Which is basically a VM running on a server that hosts multiple clients. All of this is to say that the alternative is not "cloud" or "box in a closet." The alternative is "cloud" and a ton of different server options: owned, rented, co-located, on-prem, dedicated, virtual, managed v un-managed (outsource IT vs admin your own) and the list goes on and on.

maccard 1y ago
We run a subset of our CI workload on on-prem workstations because the cost/performance ratio of consumer hardware is so much higher than servers. 1TB NVMe drive, with a 7950x/i9, 64GB RAM and gigabit networking is < $1000. It actually completes our CI job faster than AWS restarts a gpu instance. 100% of our failure rates with this machine have been "carpet cleaners unplugged the machine" in 2 years. Last year we had nobody in the office (due to carpet cleaning). This year we sent someone in straight after the cleaning to fix it.
Spivak 1y ago
But is the distinction meaningful? The alternative to a shed in Virginia is a different shed in Montana? I mean sure there are a lot of different sheds out there but they're all still sheds. They're all shared responsibility models where the line is drawn in different areas, some outages will be because of your fuckup, some will be theirs. Not saying as an industry we shouldn't diversify a little but it doesn't fundamentally change the relationship each company has to their hosting provider.
SpicyLemonZest 1y ago
I've never managed IT professionally myself (pre-cloud or otherwise), so a lot of my information comes from family members who do, but my impression is that bare metal rental and colo centers weren't realistic options for any but the most technically sophisticated organizations. I know schools, stores, even research centers who went straight from on-prem to managed cloud with no real consideration for anything in between.
- gspencley 1y ago
  My first paid position as a software developer was for a small, dot-com startup in Windsor, ON Canada. We co-located in Detroit - which meant border crossings (though this was pre-911 so crossing as a Canadian citizen was easier) - and had just a couple of servers on a rack. We were software engineers and had people who knew what they were doing. So yeah we were technically proficient. But I'm not sure I'd call that company among "the most technically sophisticated of organizations". We were tiny. In fact, when I first started working there, we were working out of a house with a workforce of like 25 people max. When that company went under during the dot-com crash, I started my first business shortly after. It was 2003, I was 21 years-old and this business allowed me to work from home and feed my family until I re-entered the job market in 2018. For 15 years, I was a one-person organization, and because my business operated "free" adult-entertainment websites, bandwidth was my most significant expense. For that reason, even when Cloud became a thing (which it wasn't in 2003), I never migrated because of the bandwidth costs alone. Cloudflare was a major game changer but even it didn't exist when I first started out. There were CDNs like Akamai but they were crazy expensive and out of my league. So at its peak, I had about 12 bare metal servers around the world (all rented from the same hosting company - original called Server Matrix it then became Softlayer and then was bought by IBM and went to shit and is now IBM Cloud). I admin'd those on top of writing and maintaining all of the code and running the business independently with occasional help from my wife. I am obviously very technically competent. I'm a Principal Software Engineer today. But technically sophisticated? There wasn't much sophistication about it. I did bare metal servers because it was the only cost-effective way to run my business. It was attainable and it worked. And it worked in a way that Cloud couldn't when Cloud came on the scene - so I never went Cloud with that operation just due to cost alone.

franz_vlkshp 1y ago

on the other hand, that's a small price to pay to having total control and physical access to your own infrastructure. if the sysadmin did his job properly, an incident like that shouldn't require anything else but to plug the server back in and hit the power switch. but then if he did his job properly, no one but IT should be tripping on power cables to begin with.

hdgvhicv 1y ago
My home infrastructure is immune from the “unplug” problem by being hosted on two different 10 year old £30 raspberry pis in different rooms. But apparently that’s too hard for the average.

jacobsenscott 1y ago

Largely mitigated by twist lock sockets.