4 ms·

"It seems like you’ve never worked with critical infra." My entire career has been spent building, and maintaining, critical infra.[1] Further, in my voluntee

by rsync 2y ago

"It seems like you’ve never worked with critical infra." My entire career has been spent building, and maintaining, critical infra.[1] Further, in my volunteer time, I come into contact with medical, dispatch and life-safety systems and equipment built on Windows and my question remains the same: Why is Windows anywhere near critical infra ? Just because it is common doesn't mean it's any less shameful and inadequate. I repeat: We've fully understood these risks and frailties for 25 years. [1] As a craft, and a passion - not because of "exciting career opportunities in IT".

sgarland 2y ago
Is this the rsync.net HN account? If so, lmao @ the comment you replied to. > As a craft, and a passion I believe you’ve nailed the core problem. Many people in tech are not in it because they genuinely love it, do it in their off time, and so on. Companies, doubly so. I get it, you have to make money, but IME, there is a WORLD of difference in ability and self-solving ability between those who love this shit, and those who just do it for the money. What’s worse is that actual fundamental knowledge is being lost. I’ve tried at multiple companies to shift DBs off of RDS / Aurora and onto at the very least, EC2s. “We don’t have the personnel to support that.” “Me. I do this at home, for fun. I have a rack. I run ZFS. Literally everything in this RFC, I know how to do.” “Well, we don’t have anyone else.” And that’s the damn tragedy. I can count on one hand the number of people I know with a homelab who are doing anything other than storing media. But you try telling people that they should know how to administer Linux before they know how to administer a K8s cluster, and they look at you like you’re an idiot.
- nikau 2y ago
  Also a lot of the passionate security people such as myself moved on to other fields as it has just become bullshit artists sucking on the vendors teat and filling out risk matrix sheets, but no accountability when their risk assessments invariably turn out to be wrong.
  - sgarland 2y ago
    That reminds me, I should check Twitter to see the most recent batch of “cybersecurity experts” take on Crowdstrike. Always a good time.
- est31 2y ago
  The old school sysadmins who know technology well are still around but there is increasingly less of them while the demand skyrockets as our species gives computers an increasing number of responsibilities. There is tremendous demand for technology that works well and works reliably. Sure, setting up a database running on an EC2 instance is easy. But do you know all of the settings to make the db safe to access? Do you maintain it well, patch it, replicate it, etc? This can all be done by one of the old school sysadmins. But they are rare to find, and not easy to replace. It's hard to judge from the outside, even if you are an expert in the field. So when the job market doesn't have the amount of sysadmins/devops engineers available, then the cloud offers a good replacement. Even if you as an individual company can solve it by offering more money and having a tougher selection process, this doesn't scale over the entire field, as at that point the whole number of available experts comes in. Aurora is definitely expensive, but there is cheaper alternatives to it. Full disclosure, I'm employed by one of these alternative vendors (Neon). You don't have to use it, but many people do and it makes their life easier. The market is expected to grow a lot. Clouds seem to be one of the ways our industry is standardizing.
  - sgarland 2y ago
    I’m not even a sysadmin, I just learned how to do stuff in Gentoo in the early ‘00s. Undoubtedly there are graybeards who will laugh at the ease of tooling that was available to me. > But do you know all of the settings to make the db safe to access? Do you maintain it well, patch it, replicate it, etc? Yes, but to be fair, I’m a DBRE (and SRE before that). I’m not advocating that someone without fairly deep knowledge attempt to do this in prod at a company of decent size. But your tiny startup? Absolutely; chuck a default install of Postgres or MySQL onto Debian, and optionally tune 2 – 3 settings (shared_buffers, effective_cache_size, and random_page_cost for Postgres; (innodb_buffer_pool_* and sync_array_size for MySQL – the latter isn’t necessary until you have high concurrency, but it also can’t be changed without a restart so may as well). Pick any major backup solution for your DB (Barman for Postgres, XtraBackup for MySQL, etc.), and TEST YOUR BACKUPS. That’s about it. Apply any security patches (or use unattended-upgrades, just be careful) as they’re released, and don’t do anything outside of your distro’s package management. You’ll be fine. Re: Neon, I’ve not used it, but I’ve read your docs extensively. It’s the most interesting Postgres-aaS product I’ve seen, alongside postgres.ai, but you’re (I think) targeting slightly different audiences. I wish you luck!
- rewgs 2y ago
  raises hand you guys hiring? I’ll be proof that there is indeed “anyone else.”
emmelaich 2y ago
Not saying they're sufficient reasons but .. 1. more Windows programmers than Linux so they're cheaper. 2. more third-party software for e.g. reporting, graphing to integrate with 3. no one got fired for buying Microsoft 4. any PC can run Windows; IT departments like that.
sentientslug 2y ago
My comment was tongue in cheek, of course it should not be this way but as you know it oftentimes is.

by rsync 2y ago

sgarland 2y ago

Is this the rsync.net HN account? If so, lmao @ the comment you replied to. > As a craft, and a passion I believe you’ve nailed the core problem. Many people in tech are not in it because they genuinely love it, do it in their off time, and so on. Companies, doubly so. I get it, you have to make money, but IME, there is a WORLD of difference in ability and self-solving ability between those who love this shit, and those who just do it for the money. What’s worse is that actual fundamental knowledge is being lost. I’ve tried at multiple companies to shift DBs off of RDS / Aurora and onto at the very least, EC2s. “We don’t have the personnel to support that.” “Me. I do this at home, for fun. I have a rack. I run ZFS. Literally everything in this RFC, I know how to do.” “Well, we don’t have anyone else.” And that’s the damn tragedy. I can count on one hand the number of people I know with a homelab who are doing anything other than storing media. But you try telling people that they should know how to administer Linux before they know how to administer a K8s cluster, and they look at you like you’re an idiot.

nikau 2y ago
Also a lot of the passionate security people such as myself moved on to other fields as it has just become bullshit artists sucking on the vendors teat and filling out risk matrix sheets, but no accountability when their risk assessments invariably turn out to be wrong.
- sgarland 2y ago
  That reminds me, I should check Twitter to see the most recent batch of “cybersecurity experts” take on Crowdstrike. Always a good time.
est31 2y ago
The old school sysadmins who know technology well are still around but there is increasingly less of them while the demand skyrockets as our species gives computers an increasing number of responsibilities. There is tremendous demand for technology that works well and works reliably. Sure, setting up a database running on an EC2 instance is easy. But do you know all of the settings to make the db safe to access? Do you maintain it well, patch it, replicate it, etc? This can all be done by one of the old school sysadmins. But they are rare to find, and not easy to replace. It's hard to judge from the outside, even if you are an expert in the field. So when the job market doesn't have the amount of sysadmins/devops engineers available, then the cloud offers a good replacement. Even if you as an individual company can solve it by offering more money and having a tougher selection process, this doesn't scale over the entire field, as at that point the whole number of available experts comes in. Aurora is definitely expensive, but there is cheaper alternatives to it. Full disclosure, I'm employed by one of these alternative vendors (Neon). You don't have to use it, but many people do and it makes their life easier. The market is expected to grow a lot. Clouds seem to be one of the ways our industry is standardizing.
- sgarland 2y ago
  I’m not even a sysadmin, I just learned how to do stuff in Gentoo in the early ‘00s. Undoubtedly there are graybeards who will laugh at the ease of tooling that was available to me. > But do you know all of the settings to make the db safe to access? Do you maintain it well, patch it, replicate it, etc? Yes, but to be fair, I’m a DBRE (and SRE before that). I’m not advocating that someone without fairly deep knowledge attempt to do this in prod at a company of decent size. But your tiny startup? Absolutely; chuck a default install of Postgres or MySQL onto Debian, and optionally tune 2 – 3 settings (shared_buffers, effective_cache_size, and random_page_cost for Postgres; (innodb_buffer_pool_* and sync_array_size for MySQL – the latter isn’t necessary until you have high concurrency, but it also can’t be changed without a restart so may as well). Pick any major backup solution for your DB (Barman for Postgres, XtraBackup for MySQL, etc.), and TEST YOUR BACKUPS. That’s about it. Apply any security patches (or use unattended-upgrades, just be careful) as they’re released, and don’t do anything outside of your distro’s package management. You’ll be fine. Re: Neon, I’ve not used it, but I’ve read your docs extensively. It’s the most interesting Postgres-aaS product I’ve seen, alongside postgres.ai, but you’re (I think) targeting slightly different audiences. I wish you luck!
rewgs 2y ago
raises hand you guys hiring? I’ll be proof that there is indeed “anyone else.”

emmelaich 2y ago

Not saying they're sufficient reasons but .. 1. more Windows programmers than Linux so they're cheaper. 2. more third-party software for e.g. reporting, graphing to integrate with 3. no one got fired for buying Microsoft 4. any PC can run Windows; IT departments like that.

sentientslug 2y ago

My comment was tongue in cheek, of course it should not be this way but as you know it oftentimes is.