4 ms·

One good DevOps person can achieve much more than a single sysadmin. Most old school sysadmins were doing everything manually with relatively little automation.

One good DevOps person can achieve much more than a single sysadmin. Most old school sysadmins were doing everything manually with relatively little automation. You are certainly correct about the the overly complex AWS reference architectures. I've seen relatively simple applications with just as much infrastructure code (generally "terraform", occasionally CloudFormation JSON) as application code. It's crazy.

dijit 4y ago
You’re quite mistaken on the first point. Most automation that you know of as “devops tools” are borne from sysadmins. Terraform was written by a sysadmin; puppet, ansible, saltstack and cfengine too. It is revisionist to think sysadmins were not automating their jobs. Old school sysadmins used to know C, bash and Perl. New school devops just traded that for a handful of DSLs and Python. I’m sure some technicians working the IT department managed to get by running scripts created by sysadmins and claiming not to code, but it was definitely the common case that 20 years ago sysadmins could code and worked tirelessly to “automate themselves out of a job” (literally a mantra i was told as a sysadmin 15 years ago)
- icedchai 4y ago
  I'm sure we all had different experiences with old school sysadmins. The ones I'm familiar with (early 90's, ISP industry) could do shell scripting and some perl. C was way out of their skill sets, except for a bit of copy-and-paste. The people who wrote tools like terraform and ansible were engineers with system administration skills. Those are very rare.
  - dijit 4y ago
    I’m biased. I worked in teams with such people basically my whole career. The pay increased, the titles and tools changed but the mentality didn’t. People just started beating their chest about devops and pooping on the legacy of sysadmins which is what most devops/SRE are. Sounds like they were automating though. “Cattle not pets” was a sysadmin mantra, but the business wanted pets most of the time.
    - icedchai 4y ago
      There was certainly some automation being done by the sysadmins I was familiar with (I did say "relatively little" in my original post, meaning compared to today!) It was definitely more pets than cattle at the places I was familiar with. These were small-ish companies, and the business didn't care as long as the systems worked.
      - dijit 4y ago
        Sounds like a “devops” in that same position would be doing the same then.
- jen20 4y ago
  > Terraform was written by a sysadmin I’m with you on most of your post, but this is straight up wrong.
  - dijit 4y ago
    The original author of terraform (Mitchel Hashimoto) was not an Operations Engineer before writing Terraform? https://www.linkedin.com/in/mitchellh/ https://www.linkedin.com/in/mitchellh/ Or are you saying that he's not the author? Commits seem to suggest otherwise. commit fb9c58f0e20a85a5358142ddb39a2f74d3bbe69b Author: Mitchell Hashimoto <mitchell.hashimoto@gmail.com> Date: Fri May 23 11:03:38 2014 -0700 config: better error message commit 089822a36f0a5e4c583ab401b7496b48bfa31d65 Author: Mitchell Hashimoto <mitchell.hashimoto@gmail.com> Date: Fri May 23 10:52:19 2014 -0700 config: some comments commit ec3f72703c82f50787ef86498d856a5798900ad8 Author: Mitchell Hashimoto <mitchell.hashimoto@gmail.com> Date: Thu May 22 16:56:28 2014 -0700 Initial work on config commit 649cf336e86da10910f6e50ea3cfaad507a2e336 Author: Mitchell Hashimoto <mitchell.hashimoto@gmail.com> Date: Wed May 21 16:28:53 2014 -0700 Initial commit
    - arinlen 4y ago
      > (...) an Operations Engineer (...) An operations engineer and a sysadmin are two entirely different roles. A system administrator administers systems while an operations engineer dedicated their time writing code to automate processes.
      - dijit 4y ago
        You're conflating an IT Technician and a Sysadmin. Sysadmins automate their work. Operations Engineer is just another title for sysadmin. Systems Engineer is also another title for the same thing. Platform Engineering (depending on where you are) is yet another variation. If you're responsible for the operating system, monitoring, clustering, hardware etc; then you're firmly in "sysadmin" land.
        4y ago
Melatonic 4y ago
Not necessarily true - many were doing tons of scripting.
jjav 4y ago
> Most old school sysadmins were doing everything manually with relatively little automation. That's not true of any place I experienced in the early to late 90s. If you meant earlier, perhaps, I wasn't there. The growth of perl, for example, was in great part from the sysadmin community automating everything.
- icedchai 4y ago
  Most of the scripts I was familiar with from that time were one-offs. The code wasn't very reusable. They were automating a task on one machine. Today's "DevOps" are automating things across N machines. It is a matter of scale.
  - jjav 4y ago
    > They were automating a task on one machine. Remember that in those days you likely only had one machine. By "you" I mean the whole department. Everyone was logged into it and it handled email, talk, documents, compilation and debugging, etc. That doesn't take away from the fact that the sysadmins were automating everything they needed to do, in the enviornment that existed.
    - doctor_eval 4y ago
      Exactly - I used to be the sysadm on a single machine in a medical laboratory in the 90s. It was a DG Aviion and with the storage unit took up a whole room. Maybe 300 people used it simultaneously. The idea that I would need to automate the installation of an OS and applications on a fleet of machines was never contemplated because it didn’t make sense. I had one machine and OS upgrades arrived in the post every 6 months or so - on cartridge tape. It was about need, not competence.
      - icedchai 4y ago
        The places I'm talking about were mid 90's, dozens of AlphaServers, Sun Sparcs, HP/UX systems, IBM AIX machines, etc. It was more than a single machine, but less than 50. There was shocking little automation. Everything was a pet, with crazy NFS mounts all over the place, custom scripts in /usr/local/etc not in version control, etc. If there was ever a power loss, it was a pain getting everything back up.
boltzmann-brain 4y ago
I mean, you're not wrong in that there's definitely a massive difference between the power of tools that a devops engineer can use today vs what a sys admin had access to back in the day before the devops movement was established (and that was a clear break point that was announced all over the likes of HN and there was a manifesto and all that). But nowadays if you want to manage your own metal then you can use those tools as well. You can still run hypervisors VMs on your own servers. You're just not paying Amazon 6x the price of what it would cost you, and you're not relegated to low-performing compute, memory, and IO elements that Amazon standardized on.

dijit 4y ago

You’re quite mistaken on the first point. Most automation that you know of as “devops tools” are borne from sysadmins. Terraform was written by a sysadmin; puppet, ansible, saltstack and cfengine too. It is revisionist to think sysadmins were not automating their jobs. Old school sysadmins used to know C, bash and Perl. New school devops just traded that for a handful of DSLs and Python. I’m sure some technicians working the IT department managed to get by running scripts created by sysadmins and claiming not to code, but it was definitely the common case that 20 years ago sysadmins could code and worked tirelessly to “automate themselves out of a job” (literally a mantra i was told as a sysadmin 15 years ago)

icedchai 4y ago
I'm sure we all had different experiences with old school sysadmins. The ones I'm familiar with (early 90's, ISP industry) could do shell scripting and some perl. C was way out of their skill sets, except for a bit of copy-and-paste. The people who wrote tools like terraform and ansible were engineers with system administration skills. Those are very rare.
- dijit 4y ago
  I’m biased. I worked in teams with such people basically my whole career. The pay increased, the titles and tools changed but the mentality didn’t. People just started beating their chest about devops and pooping on the legacy of sysadmins which is what most devops/SRE are. Sounds like they were automating though. “Cattle not pets” was a sysadmin mantra, but the business wanted pets most of the time.
  - icedchai 4y ago
    There was certainly some automation being done by the sysadmins I was familiar with (I did say "relatively little" in my original post, meaning compared to today!) It was definitely more pets than cattle at the places I was familiar with. These were small-ish companies, and the business didn't care as long as the systems worked.
    - dijit 4y ago
      Sounds like a “devops” in that same position would be doing the same then.
jen20 4y ago
> Terraform was written by a sysadmin I’m with you on most of your post, but this is straight up wrong.
- dijit 4y ago
  The original author of terraform (Mitchel Hashimoto) was not an Operations Engineer before writing Terraform? https://www.linkedin.com/in/mitchellh/ https://www.linkedin.com/in/mitchellh/ Or are you saying that he's not the author? Commits seem to suggest otherwise. commit fb9c58f0e20a85a5358142ddb39a2f74d3bbe69b Author: Mitchell Hashimoto <mitchell.hashimoto@gmail.com> Date: Fri May 23 11:03:38 2014 -0700 config: better error message commit 089822a36f0a5e4c583ab401b7496b48bfa31d65 Author: Mitchell Hashimoto <mitchell.hashimoto@gmail.com> Date: Fri May 23 10:52:19 2014 -0700 config: some comments commit ec3f72703c82f50787ef86498d856a5798900ad8 Author: Mitchell Hashimoto <mitchell.hashimoto@gmail.com> Date: Thu May 22 16:56:28 2014 -0700 Initial work on config commit 649cf336e86da10910f6e50ea3cfaad507a2e336 Author: Mitchell Hashimoto <mitchell.hashimoto@gmail.com> Date: Wed May 21 16:28:53 2014 -0700 Initial commit
  - arinlen 4y ago
    > (...) an Operations Engineer (...) An operations engineer and a sysadmin are two entirely different roles. A system administrator administers systems while an operations engineer dedicated their time writing code to automate processes.
    - dijit 4y ago
      You're conflating an IT Technician and a Sysadmin. Sysadmins automate their work. Operations Engineer is just another title for sysadmin. Systems Engineer is also another title for the same thing. Platform Engineering (depending on where you are) is yet another variation. If you're responsible for the operating system, monitoring, clustering, hardware etc; then you're firmly in "sysadmin" land.
      - 4y ago

Melatonic 4y ago

Not necessarily true - many were doing tons of scripting.

jjav 4y ago

> Most old school sysadmins were doing everything manually with relatively little automation. That's not true of any place I experienced in the early to late 90s. If you meant earlier, perhaps, I wasn't there. The growth of perl, for example, was in great part from the sysadmin community automating everything.

icedchai 4y ago
Most of the scripts I was familiar with from that time were one-offs. The code wasn't very reusable. They were automating a task on one machine. Today's "DevOps" are automating things across N machines. It is a matter of scale.
- jjav 4y ago
  > They were automating a task on one machine. Remember that in those days you likely only had one machine. By "you" I mean the whole department. Everyone was logged into it and it handled email, talk, documents, compilation and debugging, etc. That doesn't take away from the fact that the sysadmins were automating everything they needed to do, in the enviornment that existed.
  - doctor_eval 4y ago
    Exactly - I used to be the sysadm on a single machine in a medical laboratory in the 90s. It was a DG Aviion and with the storage unit took up a whole room. Maybe 300 people used it simultaneously. The idea that I would need to automate the installation of an OS and applications on a fleet of machines was never contemplated because it didn’t make sense. I had one machine and OS upgrades arrived in the post every 6 months or so - on cartridge tape. It was about need, not competence.
    - icedchai 4y ago
      The places I'm talking about were mid 90's, dozens of AlphaServers, Sun Sparcs, HP/UX systems, IBM AIX machines, etc. It was more than a single machine, but less than 50. There was shocking little automation. Everything was a pet, with crazy NFS mounts all over the place, custom scripts in /usr/local/etc not in version control, etc. If there was ever a power loss, it was a pain getting everything back up.

boltzmann-brain 4y ago

I mean, you're not wrong in that there's definitely a massive difference between the power of tools that a devops engineer can use today vs what a sys admin had access to back in the day before the devops movement was established (and that was a clear break point that was announced all over the likes of HN and there was a manifesto and all that). But nowadays if you want to manage your own metal then you can use those tools as well. You can still run hypervisors VMs on your own servers. You're just not paying Amazon 6x the price of what it would cost you, and you're not relegated to low-performing compute, memory, and IO elements that Amazon standardized on.