6 ms·

So we agree on the definition of what a sysadmin, systems engineers, devops or whatever people want to call it should be. A sysadmin must program, that's a give

by jvehent 15y ago

So we agree on the definition of what a sysadmin, systems engineers, devops or whatever people want to call it should be. A sysadmin must program, that's a given. When did we forget that exactly ? But why are so many devs shooting at the sysadmins these days ? I never read anything from sysadmins complaining about the astonishingly low level of knowledge of a lot of devs. And they are numerous, willingly ignoring even the most basic performance issues and vomiting thousands of lines of codes, hundreds of tons of libraries, to simple problems. It seems that devs are frustrated to not be able to push their random app to the web in 5 minutes, and throw their love to PaaS like an apple geek discovering the new iphone, blaming the sysadmins for not providing the same thing at their own company without even understanding (or trying to understand) how operating systems work, and how their own code should be built. The real issue here is that when devs and sysadmins DON'T work together, and ignore each others problems, constantly blaming each other, then the result sucks. When the sysadmins are 20% devs, and the devs 20% sysadmins, and they drink their coffees together in the morning, you don't see that kind of problem. They built good, reliable, scalable infrastructure, whether it's in the cloud or in the basement of their house.

eropple 15y ago
I never read anything from sysadmins complaining about the astonishingly low level of knowledge of a lot of devs. They should be. At my current employer we're pretty good about this, but in previous misadventures I've run into a lot of people who don't even understand HTTP or how a web server works (if I have to explain thread-per-connection to a guy who's been doing web development for five years, there's a problem!).
- jrussbowman 15y ago
  I used to think like that. But then, doing web development myself on the side I've realized no, they don't. Languages and frameworks these days pretty well abstract the underlying technology. Developers end up writing some pretty amazing stuff. It's up to the sysadmin to weigh in on what changes are necessary to make it work well enough to succeed. A seasoned developer will know more, because they've been through the wringer a few times. The new guys, not so much. Remember, your developer spends their time making it work on a dev box, or even their own laptop. Performance testing and such is an art that if they had time to learn it... well I guess they'd be a sysadmin.
  - eropple 15y ago
    I am that new-guy web developer (or I was a year ago, anyway, I graduated college in 2010). I am the developer working on a dev box, and I do do performance testing. I knew it then, because it's important. Web frameworks do not meaningfully abstract the underlying workings of a web server; hell, the inability to do so is why C# 5 has the await keyword (to allow for better asynchronous IO) and why Servlet 3 has its own async stuff (not to mention Play Framework, which does all manner of shenanigans under-the-hood!). If you actually give a shit about web development beyond a paycheck, this is stuff you just gotta know. If you don't understand the system, your assumptions and assertions are so much more likely to be untenable that it becomes dangerous. The below questions wouldn't fly at my current employer due to a lot of legacy concerns, but my own list of interview questions would include a number of stack questions in addition to programming questions: -Explain asynchronous I/O to me. -How do you have to structure your code to leverage asynchronous I/O? (Alternately phrased: what happens if you write code that depends on synchronous I/O operations within an asynchronous I/O web framework?) -You've got 9 memcached servers with a cache hit rate of 80% resulting in a database load of 80%. What happens if you naively add another memcached server to your cluster? (a question our VP of engineering actually does ask, and I stole it 'cause it's great) You don't have to be a hardcore nginx source code diver to understand your stack. You just have to understand the principles behind it well enough to make good decisions. A sysadmin's job isn't to audit your code for your terrible understanding of the model on which your stack is predicated.
    - jrussbowman 15y ago
      I think you may have taken my comment in a harsher context than intended. First, I don't think it's good for a developer to have all the crap in their head that I do in mine. They'd be spending way more time pre-optmizing code than they would just getting stuff done. A good developer will write a well thought out TDO that infrastructure and business units can provide feedback to. Your questions about asynchronous i/o and memcached are great, and if memcached was a requirement for a sysadmin at a position it would be a good question to make sure they understand the software. As a sysadmin though, you're also balancing - Can the new system exist on an existing server/chroot/virtual machine - What monitoring is necessary for the application afterwards - Does the application meet all the standard requirements for database access ... out PFC is over 100 steps, this is just some of the bigger ones - Can the sysadmin support the system after go live? The developers job is to create it, the sysadmins job is to keep it running long term. Also, you are correct. The sysadmin shouldn't be auditing your code. The sysadmin should have a stack in place that they know works, and be able to help you do one of 2 things: - Give advice on how your system can best fit in that stack - Help you change the stack to best fit your application Which direction you go on those two choices depends on the nature of the application, how mission critical is, required timelines for launch, budget and more.
ryanpers 15y ago
probably because at nearly every place I've worked at the sysadmin/ops group was the #1 obstacle to getting things done. Sure some of it was "saving you from yourself", but a lot of was pure obstinatance for no other reason than I think sysadmins just cant turn it off anymore... much as your mom says "dont frown your face might get stuck", it appears a lot of people (not just sysadmins, developers too - you know the one, in the corner who is old and surly, been there forever, and hasnt been promoted in about as long). The deck is stacked against sysadmins - developers are the people who deliver business value to leaders, and sysadmins are the ones who cost money and say 'no'. When you are on a tight timetable, the person who says "sure" is the one who gets the high fives... assuming things work out, of course.
- rhizome 15y ago
  Reliability can certainly have a business value.
- Goladus 15y ago
  >sysadmin/ops group was the #1 obstacle to getting things done. This happens when the company discovers that when the system goes down they stop making money. If you've managed to design and build a system to be very resilient and reliable on the chosen platform and has proved to need minimal maintenance, then chances are good you have a very small ops team who doesn't get in the way much.
gaius 15y ago
I never read anything from sysadmins complaining about the astonishingly low level of knowledge of a lot of devs There are plenty of DBAs complaining about the low knowledge level of a lot of devs, e.g. using an ORM instead of writing SQL and wondering why the app runs like a dog.

by jvehent 15y ago

eropple 15y ago

I never read anything from sysadmins complaining about the astonishingly low level of knowledge of a lot of devs. They should be. At my current employer we're pretty good about this, but in previous misadventures I've run into a lot of people who don't even understand HTTP or how a web server works (if I have to explain thread-per-connection to a guy who's been doing web development for five years, there's a problem!).

jrussbowman 15y ago
I used to think like that. But then, doing web development myself on the side I've realized no, they don't. Languages and frameworks these days pretty well abstract the underlying technology. Developers end up writing some pretty amazing stuff. It's up to the sysadmin to weigh in on what changes are necessary to make it work well enough to succeed. A seasoned developer will know more, because they've been through the wringer a few times. The new guys, not so much. Remember, your developer spends their time making it work on a dev box, or even their own laptop. Performance testing and such is an art that if they had time to learn it... well I guess they'd be a sysadmin.
- eropple 15y ago
  I am that new-guy web developer (or I was a year ago, anyway, I graduated college in 2010). I am the developer working on a dev box, and I do do performance testing. I knew it then, because it's important. Web frameworks do not meaningfully abstract the underlying workings of a web server; hell, the inability to do so is why C# 5 has the await keyword (to allow for better asynchronous IO) and why Servlet 3 has its own async stuff (not to mention Play Framework, which does all manner of shenanigans under-the-hood!). If you actually give a shit about web development beyond a paycheck, this is stuff you just gotta know. If you don't understand the system, your assumptions and assertions are so much more likely to be untenable that it becomes dangerous. The below questions wouldn't fly at my current employer due to a lot of legacy concerns, but my own list of interview questions would include a number of stack questions in addition to programming questions: -Explain asynchronous I/O to me. -How do you have to structure your code to leverage asynchronous I/O? (Alternately phrased: what happens if you write code that depends on synchronous I/O operations within an asynchronous I/O web framework?) -You've got 9 memcached servers with a cache hit rate of 80% resulting in a database load of 80%. What happens if you naively add another memcached server to your cluster? (a question our VP of engineering actually does ask, and I stole it 'cause it's great) You don't have to be a hardcore nginx source code diver to understand your stack. You just have to understand the principles behind it well enough to make good decisions. A sysadmin's job isn't to audit your code for your terrible understanding of the model on which your stack is predicated.
  - jrussbowman 15y ago
    I think you may have taken my comment in a harsher context than intended. First, I don't think it's good for a developer to have all the crap in their head that I do in mine. They'd be spending way more time pre-optmizing code than they would just getting stuff done. A good developer will write a well thought out TDO that infrastructure and business units can provide feedback to. Your questions about asynchronous i/o and memcached are great, and if memcached was a requirement for a sysadmin at a position it would be a good question to make sure they understand the software. As a sysadmin though, you're also balancing - Can the new system exist on an existing server/chroot/virtual machine - What monitoring is necessary for the application afterwards - Does the application meet all the standard requirements for database access ... out PFC is over 100 steps, this is just some of the bigger ones - Can the sysadmin support the system after go live? The developers job is to create it, the sysadmins job is to keep it running long term. Also, you are correct. The sysadmin shouldn't be auditing your code. The sysadmin should have a stack in place that they know works, and be able to help you do one of 2 things: - Give advice on how your system can best fit in that stack - Help you change the stack to best fit your application Which direction you go on those two choices depends on the nature of the application, how mission critical is, required timelines for launch, budget and more.

ryanpers 15y ago

probably because at nearly every place I've worked at the sysadmin/ops group was the #1 obstacle to getting things done. Sure some of it was "saving you from yourself", but a lot of was pure obstinatance for no other reason than I think sysadmins just cant turn it off anymore... much as your mom says "dont frown your face might get stuck", it appears a lot of people (not just sysadmins, developers too - you know the one, in the corner who is old and surly, been there forever, and hasnt been promoted in about as long). The deck is stacked against sysadmins - developers are the people who deliver business value to leaders, and sysadmins are the ones who cost money and say 'no'. When you are on a tight timetable, the person who says "sure" is the one who gets the high fives... assuming things work out, of course.

rhizome 15y ago
Reliability can certainly have a business value.
Goladus 15y ago
>sysadmin/ops group was the #1 obstacle to getting things done. This happens when the company discovers that when the system goes down they stop making money. If you've managed to design and build a system to be very resilient and reliable on the chosen platform and has proved to need minimal maintenance, then chances are good you have a very small ops team who doesn't get in the way much.

gaius 15y ago

I never read anything from sysadmins complaining about the astonishingly low level of knowledge of a lot of devs There are plenty of DBAs complaining about the low knowledge level of a lot of devs, e.g. using an ORM instead of writing SQL and wondering why the app runs like a dog.