5 ms·

I am not sure you can do this I believe sysadmin/sre/devops person wears too many hats. What are you going to test ?

johnny_snq 10y ago
We are going to give real life scenarios based on the various roles you are testing with.(basicly a ssh login to the test machine) And we plan to asses the candidates way of solving the problems based on what commands and what process it uses to identify and solve the problem.
- globalgobble 10y ago
  Please allow me my honesty but that's a stupid way of thinking about candidates and the roles. Or I do not get it. So you are basically going to judge the candidate based on what criteria ? If they use vi instead of cat to append to a file ? So decide what is the best way of solving the problem ? Or you give +1pt for using tail instead of opening the whole file in VI ? Are you aware of the fact that there are many types of systems and products that sysadmins use ? They do not behave the same. It makes no sense to use "top" on Solaris even the number for "uptime" reported by Solaris are different. What about Windows sysadmins ? What about stuff like Veritas ? What about docker ?
  - languagewars 10y ago
    When I used to work in OS support my actions were optimized for the assumptions of a default minimal install and being able to describe actions to a person of questionable knowledge over the phone and interpret what they would most likely try to read back at me. The certifications were along those lines too.. While the exact nature of the certifications was ridiculous and would be even more ridiculous in terms of judging a general user's skills, dropping an administrator on minimal installs with broken ttys, etc makes a lot of sense. If you can't recover a system when vi is not an option then you have a problem, if you spend a lot of time recovering a good term when you could have used tail, then you are also not ideal. The only suggestion I would make is that a good tool is present and working then I don't expect a penalty. I expect tools to not be present accordance with standard minimalist setups and to pay extreme penalties if I need to call a package manager or move to a less minimalist install to complete the task.
    - globalgobble 10y ago
      I strongly disagree. We should not test edge cases and specific scenarios. We are trying to test the candidate and his abilities in daily tasks. If I say I use vi instead of "commands" I will probably solve most of the "text" editing tasks faster than an admin that only knows how to get around using commands. I can come up with tons of scenarios when where you do not really know the internals you can get stuck easily. Like if I do chmod -x on the chmod binary. Does it prove anything if the candidate is not able to solve such task ?
      - CmdrSprinkles 10y ago
        While I mostly agree with you, I can speak from experience with how we handled the first point: We were interviewing a new admin for the sysadmin/maintenance team and her answers to "how would you solve this problem?" were "Open that file and check that these variables are set and, if not, set them". One of the more jerkish people there went down the rabbit hole of "How are you going to edit that file? What if you don't have vim? Okay, what if you don't have emacs? Nano is gone too" leading to the response that made us hire her: "Well, in that case the system is probably completely hosed and we need to stop configuring things and focus on recovery. But I think you are looking for me to tell you a sequence of commands that involve cat. If you give me five minutes I can check stack overflow and get back to you with those" Which I personally think should be the response for stuff like that. Understand there is an alternative and a way to resolve the corner cases and know how to get that info if needed.
        globalgobble 10y ago
        Brilliant answer :-). I totally agree with you. But this is the problem..you cannot implement such a thing to a tool like sysadminarena. Actually I believe being a sysops/devops or whatever you want to call it is one of the "most" challenging jobs in the IT industry. You get zillions of technologies you have to administer and maintain but it's impossible to master all of them. What makes a good admin is the ability to learn,adapt,ask for a help and knowing where to look for an answer. It's easy to look up the parameters if you know the fundamentals. Interviewing for me as a devops person is one of the most frustrating experiences. You are often asked questions that are specific to the environment of a company you are interviewing with. It's unbelievable how many times I've been asked about parameters of tar, du etc. It's totally stupid. And imagine interviewing with a company that uses only a specific subset of technologies and they heavily emphasize on that.
  - johnny_snq 10y ago
    We are aware that there are more ways to skin a cat. I think it's fair to evaluate a candidate based on wether if he can solve an issue or not and the amount of time it took. I agree it would be stupid to score based on individual commands and we didn't plan to go that low level. But we plan to count the number of commands with typos or wrong parameters. Don't you agree this would be a way to differentiate between 2 people that solved the same problems in the same amount of time?
    - globalgobble 10y ago
      Sorry I have to disagree here as well. I do not think the time or number of commands or typos should be measured. I do not think you can do any quantification here. I believe a lot of sysadmins (talking about unix) do w;uptime;inspect syslog as soon as they connect to the system. Most of the time it's just not necessary to do this but for a lot of sysadmins it's a habit that actually saved their asses several times. Does it make sense for a sysadmin to know a numeric notation for chmod ? Probably. Would I ask that on an interview ? Nope If you need to ask why then you have probably never worked in a heterogeneous environment.
- throwawayosiu1 10y ago
  1. How exactly are the sysadmins graded? (because in the end, let's be real...this tool will most probably be used by HR folks to filter out candidates and they'd not care too much if you send over a screencast/replay of what the candidate did) 1.1) Sys admins generally have their own set of tools (which would be customized per company I guess)...would these be installed by default? (I'm assuming you are provisioning docker containers / candidate / test and then destroying them after use). 1.2) I've used the ones that are sent for developers and it's been a pretty terrible experience at best (it's buggy, won't accept valid code etc.). If this is the case for sysadmins, it's even more worse (but then again, I guess you can put a spin on it and claim it's for the best as they'd have to deal with unpredictiability). 2. How will different technologies be integrated to test candidates on? (for example, testing knowledge on multi node kubernetes cluster and so on).
  - johnny_snq 10y ago
    1. I'm thinking the best grading sistem would be yes / no ? rigth ? If you manage to solve the problem and tests pass you pass. 1.1 If you get root access on a machine you have the freedom to add whatever tools you feel fine. 1.2 I don't think doing a terminal emulation is the way to go. 2. Advanced tests are on our plans but we haven't got everything figured out yet.

We are going to give real life scenarios based on the various roles you are testing with.(basicly a ssh login to the test machine) And we plan to asses the candidates way of solving the problems based on what commands and what process it uses to identify and solve the problem.

globalgobble 10y ago
Please allow me my honesty but that's a stupid way of thinking about candidates and the roles. Or I do not get it. So you are basically going to judge the candidate based on what criteria ? If they use vi instead of cat to append to a file ? So decide what is the best way of solving the problem ? Or you give +1pt for using tail instead of opening the whole file in VI ? Are you aware of the fact that there are many types of systems and products that sysadmins use ? They do not behave the same. It makes no sense to use "top" on Solaris even the number for "uptime" reported by Solaris are different. What about Windows sysadmins ? What about stuff like Veritas ? What about docker ?
- languagewars 10y ago
  When I used to work in OS support my actions were optimized for the assumptions of a default minimal install and being able to describe actions to a person of questionable knowledge over the phone and interpret what they would most likely try to read back at me. The certifications were along those lines too.. While the exact nature of the certifications was ridiculous and would be even more ridiculous in terms of judging a general user's skills, dropping an administrator on minimal installs with broken ttys, etc makes a lot of sense. If you can't recover a system when vi is not an option then you have a problem, if you spend a lot of time recovering a good term when you could have used tail, then you are also not ideal. The only suggestion I would make is that a good tool is present and working then I don't expect a penalty. I expect tools to not be present accordance with standard minimalist setups and to pay extreme penalties if I need to call a package manager or move to a less minimalist install to complete the task.
  - globalgobble 10y ago
    I strongly disagree. We should not test edge cases and specific scenarios. We are trying to test the candidate and his abilities in daily tasks. If I say I use vi instead of "commands" I will probably solve most of the "text" editing tasks faster than an admin that only knows how to get around using commands. I can come up with tons of scenarios when where you do not really know the internals you can get stuck easily. Like if I do chmod -x on the chmod binary. Does it prove anything if the candidate is not able to solve such task ?
    - CmdrSprinkles 10y ago
      While I mostly agree with you, I can speak from experience with how we handled the first point: We were interviewing a new admin for the sysadmin/maintenance team and her answers to "how would you solve this problem?" were "Open that file and check that these variables are set and, if not, set them". One of the more jerkish people there went down the rabbit hole of "How are you going to edit that file? What if you don't have vim? Okay, what if you don't have emacs? Nano is gone too" leading to the response that made us hire her: "Well, in that case the system is probably completely hosed and we need to stop configuring things and focus on recovery. But I think you are looking for me to tell you a sequence of commands that involve cat. If you give me five minutes I can check stack overflow and get back to you with those" Which I personally think should be the response for stuff like that. Understand there is an alternative and a way to resolve the corner cases and know how to get that info if needed.
      - globalgobble 10y ago
        Brilliant answer :-). I totally agree with you. But this is the problem..you cannot implement such a thing to a tool like sysadminarena. Actually I believe being a sysops/devops or whatever you want to call it is one of the "most" challenging jobs in the IT industry. You get zillions of technologies you have to administer and maintain but it's impossible to master all of them. What makes a good admin is the ability to learn,adapt,ask for a help and knowing where to look for an answer. It's easy to look up the parameters if you know the fundamentals. Interviewing for me as a devops person is one of the most frustrating experiences. You are often asked questions that are specific to the environment of a company you are interviewing with. It's unbelievable how many times I've been asked about parameters of tar, du etc. It's totally stupid. And imagine interviewing with a company that uses only a specific subset of technologies and they heavily emphasize on that.
- johnny_snq 10y ago
  We are aware that there are more ways to skin a cat. I think it's fair to evaluate a candidate based on wether if he can solve an issue or not and the amount of time it took. I agree it would be stupid to score based on individual commands and we didn't plan to go that low level. But we plan to count the number of commands with typos or wrong parameters. Don't you agree this would be a way to differentiate between 2 people that solved the same problems in the same amount of time?
  - globalgobble 10y ago
    Sorry I have to disagree here as well. I do not think the time or number of commands or typos should be measured. I do not think you can do any quantification here. I believe a lot of sysadmins (talking about unix) do w;uptime;inspect syslog as soon as they connect to the system. Most of the time it's just not necessary to do this but for a lot of sysadmins it's a habit that actually saved their asses several times. Does it make sense for a sysadmin to know a numeric notation for chmod ? Probably. Would I ask that on an interview ? Nope If you need to ask why then you have probably never worked in a heterogeneous environment.
throwawayosiu1 10y ago
1. How exactly are the sysadmins graded? (because in the end, let's be real...this tool will most probably be used by HR folks to filter out candidates and they'd not care too much if you send over a screencast/replay of what the candidate did) 1.1) Sys admins generally have their own set of tools (which would be customized per company I guess)...would these be installed by default? (I'm assuming you are provisioning docker containers / candidate / test and then destroying them after use). 1.2) I've used the ones that are sent for developers and it's been a pretty terrible experience at best (it's buggy, won't accept valid code etc.). If this is the case for sysadmins, it's even more worse (but then again, I guess you can put a spin on it and claim it's for the best as they'd have to deal with unpredictiability). 2. How will different technologies be integrated to test candidates on? (for example, testing knowledge on multi node kubernetes cluster and so on).
- johnny_snq 10y ago
  1. I'm thinking the best grading sistem would be yes / no ? rigth ? If you manage to solve the problem and tests pass you pass. 1.1 If you get root access on a machine you have the freedom to add whatever tools you feel fine. 1.2 I don't think doing a terminal emulation is the way to go. 2. Advanced tests are on our plans but we haven't got everything figured out yet.