3 ms·

hey! We currently don't auto-mark and you are spot on, the hiring manager can see exactly what the candidate ran and judge for themself (e.g. some scenarios you

by hkh 6y ago

hey! We currently don't auto-mark and you are spot on, the hiring manager can see exactly what the candidate ran and judge for themself (e.g. some scenarios you can just re-start the machine, but that doesn't solve the root cause). But I think we will need to create a report on how well the candidate did. Based on what we've seen, it seems better to award lots of little points vs a few big points for the "right answer". What would you expect/want to see in a results report? Cheers, Hassan

sgarland 6y ago
Agree on granular points. While a large chunk should certainly be "was it successful," points for effort and/or general knowledge are good. 1. Did they use man pages? While Google and Stackoverflow have the answers to everything, being able to use built-in help is useful. Obviously if they just know how to structure a command, don't detract points here. 2. How clever are they with bash? This one is hard to objectively measure, but for example, someone using awk to a large extent rather than chaining together calls to grep and sed may know more Linux. Again, I don't know that you should lose points if you're following the Linux philosophy of "Each Program Does One Thing Well," but it's an interesting datapoint. 3. Are they aware of the limitations of tools? I didn't do all the exercises so I'm not sure if you have this, but having to find log entries in multi-GB files can quickly demonstrate knowledge. If you blindly pass `grep -i foo` to a giant file, it's going to take quite a while - even more so if the file is compressed, as would be expected for a production system with logrotate. Knowing that the -i flag slows down grep, and that passing a limited charset (e.g. LC_ALL=C) to grep can speed things up is good. 3a. Make them / lead them to using find with xargs or exec. Once again, understanding system calls and how your command string creation can drastically alter performance is a good indicator of skill. I suppose overall, I'd want to see the output of the script they wrote (you don't have to auto-mark, you can just have the report print expected and actual, so fuzziness is up to the manager), the time they took, what man pages they read if any, and top commands issued.

by hkh 6y ago

sgarland 6y ago

Agree on granular points. While a large chunk should certainly be "was it successful," points for effort and/or general knowledge are good. 1. Did they use man pages? While Google and Stackoverflow have the answers to everything, being able to use built-in help is useful. Obviously if they just know how to structure a command, don't detract points here. 2. How clever are they with bash? This one is hard to objectively measure, but for example, someone using awk to a large extent rather than chaining together calls to grep and sed may know more Linux. Again, I don't know that you should lose points if you're following the Linux philosophy of "Each Program Does One Thing Well," but it's an interesting datapoint. 3. Are they aware of the limitations of tools? I didn't do all the exercises so I'm not sure if you have this, but having to find log entries in multi-GB files can quickly demonstrate knowledge. If you blindly pass `grep -i foo` to a giant file, it's going to take quite a while - even more so if the file is compressed, as would be expected for a production system with logrotate. Knowing that the -i flag slows down grep, and that passing a limited charset (e.g. LC_ALL=C) to grep can speed things up is good. 3a. Make them / lead them to using find with xargs or exec. Once again, understanding system calls and how your command string creation can drastically alter performance is a good indicator of skill. I suppose overall, I'd want to see the output of the script they wrote (you don't have to auto-mark, you can just have the report print expected and actual, so fuzziness is up to the manager), the time they took, what man pages they read if any, and top commands issued.