3 ms·
Take an actual problem that your team solved. Distill it down to essential elements. Wargame out plausible solutions to it if one didn't have your team's cont
by patio11 10y ago
Take an actual problem that your team solved. Distill it down to essential elements. Wargame out plausible solutions to it if one didn't have your team's context on your particular situation, particularly plausible solutions which would be top-of-mind to a smart, stressed person operating under incredible time pressure. (Your actual IRL solution should probably be one of them, but don't assume it is the only solution, the most likely solution, or the only passing solution.) Take insights out of those wargamed answers (e.g. "Ahh, this is performant", "Ahh, this approach fails loudly when our infrastructure is in a degraded state", "Oof, this creates a SPOF... not desirable here") and write them in a list. Group them by theme. Try to have somewhere between 5 and 10 themes.
For each theme on the list; make 3~5 variants of things which show the insight(s) related to the theme at varying levels of quality/discernment/depth/etc. Assign an appropriate numerical weight to that tier of answer. Your list is now two-dimensional; in teaching circles it would be called "a rubric."
Grade interviews against the rubric. On security, this candidate said X, Y, and Z, which are elements which most closely match a Good answer; 7 of 10 possible points awarded.
Even better, grade written notes taken during interviews against the rubric.
Now, and this is the brutally organizationally difficult part, stop overriding the rubric when it spits out point values which imply next steps regarding a particular candidate that are clearly not what one wanted to do. Instead, fix the scenario and/or rubric.
- cheez 10y agoThis a nice list of interview questions. How does one actually go about getting the experience to answer these with expertise?
- sethev 10y agopatio11's comment reads more like a method for evaluating answers. You present the candidate with a boiled down description of a real problem and then evaluate what they say against the rubric. All that to say you answer the questions by actually having expertise in the subject area of the question.
- cheez 10y agoThat might be true, but same question :)
- umlaut 10y agoBy screwing up, owning it, and fixing it. It's impossible to get real operational knowledge without actually working on it. If you're in school, design competitions are awesome. If you're working in IT already, take responsibility for some processes.
- garysieling 10y agoI would suggest a combination of being a point person to solve production issues (either on a pager or being a developer who fixes problems after an outage), or a more involved side project (try keeping Wordpress running on a VM when something is on HN, etc)
- Afton 10y ago> Wargame out plausible solutions to it if one didn't have your team's context on your particular situation, My least favorite interview ever, was essentially "Implement this generic problem that our team had to solve, but you don't have the context of our infrastructure, and I'll criticize you every time you deviate from the actual solution we came up with because your solution wouldn't work in our context". It really felt like a "read my mind" type of problem. No further point. Just glad to see you point out the relatively obvious point that you need to evaluate a question for someone who doesn't have your current context.
- emeraldd 10y agoA good counter to this problem for the interviewer is to pick something that is extra-domain to your problem space. A previous employer I worked with used "Design the object model of a Chicken" as a starting point. The interviewers would then act as a potential client and we'd go back and forth with the candidate over their approach. Definitely rough on candidates, but it does give a reasonable window into their thought processes and who the interact with other people.
- karmakaze 10y agoWe do this, but in an interactive way, to see the approach taken, additional context given, adjustments made along the way. This really shows how a candidate thinks and if they can on-their-feet. Often we will run out of alternatives before coming to a solution, with many avenues explored. Rarely we will discover a path that may have worked, possibly even better.
- codr4life 10y agoI was put through that once, it sucks. You're completely ignoring all the wrong turns you took to get there, the number of people involved etc. Expecting someone to just jump into your shoes and pull of the same trick is not very constructive. What if you framed the problem wrong? Maybe this is the candidate that would have helped you come up with a superior solution given enough freedom and time?
- Tenhundfeld 10y ago