3 ms·

Thanks for responding. Question 1 is basically how do you define impact? What does a candidate who has not made an impact (or made a negative impact) look like

Thanks for responding. Question 1 is basically how do you define impact? What does a candidate who has not made an impact (or made a negative impact) look like in an interview? The reason I ask is that it strikes me that there are so many intangibles in business outcomes. I'm also curious about how you conduct and benchmark your reference interviewing - I have given lengthy reference interviews for colleagues taking executive roles, but those references were extremely rose-colored. There's no incentive for me to be candid.

mason55 5y ago
(Not the OP) One thing I always ask about but find to be especially helpful with less technical roles is to go into the whole process. Start with what they accomplished but keep digging. What other options did you consider? What were the trade offs? How did you decide your approach was best? What lessons did you learn and how did you apply them in the future? Those questions can all be BSed but at a certain point if you have good answers for them then you at least know what you’re talking about even if it’s all made up. Then I usually combine with some kind of concrete exercise that relates to the actual job. For product managers I like to give some kind of product strategy (either something we’ve done or something we’d never do to avoid people feeling like they are doing free work) and have them put together a product proposal. The more junior they are the more guidance they get and the more it’s a test of turning a discussion of features into clear requirement. If they’re more senior it’s about turning strategy into a clear product. The second part has caught a number of people who said the right things but couldn’t execute, or at least not in a way that was compatible with our culture.
- kritiko 5y ago
  This is really insightful. Is the product strategy problem typically a take home? In my experience, “product whiteboard” exercises have always felt like a bit of a party trick when I’ve encountered them. The discovery phase gets so truncated in order to get to cover design/implementation etc. Basically a Fermi problem but “show me your framework.”
  - mason55 5y ago
    Oh yeah, definitely take home. After we’ve made it pretty far so that the candidate doesn’t feel like we’re just asking them to do an exercise as a form of screening. I also try to frame it as the exercise is instead of two more interviews where people would ask the same questions, and tell them not to spend more than a couple hours, although there’s no time limit. Trying to be really conscious of all the limitations and asks of “take home work” for an interview. I will say, especially as they get more senior, I tell them to make assumptions and document the assumptions they made. It’s part of the job (making good assumptions and communicating them to everyone) and I think it fits well into the framework of a take home product strategy exercise.
rahimnathwani 5y ago
mason55's is spot on: keep digging. If someone really did something, they can easily answer any detailed question about it, including what alternatives they considered/tried. Re #2, you have to control the interview, and come armed with questions to draw people out. 'kritiko mentioned their difficulty defining concrete measures for the outcome of their work. Can you tell me about the impact that's had on the team?' Re: #1, I recommend reading 'Radical Focus' and Move (Patty Azzarello). A PM should be able to define measures of impact, and be able to communicate why they are important. If they can't do that, how could anyone trust them to prioritize features/whatever?

mason55 5y ago

(Not the OP) One thing I always ask about but find to be especially helpful with less technical roles is to go into the whole process. Start with what they accomplished but keep digging. What other options did you consider? What were the trade offs? How did you decide your approach was best? What lessons did you learn and how did you apply them in the future? Those questions can all be BSed but at a certain point if you have good answers for them then you at least know what you’re talking about even if it’s all made up. Then I usually combine with some kind of concrete exercise that relates to the actual job. For product managers I like to give some kind of product strategy (either something we’ve done or something we’d never do to avoid people feeling like they are doing free work) and have them put together a product proposal. The more junior they are the more guidance they get and the more it’s a test of turning a discussion of features into clear requirement. If they’re more senior it’s about turning strategy into a clear product. The second part has caught a number of people who said the right things but couldn’t execute, or at least not in a way that was compatible with our culture.

kritiko 5y ago
This is really insightful. Is the product strategy problem typically a take home? In my experience, “product whiteboard” exercises have always felt like a bit of a party trick when I’ve encountered them. The discovery phase gets so truncated in order to get to cover design/implementation etc. Basically a Fermi problem but “show me your framework.”
- mason55 5y ago
  Oh yeah, definitely take home. After we’ve made it pretty far so that the candidate doesn’t feel like we’re just asking them to do an exercise as a form of screening. I also try to frame it as the exercise is instead of two more interviews where people would ask the same questions, and tell them not to spend more than a couple hours, although there’s no time limit. Trying to be really conscious of all the limitations and asks of “take home work” for an interview. I will say, especially as they get more senior, I tell them to make assumptions and document the assumptions they made. It’s part of the job (making good assumptions and communicating them to everyone) and I think it fits well into the framework of a take home product strategy exercise.

rahimnathwani 5y ago

mason55's is spot on: keep digging. If someone really did something, they can easily answer any detailed question about it, including what alternatives they considered/tried. Re #2, you have to control the interview, and come armed with questions to draw people out. 'kritiko mentioned their difficulty defining concrete measures for the outcome of their work. Can you tell me about the impact that's had on the team?' Re: #1, I recommend reading 'Radical Focus' and Move (Patty Azzarello). A PM should be able to define measures of impact, and be able to communicate why they are important. If they can't do that, how could anyone trust them to prioritize features/whatever?