5 ms·

we interview people whose only prior work experience is "Line of Business .NET Developer" I'm assuming you are not interviewing every or even most.NET develope

by zaidf 13y ago

we interview people whose only prior work experience is "Line of Business .NET Developer" I'm assuming you are not interviewing every or even most.NET developers and trying your luck to find the one who can write that great exploit. So, from all the candidates with resumes that would be considered subpar before the process overhaul, how do you determine which ones have potential to write that great exploit?

tptacek 13y ago
That's what a work-sample test is. Candidates here do a series of challenges that are quantifiable (scored), and calibrated to match the day-to-day of working here. We don't ever wonder whether a .NET developer is capable of reverse-engineering a protocol and finding security bugs in it, because we invented a series of protocols and a series of security bugs, and we make candidates try to do that.
- zaidf 13y ago
  Ah, I thought the test was given during the in-person interview. We recently started putting sales candidates on the phone to cold call. The most critical thing we measure is how many minutes they spend on the phone. It's been a pretty strong signal of how they would perform if we were to hire them.
  - tptacek 13y ago
    That is a classic work-sample test.
  - mmorris 13y ago
    This might be obvious to those with more sales experience, but can you explain what exactly you're looking for in regards to "how many minutes they spend on the phone"? In other words, is the success of the sales people about spending more time on the phone in total? Or is it about spending more or less time per call? Thanks in advance!
    - zaidf 13y ago
      This might be obvious to those with more sales experience, but can you explain what exactly you're looking for in regards to "how many minutes they spend on the phone"? Our sales process is based on our reps cold calling businesses. We learned after a couple of mishirings that the most difficult part of the job is being able to manage emotion and taking rejection. For example, as a rep it is completely normal to be rejected in some manner 50 times a day. So if you get discouraged or upset after your third rejection, it is unlikely that you would have the motivation or drive to make more calls and therefore, you're likely not the right fit. All of this is usually summed up in the metric time spent talking across all calls. While ultimately the time on the phone translates into revenue, we like to emphasize factors that is in our control. We can't control whether someone ultimately signs up or not but we do control how much break we take in between calls.
  - fragmede 13y ago
    Sales is a different beast. If you need 2 sales guys, hire 3, give them a month and fire the worst one. That is horrifying to me as a developer, but apparently this is fairly standard for sales.
    - ahomescu1 13y ago
      First place gets a Cadillac, second place gets steak knives, the rest get fired? (a quote from Glengarry Glen Ross)
    - ghaff 13y ago
      That's a BIT of an overstatement depending upon the particulars. Outside sales for complex products can require a quarter or two ramp-up so you want to be somewhat selective. If you're hiring someone for that sort of position though they probably have some sort of track record. For an inside sales position, you're not all that far off though. As I've seen it put in another discussion, "sales managers don't have any trouble firing people."
- emillon 13y ago
  If applying for Matasano is as fun as microcorruption I might as well apply there. Do you consider H1B candidates?
herge 13y ago
Through the Matasano challenges, for one.
- tptacek 13y ago
  Our public challenges are part of outreach, not qualification. We are very worried about drawing conclusions from them in our hiring process, because those challenges are public and people often work together on them.
  - spenuke 13y ago
    The public challenges seem to embrace newcomers. Assuming someone does well on their own - should that person expect to do well in the hiring process? Or is there a significant difference in skill set / level between the public an private tests?
  - Rami114 13y ago
    Don't stop, thanks to the two public programs you've run so far I've: * Got into Ruby and Go (trying to get a feel for what language felt most comfortable for bit manip.) * Got into assembly (for fun so far) * Got a good appreciation for bit-level operations * A bazillion more little facts and insights that help in daily work life I bet if you polled how it's helped other people there'd be all kinds of great examples. For work, I'd say the best impact has been a personal drive to pentest our own product, resulting in a lot of bugs found and fixed. That included managing to root our servers via a product flaw, which was scary and amazingly fun. So yeah, thumbs up, don't stop now (please). As for the above catering to newcomers, sure but it ramps up incredibly and pushes some boundaries if you've never touched that stuff before.

by zaidf 13y ago

tptacek 13y ago

That's what a work-sample test is. Candidates here do a series of challenges that are quantifiable (scored), and calibrated to match the day-to-day of working here. We don't ever wonder whether a .NET developer is capable of reverse-engineering a protocol and finding security bugs in it, because we invented a series of protocols and a series of security bugs, and we make candidates try to do that.

zaidf 13y ago
Ah, I thought the test was given during the in-person interview. We recently started putting sales candidates on the phone to cold call. The most critical thing we measure is how many minutes they spend on the phone. It's been a pretty strong signal of how they would perform if we were to hire them.
- tptacek 13y ago
  That is a classic work-sample test.
- mmorris 13y ago
  This might be obvious to those with more sales experience, but can you explain what exactly you're looking for in regards to "how many minutes they spend on the phone"? In other words, is the success of the sales people about spending more time on the phone in total? Or is it about spending more or less time per call? Thanks in advance!
  - zaidf 13y ago
    This might be obvious to those with more sales experience, but can you explain what exactly you're looking for in regards to "how many minutes they spend on the phone"? Our sales process is based on our reps cold calling businesses. We learned after a couple of mishirings that the most difficult part of the job is being able to manage emotion and taking rejection. For example, as a rep it is completely normal to be rejected in some manner 50 times a day. So if you get discouraged or upset after your third rejection, it is unlikely that you would have the motivation or drive to make more calls and therefore, you're likely not the right fit. All of this is usually summed up in the metric time spent talking across all calls. While ultimately the time on the phone translates into revenue, we like to emphasize factors that is in our control. We can't control whether someone ultimately signs up or not but we do control how much break we take in between calls.
- fragmede 13y ago
  Sales is a different beast. If you need 2 sales guys, hire 3, give them a month and fire the worst one. That is horrifying to me as a developer, but apparently this is fairly standard for sales.
  - ahomescu1 13y ago
    First place gets a Cadillac, second place gets steak knives, the rest get fired? (a quote from Glengarry Glen Ross)
  - ghaff 13y ago
    That's a BIT of an overstatement depending upon the particulars. Outside sales for complex products can require a quarter or two ramp-up so you want to be somewhat selective. If you're hiring someone for that sort of position though they probably have some sort of track record. For an inside sales position, you're not all that far off though. As I've seen it put in another discussion, "sales managers don't have any trouble firing people."
emillon 13y ago
If applying for Matasano is as fun as microcorruption I might as well apply there. Do you consider H1B candidates?

herge 13y ago

Through the Matasano challenges, for one.

tptacek 13y ago
Our public challenges are part of outreach, not qualification. We are very worried about drawing conclusions from them in our hiring process, because those challenges are public and people often work together on them.
- spenuke 13y ago
  The public challenges seem to embrace newcomers. Assuming someone does well on their own - should that person expect to do well in the hiring process? Or is there a significant difference in skill set / level between the public an private tests?
- Rami114 13y ago
  Don't stop, thanks to the two public programs you've run so far I've: * Got into Ruby and Go (trying to get a feel for what language felt most comfortable for bit manip.) * Got into assembly (for fun so far) * Got a good appreciation for bit-level operations * A bazillion more little facts and insights that help in daily work life I bet if you polled how it's helped other people there'd be all kinds of great examples. For work, I'd say the best impact has been a personal drive to pentest our own product, resulting in a lot of bugs found and fixed. That included managing to root our servers via a product flaw, which was scary and amazingly fun. So yeah, thumbs up, don't stop now (please). As for the above catering to newcomers, sure but it ramps up incredibly and pushes some boundaries if you've never touched that stuff before.