5 ms·

What is the lowers/raises approach? Google isn’t up to the job (pun was not initially intended, but here we are).

by jsinai 6y ago

btgeekboy 6y ago
I think it’s the candidate vs team. Is the candidate better or worse than half of the team?
- lostcolony 6y ago
  Yep. "does this candidate raise the bar, or lower it". I.e., they have to be better than the average at the company already. As decided by you, from 50 minutes of interaction in a completely artificial, stressful situation.
a-priori 6y ago
I don't know what company they're referring to but at Amazon, the criteria for hiring someone is that interviewee would be better than 50% of the people currently doing that role at the company. They should 'raise the bar' in order to be hired.
- volkk 6y ago
  how do you even gauge that? it sounds like you'd need to keep re-interviewing the current people there to make it relative and fair. which of course isnt going to happen so "raising" the bar doesnt make any sense whatsoever
- mattkrause 6y ago
  Doesn't this make it (literally) exponentially harder to hire people?
  - decafninja 6y ago
    I think that's part of the point. i.e. the usual line about wanting to avoid false positives (bad engineers putting on an act) even if it means sacrificing many false negatives (good engineers who suck at interviewing, or even just simply wasn't having a good day on the day of the interview). Ironically the modern technical whiteboarding interview often does require the candidate to effectively act and put on a show,
    - mattkrause 6y ago
      I knocked together a little simulation. Suppose programming ability is normally distributed with a mean of 100 and and SD of 15 (values cribbed from IQ). A team starts with 10 people drawn from that population. More applicants are drawn from that population: if an applicant's ability is above the median of the current team, they are hired, replacing a randomly-selected member of the team; otherwise, we pass. At the first timestep, half the applicants get hired right away. It takes a few hundred total failed interviews to make the 25th hire, and a few hundred thousand(!) for the 100th hire. These have very long tails too: the longest run (of 1000) took 800M total interviews to hire 100 people. There's no way this rule is actually being applied in practice.
      - theflyinghorse 6y ago
        raising the bar would also mean that their pool of people who are merely good enough gets very small very fast and Amazon just doesn't have the reputation to attract the absolute best
vishnugupta 6y ago
Lowers/raises the bar. As part of interview feedback an interviewer is supposed to rate the interviewee as raises/meets/lowers. Typically, just one lowers in an a loop of 6 interviews is sufficient to reject. On the other hand, a loop with all "meets" isn't a sufficient either. It needs about 3-4 "raises" to overcome just one "lowers". With two "lowers" the candidate won't even make it to the debrief or worse the loop will be cut-short. The bar-raiser (it's their job) are expected to specifically look for candidates who raise the bar. Raising the bar == better than 50% of "employees" (not job seekers) at their level. The rationale, when the bar-raiser program was created, was that the collective competency in a corporation should increase as more employees are inducted. Source: I was a bar-raiser at Amazon; 400+ interviews, 300+ as bar-raiser, conducted hiring boot camp, bar-raiser trainings, trained bar-raisers.
- oa335 6y ago
  How does the organization gauge its success in bar-raising? Did you guys just progressively hire less people over time?
  - flak48 6y ago
    Companies tend to keep track of the onsite success to failure ratio and then if it's too low examine if the bar raising is too aggressive. Which ends up going in the direction of meetings like "Should we standardize on what questions we are asking in interviews?" or "Should we standardize what pass means or fail means for a particular problem?"
  - vishnugupta 6y ago
    Bar-raiser program didn't have a feedback loop to track its effectiveness. Bar raisers periodically met to discuss topics related to hiring and increasing interview bandwidth but we didn't evaluate the process as such. That said, the program (like any such program I guess) didn't scale well with the immense pressure on increasing the head-count which in itself was a result of delivering more. Hiring managers and recruiters (who are incentivised to hire more) would rig up interview loops in a way to get their candidates through. The program itself came under increasing scrutiny (I guess for the right reasons) as being too restrictive. I distinctly remember period between 2008-2012 they hired at a blistering pace, all over the world. In fact in Seattle when they moved into SLU campus the buildings went from empty to nearly 100% occupancy in a matter of year or so. You could even see it based on the number of product launches from 2012 onwards (AWS, the hardware products, new country launches etc.,). And now Amazon is such a behemoth that it doesn't even make sense to speak about it or analyse it as a single entity anymore. My prediction is in 5-8 years it'll be broken up into at least three companies under Amazon as a holding corporation --- cloud, retail, and consumer devices.

What is the lowers/raises approach? Google isn’t up to the job (pun was not initially intended, but here we are).

by jsinai 6y ago

What is the lowers/raises approach? Google isn’t up to the job (pun was not initially intended, but here we are).

btgeekboy 6y ago

I think it’s the candidate vs team. Is the candidate better or worse than half of the team?

lostcolony 6y ago
Yep. "does this candidate raise the bar, or lower it". I.e., they have to be better than the average at the company already. As decided by you, from 50 minutes of interaction in a completely artificial, stressful situation.

a-priori 6y ago

I don't know what company they're referring to but at Amazon, the criteria for hiring someone is that interviewee would be better than 50% of the people currently doing that role at the company. They should 'raise the bar' in order to be hired.

volkk 6y ago
how do you even gauge that? it sounds like you'd need to keep re-interviewing the current people there to make it relative and fair. which of course isnt going to happen so "raising" the bar doesnt make any sense whatsoever
mattkrause 6y ago
Doesn't this make it (literally) exponentially harder to hire people?
- decafninja 6y ago
  I think that's part of the point. i.e. the usual line about wanting to avoid false positives (bad engineers putting on an act) even if it means sacrificing many false negatives (good engineers who suck at interviewing, or even just simply wasn't having a good day on the day of the interview). Ironically the modern technical whiteboarding interview often does require the candidate to effectively act and put on a show,
  - mattkrause 6y ago
    I knocked together a little simulation. Suppose programming ability is normally distributed with a mean of 100 and and SD of 15 (values cribbed from IQ). A team starts with 10 people drawn from that population. More applicants are drawn from that population: if an applicant's ability is above the median of the current team, they are hired, replacing a randomly-selected member of the team; otherwise, we pass. At the first timestep, half the applicants get hired right away. It takes a few hundred total failed interviews to make the 25th hire, and a few hundred thousand(!) for the 100th hire. These have very long tails too: the longest run (of 1000) took 800M total interviews to hire 100 people. There's no way this rule is actually being applied in practice.
    - theflyinghorse 6y ago
      raising the bar would also mean that their pool of people who are merely good enough gets very small very fast and Amazon just doesn't have the reputation to attract the absolute best

vishnugupta 6y ago

Lowers/raises the bar. As part of interview feedback an interviewer is supposed to rate the interviewee as raises/meets/lowers. Typically, just one lowers in an a loop of 6 interviews is sufficient to reject. On the other hand, a loop with all "meets" isn't a sufficient either. It needs about 3-4 "raises" to overcome just one "lowers". With two "lowers" the candidate won't even make it to the debrief or worse the loop will be cut-short. The bar-raiser (it's their job) are expected to specifically look for candidates who raise the bar. Raising the bar == better than 50% of "employees" (not job seekers) at their level. The rationale, when the bar-raiser program was created, was that the collective competency in a corporation should increase as more employees are inducted. Source: I was a bar-raiser at Amazon; 400+ interviews, 300+ as bar-raiser, conducted hiring boot camp, bar-raiser trainings, trained bar-raisers.

oa335 6y ago
How does the organization gauge its success in bar-raising? Did you guys just progressively hire less people over time?
- flak48 6y ago
  Companies tend to keep track of the onsite success to failure ratio and then if it's too low examine if the bar raising is too aggressive. Which ends up going in the direction of meetings like "Should we standardize on what questions we are asking in interviews?" or "Should we standardize what pass means or fail means for a particular problem?"
- vishnugupta 6y ago
  Bar-raiser program didn't have a feedback loop to track its effectiveness. Bar raisers periodically met to discuss topics related to hiring and increasing interview bandwidth but we didn't evaluate the process as such. That said, the program (like any such program I guess) didn't scale well with the immense pressure on increasing the head-count which in itself was a result of delivering more. Hiring managers and recruiters (who are incentivised to hire more) would rig up interview loops in a way to get their candidates through. The program itself came under increasing scrutiny (I guess for the right reasons) as being too restrictive. I distinctly remember period between 2008-2012 they hired at a blistering pace, all over the world. In fact in Seattle when they moved into SLU campus the buildings went from empty to nearly 100% occupancy in a matter of year or so. You could even see it based on the number of product launches from 2012 onwards (AWS, the hardware products, new country launches etc.,). And now Amazon is such a behemoth that it doesn't even make sense to speak about it or analyse it as a single entity anymore. My prediction is in 5-8 years it'll be broken up into at least three companies under Amazon as a holding corporation --- cloud, retail, and consumer devices.