6 ms·

I worked at Google, and performed quite a few interviews. At least at that time, the interviewer had no idea where in the company the software engineer would g

by KMag 11y ago

I worked at Google, and performed quite a few interviews. At least at that time, the interviewer had no idea where in the company the software engineer would go. The people who got hired would go into a pool, and the managers who needed staff would then horse-trade for them. They needed to hire people who could be plugged in to tons of different positions, including some that genuinely had tough problems where if they accidentally did something in O(N^2), their half-day compute job would suddenly literally take centuries to complete. Also, part of the point of a Google style interview is to keep pushing you to the point where your ability fails, then push in another direction to the point your ability fails. That way, they get an idea of the dimensions of your skillset. The vast majority of the people I recommended hiring didn't answer the questions perfectly. Or, someone could not quite put enough thought into the integer encoding for Protocol Buffers. The encoding actually used is that the first bit of every byte is a flag for if there's a next byte, for a maximum of 10 bytes. The only advantage to this over encoding the length of the integer as the number of leading ones in the first byte (like UTF-8) is if they want the option to have the option later of a forward-compatible encoding for longer integer types, without just supporting multi-precision integers as byte arrays. If you slide all of those flag bits to the first byte, you don't have any more or any fewer flag bits, so it takes as much space, but you can use a jump table and the ability to use native 8-byte and 4-byte simple loads instead of many more masking operations and conditional branches. Giving up on anything longer than int64_t means that the maximum encoding length becomes 9 bytes instead of 10. One day, the speed of the indexing system just dropped in half. It turns out that someone wrote a job that used machine learning to generate a bunch of regexes for some signal. They were recompiling the regexes for every page in the index. Google really needs most of their engineers that will naturally spot this kind of thing in code reviews, because if every engineer on the indexing team made a mistake like that once a year, the indexing system would always run at half speed, sometimes even 1/3 or 1/4 speed. The indexing system uses as much electricity as a small town. The tiniest of improvements can mean thousands of dollars per year in savings.

theoh 11y ago
"an idea of the dimensions of your skillset" sounds like a really reductive way of thinking. It's as if the skillset is restricted to something easy to quickly explore and visualize, like a convex object in a few dimensions. Maybe those assumptions are necessary for interviewing but they're nothing to be proud of.
- skj 11y ago
  Why would they be something to be ashamed of?
  - hueving 11y ago
    Because they are a really crappy approximation of how good someone is.
    - skj 11y ago
      Feel free to suggest a viable alternative.
      - hueving 11y ago
        I didn't imply that I have one. I was just explaining why Google shouldn't be proud of what they do.
- KMag 11y ago
  I made no mention of low dimensionality. It's clearly a very high dimensional estimation problem with time constraints on measurement. There are some correlations, so you can pick the principal components that you think are most important, and measure some dimensions that are (you cross your fingers and hope) correlated with those principal components. But, it's largely guestimates anyway. It's hard to come up with good metrics of employee performance that correlate well to company performance and even harder to come up with interview questions that have good predictive power for those metrics. Ideally, they'd relax the hiring criteria and all employees would start as 1-year or 2-year contractors. On-job performance is the best indicator of on-job performance.
Outdoorsman 11y ago
>>Also, part of the point of a Google style interview is to keep pushing you to the point where your ability fails, then push in another direction to the point your ability fails. That way, they get an idea of the dimensions of your skill-set. The vast majority of the people I recommended hiring didn't answer the questions perfectly.<< I can't even begin to tell you how valuable I consider this approach to be...pushing (or leading) a candidate in every conceivable direction until failure is reached or limits are exposed... HR professionals in almost any profession would benefit from implementing this technique to the extent that their resources allow...most don't have the luxury of time... Example: As a member of a non-profit board I was once tasked with performing screening interviews for 5 candidates who were applying for a Camp Ranger position at one of the organization's youth camps... I was not an HR expert then (or now), so I jotted down a few interview questions and scheduled interviews with the candidates...each one to begin at 8:00am at the site, and to last as long as it took...one per day... I simply walked around the site, visited with the candidates, asked about their experience, and did everything I could to encourage them to "reveal" themselves... I eliminated 2 candidates by around 10:00am, 2 more around 11:00am, and ended up driving to town for a burger lunch with the 5th...I later recommended the fifth... He was hired, and ended up doing an incredible job... I ended up giving other board members the impression that I knew what I was doing...at the time I most certainly did not...but I filed that lesson away... I am convinced that a commitment to open-ended interview time, combined with "soft" skills probing, are critical if you want to properly evaluate candidates...
p4wnc6 11y ago
> genuinely had tough problems where if they accidentally did something in O(N^2), their half-day compute job would suddenly literally take centuries to complete. Yes, but this has almost zero bearing on the actual interview. Being able to avoid this in real life means that you measure twice and cut once, you pay attention, and you ask for help and training. Being able to do something similar with dynamic programming or tree riddles in an interview has absolutely nothing to do with those longer-view skills. > Google really needs most of their engineers that will naturally spot this kind of thing in code reviews, because if every engineer on the indexing team made a mistake like that once a year, the indexing system would always run at half speed, sometimes even 1/3 or 1/4 speed. Again, this has nothing to do with puzzle interviews. In this case, you could hire smart people almost independently of whether they had encountered certain data structures before, and certainly without much concern about how fast they could spot things like this in an interview setting. And instead, hire for general aptitude, and then invest in training someone to be able to spot these things to the necessary degree. Even though you can provide impressive-seeming examples of problems where minor tweaks in complexity or subtle implementation-specific points about data structures can result in huge problems, it's still not supportive of whiteboard-hazing style interviews that test more for interview aptitude than anything else. After all, whoever it was that wrote the ML regex job, that person passed the interviews (and that person may have rightfully passed the interviews and deserved to work for Google, even after this later evidence that they weren't careful enough in one case).
- rhizome 11y ago
  I think what the GP was illustrating is that Google is extremely conservative. For them to hire people outside of the strict boundaries of, "dimensions of skills as represented in an interview," is kind of a rejection of the Robustness Principle for corporate purposes. I mean, heck, something like this? They interview multiple magnitudes of people necessary for this kind of job. I respect it as a hedge, but it probably also takes people out of the industry who could do more vital work someplace else.
  - p4wnc6 11y ago
    Even so, it only pays to be conservative via an overly selective filter if that filter actually selects for what you want. If you zealously apply some filter, believing that you're being extra cautious, but the filter doesn't actually select for what you think, or the filter isn't actually as selective as you think, then it's just a form of selectivity theater. I've met plenty of ex-Googlers who were great at reciting CS trivia, but actually not very good at real life engineering. Granted, that may be why they were ex-Googlers, but it still doesn't speak well of Google's hiring process.
deleted 11y ago
[deleted]
seiji 11y ago
The encoding actually used is that the first bit of every byte is a flag for if there's a next byte, for a maximum of 10 bytes. Fun facts: those are commonly known as "vbytes." They are the slowest kind of variable width integer encoding. The simplest (naive, and generally considered "wrong") implementations of variable-width-by-continuation-bits uses 10 bytes maximum. A proper implementation uses 9 bytes maximum. There's actually no reason to ever use 10 bytes in this encoding. If you use 10 bytes, that means your last byte only holds one bit of actual user data. That's not very cool. But, your next-to-last byte holds seven bits of user data and one bit of metadata. We can easily say "if we're at the next to last byte, don't use metadata, just use all the bits we need." All you have to do is say "if we are currently at 9 bytes, don't use a 10th byte, just use this 9th byte directly." bam. Your 9th byte now has 8 bits of user data and you don't roll over a useless 10th byte with one bit of data. The "slide all continuation bits into the first byte" sounds like a trick, but it's really using a TLV encoding where the first byte just holds a number between 1 and 8, so the entire integer+metadata is now [1 type byte][1 to 8 user data] = 2 to 9 bytes total. Using this scheme also kills any "1 byte, standalone, variable width integer" capability (unless you're storing partial values in the first T/L byte, but then that limits you to a much lower max value for one byte).
- haberman 11y ago
  > Using this scheme also kills any "1 byte, standalone, variable width integer" capability (unless you're storing partial values in the first T/L byte, but then that limits you to a much lower max value for one byte). You can get the best of both worlds. The way to do it is: separate continuation bits (1's) from data bits with a 0. This gives identical encoding-length characteristics as what you are calling "vbytes" (1 byte can encode 0-127, 2 bytes can encode 128-16383, etc) while still front-loading the continuation bits.
  - seiji 11y ago
    That's clever too, but it seems we're back to masking/shifting things all over the place too (which seemed to be an anti-design goal listed above. These can also have the "avoid 10 bytes" optimization by just reading the next 8 bytes exactly if the first byte is just all ones (no need for a zero separator; it would be in the way and push out the final bit to its own isolated byte again).
rajacombinator 11y ago
How many work on indexing? I'm guessing like 100 tops? (Genuine guess/curiosity, not sarcasm.)

by KMag 11y ago

theoh 11y ago

"an idea of the dimensions of your skillset" sounds like a really reductive way of thinking. It's as if the skillset is restricted to something easy to quickly explore and visualize, like a convex object in a few dimensions. Maybe those assumptions are necessary for interviewing but they're nothing to be proud of.

skj 11y ago
Why would they be something to be ashamed of?
- hueving 11y ago
  Because they are a really crappy approximation of how good someone is.
  - skj 11y ago
    Feel free to suggest a viable alternative.
    - hueving 11y ago
      I didn't imply that I have one. I was just explaining why Google shouldn't be proud of what they do.
KMag 11y ago
I made no mention of low dimensionality. It's clearly a very high dimensional estimation problem with time constraints on measurement. There are some correlations, so you can pick the principal components that you think are most important, and measure some dimensions that are (you cross your fingers and hope) correlated with those principal components. But, it's largely guestimates anyway. It's hard to come up with good metrics of employee performance that correlate well to company performance and even harder to come up with interview questions that have good predictive power for those metrics. Ideally, they'd relax the hiring criteria and all employees would start as 1-year or 2-year contractors. On-job performance is the best indicator of on-job performance.

Outdoorsman 11y ago

>>Also, part of the point of a Google style interview is to keep pushing you to the point where your ability fails, then push in another direction to the point your ability fails. That way, they get an idea of the dimensions of your skill-set. The vast majority of the people I recommended hiring didn't answer the questions perfectly.<< I can't even begin to tell you how valuable I consider this approach to be...pushing (or leading) a candidate in every conceivable direction until failure is reached or limits are exposed... HR professionals in almost any profession would benefit from implementing this technique to the extent that their resources allow...most don't have the luxury of time... Example: As a member of a non-profit board I was once tasked with performing screening interviews for 5 candidates who were applying for a Camp Ranger position at one of the organization's youth camps... I was not an HR expert then (or now), so I jotted down a few interview questions and scheduled interviews with the candidates...each one to begin at 8:00am at the site, and to last as long as it took...one per day... I simply walked around the site, visited with the candidates, asked about their experience, and did everything I could to encourage them to "reveal" themselves... I eliminated 2 candidates by around 10:00am, 2 more around 11:00am, and ended up driving to town for a burger lunch with the 5th...I later recommended the fifth... He was hired, and ended up doing an incredible job... I ended up giving other board members the impression that I knew what I was doing...at the time I most certainly did not...but I filed that lesson away... I am convinced that a commitment to open-ended interview time, combined with "soft" skills probing, are critical if you want to properly evaluate candidates...

p4wnc6 11y ago

> genuinely had tough problems where if they accidentally did something in O(N^2), their half-day compute job would suddenly literally take centuries to complete. Yes, but this has almost zero bearing on the actual interview. Being able to avoid this in real life means that you measure twice and cut once, you pay attention, and you ask for help and training. Being able to do something similar with dynamic programming or tree riddles in an interview has absolutely nothing to do with those longer-view skills. > Google really needs most of their engineers that will naturally spot this kind of thing in code reviews, because if every engineer on the indexing team made a mistake like that once a year, the indexing system would always run at half speed, sometimes even 1/3 or 1/4 speed. Again, this has nothing to do with puzzle interviews. In this case, you could hire smart people almost independently of whether they had encountered certain data structures before, and certainly without much concern about how fast they could spot things like this in an interview setting. And instead, hire for general aptitude, and then invest in training someone to be able to spot these things to the necessary degree. Even though you can provide impressive-seeming examples of problems where minor tweaks in complexity or subtle implementation-specific points about data structures can result in huge problems, it's still not supportive of whiteboard-hazing style interviews that test more for interview aptitude than anything else. After all, whoever it was that wrote the ML regex job, that person passed the interviews (and that person may have rightfully passed the interviews and deserved to work for Google, even after this later evidence that they weren't careful enough in one case).

rhizome 11y ago
I think what the GP was illustrating is that Google is extremely conservative. For them to hire people outside of the strict boundaries of, "dimensions of skills as represented in an interview," is kind of a rejection of the Robustness Principle for corporate purposes. I mean, heck, something like this? They interview multiple magnitudes of people necessary for this kind of job. I respect it as a hedge, but it probably also takes people out of the industry who could do more vital work someplace else.
- p4wnc6 11y ago
  Even so, it only pays to be conservative via an overly selective filter if that filter actually selects for what you want. If you zealously apply some filter, believing that you're being extra cautious, but the filter doesn't actually select for what you think, or the filter isn't actually as selective as you think, then it's just a form of selectivity theater. I've met plenty of ex-Googlers who were great at reciting CS trivia, but actually not very good at real life engineering. Granted, that may be why they were ex-Googlers, but it still doesn't speak well of Google's hiring process.

deleted 11y ago

[deleted]

seiji 11y ago

The encoding actually used is that the first bit of every byte is a flag for if there's a next byte, for a maximum of 10 bytes. Fun facts: those are commonly known as "vbytes." They are the slowest kind of variable width integer encoding. The simplest (naive, and generally considered "wrong") implementations of variable-width-by-continuation-bits uses 10 bytes maximum. A proper implementation uses 9 bytes maximum. There's actually no reason to ever use 10 bytes in this encoding. If you use 10 bytes, that means your last byte only holds one bit of actual user data. That's not very cool. But, your next-to-last byte holds seven bits of user data and one bit of metadata. We can easily say "if we're at the next to last byte, don't use metadata, just use all the bits we need." All you have to do is say "if we are currently at 9 bytes, don't use a 10th byte, just use this 9th byte directly." bam. Your 9th byte now has 8 bits of user data and you don't roll over a useless 10th byte with one bit of data. The "slide all continuation bits into the first byte" sounds like a trick, but it's really using a TLV encoding where the first byte just holds a number between 1 and 8, so the entire integer+metadata is now [1 type byte][1 to 8 user data] = 2 to 9 bytes total. Using this scheme also kills any "1 byte, standalone, variable width integer" capability (unless you're storing partial values in the first T/L byte, but then that limits you to a much lower max value for one byte).

haberman 11y ago
> Using this scheme also kills any "1 byte, standalone, variable width integer" capability (unless you're storing partial values in the first T/L byte, but then that limits you to a much lower max value for one byte). You can get the best of both worlds. The way to do it is: separate continuation bits (1's) from data bits with a 0. This gives identical encoding-length characteristics as what you are calling "vbytes" (1 byte can encode 0-127, 2 bytes can encode 128-16383, etc) while still front-loading the continuation bits.
- seiji 11y ago
  That's clever too, but it seems we're back to masking/shifting things all over the place too (which seemed to be an anti-design goal listed above. These can also have the "avoid 10 bytes" optimization by just reading the next 8 bytes exactly if the first byte is just all ones (no need for a zero separator; it would be in the way and push out the final bit to its own isolated byte again).

rajacombinator 11y ago

How many work on indexing? I'm guessing like 100 tops? (Genuine guess/curiosity, not sarcasm.)