7 ms·

I'm glad to see this getting roasted in the comments, as it's a really good example of how companies put out self-serving pseudo-statistical nonsense in an effo

I'm glad to see this getting roasted in the comments, as it's a really good example of how companies put out self-serving pseudo-statistical nonsense in an effort to promote themselves. There's no effort to quantify what "technical" or "communication" skills are - these are left to the interpretation of a the interviewer. It makes no effort to show where these engineers are going, what the interview process they're completing looks like, what impact demographics had on this, etc. I find this stuff repugnant. It perpetuates the myth that there's something really special about Silicon Valley engineers, while making only lazy and perfunctory efforts to examine any alternative explanations than "this is where the rockstar ninja coders work." Shameful.

ttyprintk 5y ago
At least they left in the "Unemployed" category.
rootusrootus 5y ago
> It perpetuates the myth that there's something really special about Silicon Valley engineers Does this myth still have much traction? If anything, my general regard for engineers in the bay area has steadily declined in the last few years. There are so many really worthless folks who have only figured out how look like they have a clue, but go any deeper and they flail. I know I'm painting with a broad brush, and that's not fair, but most of the great engineers I work with are at various other places around the country, not California.
- kieselguhr_kid 5y ago
  Personally, as a (now relocated) Bay Area native, I agree. Over, I think there's still a lot of prestige for large Silicon Valley firms, even if some of the gloss has justifiably started to fade. This article certainly assumes that myth is still in place.
- kjeetgill 5y ago
  Hey, it's not our fault! Where else are posers gonna flock to to pose? I swear, even with all the remote work, more than 95%* of the people that worked here 5 years ago still do. We live here. * made up statistic.
  - rootusrootus 5y ago
    I realize I wasn't being very nice, and I apologize for that. I'm quite sure there are plenty of very smart people who live in the bay area and do great work. With such a high concentration of engineers it makes sense plenty will suck, too. My experience is biased, of course. My company has offices all over the country and a couple years ago opened a new office in SF, and I'm 93.4% sure we don't exactly pay FAANG-competitive salaries there, which affects the quality of who we can recruit there. It's kinda like how we hire engineers in Hyderabad for 1/5 the US rate and then wonder why we more often than not get substandard performance.
    - kjeetgill 5y ago
      > I realize I wasn't being very nice, and I apologize for that. No worries at all: I agree with you. More then just salaries, there's definitely a glut of people chasing the gold rush. More power to 'em (I guess) but it is a bummer for the rest of us.
- emerged 5y ago
  I know a ton of engineers. Of them all, those working FAANG are profoundly less skilled than the others. It’s impossible to miss its so obvious. Maybe my social network is an outlier, but I really really doubt it.
  - b20000 5y ago
    why is this downvoted?
    - kieselguhr_kid 5y ago
      Fragile FAANG-ers
      - bryanrasmussen 5y ago
        Well I'm not a FAANG-er and I didn't downvote, but it seems weird that the ones at FAANG would be so obviously less skilled than the others, It would be believable that they were the same skill level but otherwise seems weird. Unless of course the weighting of his groups is off somehow, which should have been noted. Actually it doesn't seem like the statement was very informative.
        emerged 5y ago
        I mean. It was a statement of fact, you can believe it or not. The engineers I know who went into FAANG (half dozen different people) are literally the worst engineers I worked with. They have no scruples and play politics.
    - unmole 5y ago
      Because it's a just-so anecdote?
ummonk 5y ago
"There's no effort to quantify what "technical" or "communication" skills are - these are left to the interpretation of a the interviewer." And yet, these scores are measuring something and averaged across tens of thousands of technical interviews, you have enough statistical power to average out the particularities of each interviewer. I'm sorry you find the results repugnant, but the results are what they are. And the article did have a large section on limitations of their analysis.
- kieselguhr_kid 5y ago
  It's not the results I find repugnant, it's the assumption that the results have any real world validity. The _something_ they're measuring is as likely to be demographic biases as it is technical or communication skills.
  - stale2002 5y ago
    > it's the assumption that the results have any real world validity Of course it has real world validity. It has real world validity regarding how well people are likely to do in tech interviews. You can feel free to say that the way the tech industry does interviews is bad, or biased, or has all sorts of problems. But having vague measurements, such as "technical" or "communication" skills sounds very accurate to how tech interviews are actually done, in the real world. All the moral outrage everyone is having about this, seems to have nothing to do with the accuracy of the report, which is about measuring tech interview performance. And it instead seems to be regarding the tech interview process in the industry, in general.
    - kieselguhr_kid 5y ago
      It doesn't necessarily say anything other than "where you work now has some predictive value for how well you'll be perceived by interviewers" in which case these aren't "top performers", they're "people with jobs" and it's just regurgitating a truism about how it's easier to get a job if you have a job. I'm not buying this as a moral outrage question, I'm wondering if this is adding anything meaningful for us to look at or if it's just a surface-level puff piece masquerading as an analysis.
leeny 5y ago
Author here. Yes, the skills are left to the interpretation of the interviewer, but most of our interviewers are senior engineers at FAANG. We've done quite a bit of work internally to make sure your interviewers are well calibrated, and we have a living calibration score for each one (calibration is based on how the interviewees they interview end up doing in real interviews). The interviews in question are a mix of algorithmic interviews and systems design interviews. Also, if I ever use "rockstar" or "ninja" in my posts, I hope someone finds me on the street and punches me in the face. I'd deserve it.
- TurkTurkleton 5y ago
  > Also, if I ever use "rockstar" or "ninja" in my posts, I hope someone finds me on the street and punches me in the face. I'd deserve it. Are you sure you'll never find yourself blogging about bands' flamboyant frontmen (or frontwomen), or covert agents in feudal Japan?
  - leeny 5y ago
    HAH.
- kieselguhr_kid 5y ago
  Is there any statistical reason I should assume an interviewer at FAANG has some kind of special insight into what good technical and communication skills are? Were any attempts made to adjust for demographic biases? Is it possible, for instance, that Dropbox engineers are disproportionately taller white dudes with nice hair? I know that sounds a bit unserious, but all of those factors would increase the likelihood of a higher score. I appreciate that you never used "rockstar" or "ninja". That dig was a bit unfair of me.
  - leeny 5y ago
    no, we didn't adjust for demographic biases because, look, we're an anonymous platform. we periodically survey our users to get demographic data, but it's not something we ask in the app because we've never been able to resolve "tell me your race & gender" with "hey we're anonymous, and you'll be judged on your performance and nothing else". last thing i want is to perpetuate stereotype threat inadvertently. it's possible to do this right, i think, but we haven't gotten there yet.
psyc 5y ago
While I agree that this particular exercise is riddled with problems, I simply cannot image Hacker News rolling over and accepting evidence-based answers to questions of this nature, regardless of where the data came from or what the methodology was.
tomp 5y ago
um are we looking at the same graphs? the second best performing company on "technical" and "problem solving" was Bloomberg, literally the opposite of Silicon Valley
rocgf 5y ago
While there may not be any reliable measure of communication skills across the industry, the fact that the data was based on scores given my a large amount of people, that by definition means that it's accurate. Think about it carefully - if people rate people as being good at communication, then there is no reason to quantify it any other way. There are some obvious flaws here, like the quantity of data and it's normalization, but it's basically a tautology.
- ClumsyPilot 5y ago
  "Think about it carefully - if people rate people as being good at communication, then there is no reason to quantify it any other way. There are some obvious flaws here, like the quantity of data and it's normalization, but it's basically a tautology." So if a lot of people rate you as a great leader, you are a great leader? Even if you lie to their face and delover terrible results? Objective reality doesn't matter? So the greatest leader in the world is in North Korea? Communication is a clearly measurable skill. Just because a lot of people have been sold a lie, that doesn't make it true
overrun11 5y ago
> It perpetuates the myth that there's something really special about Silicon Valley engineers Why are you sure it's a myth? My prior would be to believe that engineers at the most exclusive companies with the highest hiring bars that pay 3-5 times more than average would be better programmers. The article is just one data point confirming what intuitively should be true. If Silicon Valley engineers are no better than anywhere else then someone should notify the execs at FAANG, I'm sure they'd be interested to know they are dramatically overpaying for talent. I don't understand what is so uncontroversial about this. SV companies recruit the best talent from around the world and it's where the best talent wants to work. Similarly, the best financial talents are in NYC and London, the best actors are in Hollywood etc.
- kevin_thibedeau 5y ago
  There are capable people who don't want to live where 1000sqft home costs $1M. Never mind all of the other problems with the region. Selection bias doesn't prove anything about the people you aren't selecting for.
  - coryrc 5y ago
    If houses only cost twice your gross salary, that sounds affordable to me!
  - overrun11 5y ago
    Dropbox is fully remote. All of these companies have offices around the country. I don't see how selection bias plays in here
  - usaar333 5y ago
    Are we talking capable or the very top? It's going to be a small minority of that top cohort where Bay Area COL is an issue.
- kieselguhr_kid 5y ago
  I've lived in Silicon Valley and I've lived outside of it. I'm sure it's a myth. The higher rate of pay is not an indication of higher skill.
xtracto 5y ago
> It perpetuates the myth that there's something really special about Silicon Valley engineers In my [disillusioned] experience, this holds true: Silicon Valley engineers are very good for building throwaway MVPs that they won't have to maintain more than 3 years. I've been very disillusioned by the quality of the software written by Silicon Valley companies, but in hindsight it makes sense: "Run fast break things" development culture resonates with the "raise VC money every 18 months" business culture, and then look for an exit in 5 years tops. There is no incentive in Dev or Business to really develop good software.

ttyprintk 5y ago

At least they left in the "Unemployed" category.

rootusrootus 5y ago

> It perpetuates the myth that there's something really special about Silicon Valley engineers Does this myth still have much traction? If anything, my general regard for engineers in the bay area has steadily declined in the last few years. There are so many really worthless folks who have only figured out how look like they have a clue, but go any deeper and they flail. I know I'm painting with a broad brush, and that's not fair, but most of the great engineers I work with are at various other places around the country, not California.

kieselguhr_kid 5y ago
Personally, as a (now relocated) Bay Area native, I agree. Over, I think there's still a lot of prestige for large Silicon Valley firms, even if some of the gloss has justifiably started to fade. This article certainly assumes that myth is still in place.
kjeetgill 5y ago
Hey, it's not our fault! Where else are posers gonna flock to to pose? I swear, even with all the remote work, more than 95%* of the people that worked here 5 years ago still do. We live here. * made up statistic.
- rootusrootus 5y ago
  I realize I wasn't being very nice, and I apologize for that. I'm quite sure there are plenty of very smart people who live in the bay area and do great work. With such a high concentration of engineers it makes sense plenty will suck, too. My experience is biased, of course. My company has offices all over the country and a couple years ago opened a new office in SF, and I'm 93.4% sure we don't exactly pay FAANG-competitive salaries there, which affects the quality of who we can recruit there. It's kinda like how we hire engineers in Hyderabad for 1/5 the US rate and then wonder why we more often than not get substandard performance.
  - kjeetgill 5y ago
    > I realize I wasn't being very nice, and I apologize for that. No worries at all: I agree with you. More then just salaries, there's definitely a glut of people chasing the gold rush. More power to 'em (I guess) but it is a bummer for the rest of us.
emerged 5y ago
I know a ton of engineers. Of them all, those working FAANG are profoundly less skilled than the others. It’s impossible to miss its so obvious. Maybe my social network is an outlier, but I really really doubt it.
- b20000 5y ago
  why is this downvoted?
  - kieselguhr_kid 5y ago
    Fragile FAANG-ers
    - bryanrasmussen 5y ago
      Well I'm not a FAANG-er and I didn't downvote, but it seems weird that the ones at FAANG would be so obviously less skilled than the others, It would be believable that they were the same skill level but otherwise seems weird. Unless of course the weighting of his groups is off somehow, which should have been noted. Actually it doesn't seem like the statement was very informative.
      - emerged 5y ago
        I mean. It was a statement of fact, you can believe it or not. The engineers I know who went into FAANG (half dozen different people) are literally the worst engineers I worked with. They have no scruples and play politics.
  - unmole 5y ago
    Because it's a just-so anecdote?

ummonk 5y ago

"There's no effort to quantify what "technical" or "communication" skills are - these are left to the interpretation of a the interviewer." And yet, these scores are measuring something and averaged across tens of thousands of technical interviews, you have enough statistical power to average out the particularities of each interviewer. I'm sorry you find the results repugnant, but the results are what they are. And the article did have a large section on limitations of their analysis.

kieselguhr_kid 5y ago
It's not the results I find repugnant, it's the assumption that the results have any real world validity. The _something_ they're measuring is as likely to be demographic biases as it is technical or communication skills.
- stale2002 5y ago
  > it's the assumption that the results have any real world validity Of course it has real world validity. It has real world validity regarding how well people are likely to do in tech interviews. You can feel free to say that the way the tech industry does interviews is bad, or biased, or has all sorts of problems. But having vague measurements, such as "technical" or "communication" skills sounds very accurate to how tech interviews are actually done, in the real world. All the moral outrage everyone is having about this, seems to have nothing to do with the accuracy of the report, which is about measuring tech interview performance. And it instead seems to be regarding the tech interview process in the industry, in general.
  - kieselguhr_kid 5y ago
    It doesn't necessarily say anything other than "where you work now has some predictive value for how well you'll be perceived by interviewers" in which case these aren't "top performers", they're "people with jobs" and it's just regurgitating a truism about how it's easier to get a job if you have a job. I'm not buying this as a moral outrage question, I'm wondering if this is adding anything meaningful for us to look at or if it's just a surface-level puff piece masquerading as an analysis.

leeny 5y ago

Author here. Yes, the skills are left to the interpretation of the interviewer, but most of our interviewers are senior engineers at FAANG. We've done quite a bit of work internally to make sure your interviewers are well calibrated, and we have a living calibration score for each one (calibration is based on how the interviewees they interview end up doing in real interviews). The interviews in question are a mix of algorithmic interviews and systems design interviews. Also, if I ever use "rockstar" or "ninja" in my posts, I hope someone finds me on the street and punches me in the face. I'd deserve it.

TurkTurkleton 5y ago
> Also, if I ever use "rockstar" or "ninja" in my posts, I hope someone finds me on the street and punches me in the face. I'd deserve it. Are you sure you'll never find yourself blogging about bands' flamboyant frontmen (or frontwomen), or covert agents in feudal Japan?
- leeny 5y ago
  HAH.
kieselguhr_kid 5y ago
Is there any statistical reason I should assume an interviewer at FAANG has some kind of special insight into what good technical and communication skills are? Were any attempts made to adjust for demographic biases? Is it possible, for instance, that Dropbox engineers are disproportionately taller white dudes with nice hair? I know that sounds a bit unserious, but all of those factors would increase the likelihood of a higher score. I appreciate that you never used "rockstar" or "ninja". That dig was a bit unfair of me.
- leeny 5y ago
  no, we didn't adjust for demographic biases because, look, we're an anonymous platform. we periodically survey our users to get demographic data, but it's not something we ask in the app because we've never been able to resolve "tell me your race & gender" with "hey we're anonymous, and you'll be judged on your performance and nothing else". last thing i want is to perpetuate stereotype threat inadvertently. it's possible to do this right, i think, but we haven't gotten there yet.

psyc 5y ago

While I agree that this particular exercise is riddled with problems, I simply cannot image Hacker News rolling over and accepting evidence-based answers to questions of this nature, regardless of where the data came from or what the methodology was.

tomp 5y ago

um are we looking at the same graphs? the second best performing company on "technical" and "problem solving" was Bloomberg, literally the opposite of Silicon Valley

rocgf 5y ago

While there may not be any reliable measure of communication skills across the industry, the fact that the data was based on scores given my a large amount of people, that by definition means that it's accurate. Think about it carefully - if people rate people as being good at communication, then there is no reason to quantify it any other way. There are some obvious flaws here, like the quantity of data and it's normalization, but it's basically a tautology.

ClumsyPilot 5y ago
"Think about it carefully - if people rate people as being good at communication, then there is no reason to quantify it any other way. There are some obvious flaws here, like the quantity of data and it's normalization, but it's basically a tautology." So if a lot of people rate you as a great leader, you are a great leader? Even if you lie to their face and delover terrible results? Objective reality doesn't matter? So the greatest leader in the world is in North Korea? Communication is a clearly measurable skill. Just because a lot of people have been sold a lie, that doesn't make it true

overrun11 5y ago

> It perpetuates the myth that there's something really special about Silicon Valley engineers Why are you sure it's a myth? My prior would be to believe that engineers at the most exclusive companies with the highest hiring bars that pay 3-5 times more than average would be better programmers. The article is just one data point confirming what intuitively should be true. If Silicon Valley engineers are no better than anywhere else then someone should notify the execs at FAANG, I'm sure they'd be interested to know they are dramatically overpaying for talent. I don't understand what is so uncontroversial about this. SV companies recruit the best talent from around the world and it's where the best talent wants to work. Similarly, the best financial talents are in NYC and London, the best actors are in Hollywood etc.

kevin_thibedeau 5y ago
There are capable people who don't want to live where 1000sqft home costs $1M. Never mind all of the other problems with the region. Selection bias doesn't prove anything about the people you aren't selecting for.
- coryrc 5y ago
  If houses only cost twice your gross salary, that sounds affordable to me!
- overrun11 5y ago
  Dropbox is fully remote. All of these companies have offices around the country. I don't see how selection bias plays in here
- usaar333 5y ago
  Are we talking capable or the very top? It's going to be a small minority of that top cohort where Bay Area COL is an issue.
kieselguhr_kid 5y ago
I've lived in Silicon Valley and I've lived outside of it. I'm sure it's a myth. The higher rate of pay is not an indication of higher skill.

xtracto 5y ago

> It perpetuates the myth that there's something really special about Silicon Valley engineers In my [disillusioned] experience, this holds true: Silicon Valley engineers are very good for building throwaway MVPs that they won't have to maintain more than 3 years. I've been very disillusioned by the quality of the software written by Silicon Valley companies, but in hindsight it makes sense: "Run fast break things" development culture resonates with the "raise VC money every 18 months" business culture, and then look for an exit in 5 years tops. There is no incentive in Dev or Business to really develop good software.