20 ms·
Most HR data is bad data (2015)
- autoexecbat 3y agoI'm hardly surprised by this. Anyone who has had to put people into promote/keep/meh/remove buckets is keenly aware how much gut feeling is involved
- zachmu 3y agoThis is why you need to structure your review process around objectively verifiable info as much as possible. Have clear areas of responsibility for each individual, such that you can readily evaluate what they accomplished and how well it's working. They are responsible for a particular mission and set of objectives, and they either achieve them or they don't. This can't work without individual ownership, which is antithetical to "best practices" at many places.
- mtlynch 3y agoStructuring the review around objectively verifiable data creates other problems. What if there's a developer on the team who delivers 1/3rd the features as everyone else, but they offer such effective mentorship that their five teammates deliver features 2x faster? If you design your review process around objectively verifiable data, then it's almost certain to undervalue a developer who's great at mentorship because mentorship is hard to measure. And it's not just mentorship. There are tons of contributions that seem pretty obviously beneficial but aren't objectively measurable: recognizing simpler ways of achieving business objectives, communicating clearly and effectively, being pleasant to be around, etc. But if you go the other way and leave it up to pure human judgement, then you end up with promotion being based on favoritism, office politics, and irrational biases.
- throwway120385 3y ago> But if you go the other way and leave it up to pure human judgement, then you end up with promotion being based on favoritism, office politics, and irrational biases. And my personal favorite: luck of the draw.
- willcipriano 3y ago> who delivers 1/3rd the features as everyone else, but they offer such effective mentorship that their five teammates deliver features 2x faster? I haven't really encountered this. Generally speaking people who can teach others to be much more productive like that are unsurprisingly themselves, more productive. This fictional person would be better suited to a people management or coaching role as a individual contractor they only achieve a third of the average.
- zachmu 3y agoSometimes you get the cranky old guy archetype who doesn't really get much done but is really good at making sure the kids don't make any really dumb mistakes, which is its own kind of value
- tanjtanjtanj 3y agoEvery team I've ever worked with (11) has had at least one person that would meet this archetype. Their team-based contribution usually wilts under the kind of individualized metrics being pushed for by some here and the team suffers as a result even if their own metrics get better.
- willcipriano 3y agoThen do my second case, this person gets a different title so we can evaluate them differently. Now with the major objection to objective performance review gone we can do that for the actual individual contributors.
- tanjtanjtanj 3y agoI have no need to do that as I'm actually involved in my teams and don't need arbitrary numbers to see that someone is contributing. I'm fortunate that I have been able to avoid (or actively dodge) metric perverts in my career.
- skeeter2020 3y agothis type of person is exactly who we try to identify for staff and higher roles. They could be 20% or even 50% more productive as an individual, but if they raise everyone's game even 2% across 50 or more people they are producing far more value. The reality is they crush fewer tickets individually, and that's a good thing, but it gets perverted by a purely metrics-driven assessment.
- zachmu 3y agoPeople making lots of soft contributions like mentorship / leadership tend to be the kind of well-liked personable sorts who do well on non-objective assessments, which always manage to color evaluations even when you try to make them as objective as possible.
- AdrianB1 3y agoNot in my experience. I worked in 2 occasions with some older people (~ 65) that were the guru of their departments, very respected and listened. While people learned a lot from these 2 people, they were not too agreeable, but quite grumpy, I could say. That limited their careers, even if they were by far the best experts in their departments and overall the most valuable contributors, they were never considered for promotions. On the other side, most well-liked people I met were professionally superficial at best, masters of talking and networking, but not delivering much at the end of the day. They were promoted first, so that the entire organization was full of these people promoting similar ones.
- sgarland 3y agoSame. The best people I ever worked with were grumpy graybeards (in spirit if not physically). Brusque is the kindest way to describe them. If you knew your shit, or at least demonstrated that you had exhausted your knowledge, they would help you. That said, it isn’t _required_ for you to be a jerk; I’ve also worked with incredibly talented people who were generally kind. But they did not go out of their way to network and schmooze. IME, people who are concerned with what others think of them are the least accomplished.
- pyuser583 3y agoI wonder if something about technical competence corresponds to being a jerk. I’ve seen this happen again and again. Nice people become jerks when they become good at tech. It doesn’t always happen. Plenty of compassionate tech wizards. I suspect it’s due to the fact that being a jerk gets reactions quicker. “You messed this up, fix it like I tell you or I’ll ruin your reputation” gets more effect than “Let me show you how this works.” I also suspect that programmers are often so lost in their heads, they don’t take good advice when it’s given. I’ve been guilty of this. I’ve decided a problem is from X part of the stack, that when an expert in Y says to look in Y I’m like “but that doesn’t effect X!”
- skeeter2020 3y agoThis is pretty much impossible above a senior IC level, where job responsibilities are focused on coordination, collaboration and other "glue". The responsibilities are going to constantly change over time, and be different for every individual. Trying to define & document this is a fool's errand. I think you're way better served to set broader objectives that are not SMART goals and then continually discuss, identify and update concrete delvierables that serve the spirit of the overal goal. This helps stop actions from being irellevant or misaligned with changing priorities, and provides a body of hard evidence for evaluating performance. I encourage everyone I manage to spend a few minutes a week capturing all their accomplishments, specifically the "soft" ones that don't have artifacts and will be forgotten (by both me and the mployee), and take an initial stab at categorizing the item against our corporate values (because I know that's what their formal review will look like). I had someone (who is a high performer) turn their multi-hour review preparation cycle into a 20-minute cut&paste operation.
- zachmu 3y agoOnce you get above a certain level sure, it's not mostly about IC metrics anymore. But that doesn't mean you can't make them objective. If a person has a set of defined responsibilities, they either delivered on them, or they didn't, and you should be able to judge that relatively objectively. And if those responsibilities can't be clearly defined, or are constantly changing, that's a signal of organizational dysfunction. I will agree that if you're in an org like that, it will be difficult to judge people's accomplishments objectively.
- 0cf8612b2e1e 3y agoWhat objective measurements are good for engineering? You can come up with superficial ones: lines of code written, tickets closed, hours seen in office, but I doubt those are strongly associated with effectiveness. Everyone in has probably had weeks where they “did nothing” because they were working on a tricky problem or doing work outside of what was quantified.
- zachmu 3y agoDon't confuse objective with easily quantified. Did they ship the features they were responsible for? Did those features turn out well, with a low defect rate? Did they promptly address bugs? Any manager paying attention can answer these questions pretty objectively.
- nine_zeros 3y ago> Don't confuse objective with easily quantified. You say this and then go on to do the exact opposite in > Did those features turn out well, with a low defect rate? And likely in this > Did they ship the features they were responsible for? So which is it?
- nucleardog 3y ago> Did those features turn out well, with a low defect rate? Is the objective measure of “well” a low defect rate? Because it sounds like you’re leaving a substantial amount of the value of an experienced employee on the table if your measure is “does it do exactly what we asked for it to do” rather than “does it solve our problems”.
- may_mccheese 3y agoIt's a flawed premise, the whole thing, worthless. In like any science, even half-baked science, there's a case and a control. Every drug, treatment, test you're given by the doctor... there was a phase trial, cases and controls, outcomes, statistics, etc Here we're saying "okay, our case is n=1 samples, no control, we're going to need to lean on our gestalt experience and knowledge based on the virtues of this company saying we're good at managing and/or our HR credentials" (this type of thinking also exists in medicine and is equally unsettling - usually wielded by people with experience AND power so it's difficult to disrupt due to the latter) The lack of control means you can't compare Peter to Paul because maybe Peter likes to stick his neck out and/or take one for the team as a show of teamwork and humility, maybe Paul is cautious and wants to maximize revenue from this position and loves to play politics. Peter is willing to go the extra mile for the company, but Paul will be the one with the perfect record. Now, if there were decent controls for either, they would show Peter is outperforming a control given challenging tasks, and Paul is underperforming by minimizing work done.
- chrisaycock 3y ago> when I rate you, on anything, my rating reveals to the world far more about me than it does about you. Perhaps there's an approach where we tell an employee to rate everyone, and then use those values only as a self-assessment of the employee.
- throwway120385 3y agoIt's called psychometric testing, and it is its own kind of garbage.
- throwway120385 3y agoThe author seems keenly unaware that all of this fake rigor in HR exists primarily so the company can say they have a process that is objective. This is not the same thing as actually having a process that's objective. More to the point, objectivity may not actually matter much because so much of how a team performs is down to how individuals within the team gel, and you can't train someone into having a personality and work style that gels if they don't. Your coworkers can learn to handle criticism but they can't learn to be brash and assertive if it's genuinely not in their personality.
- vundercind 3y agoMy experience has been that some variant of this is at play in all “data-driven” business, not just in HR. You can play spot-the-fatally-flawed-data/analysis all day long—but nobody cares, accuracy isn’t the point at all. It’s entirely a political tool.
- hirvi74 3y agoI am not going to claim my experience applies to everyone, but I think calling it a political tool is, at the very least, on the right track for many workplaces. I like working for my manager, and think he is a very rational and no BS kind of person. However, I have butted heads with the upper management above him at times for what I believe are good and legitimate reasons. During my last performance review, I had to fill out all the typical questions and surveys that us grunts are forced to answer. While reading/answering some of the questions, I just get this sinking feeling that it's all just a dog and pony show. So, in a moment of distraction while filling out all the questions, I decided to go talk to my manager directly. I simply asked him to his face, "Is there any answer I can supply to these questions that will change the opinions of upper management about me?" All my manager said back was a simple, "Nope!"
- numbsafari 3y agoSomeone I know quite well works in aerospace. The team they were on was identified as outperforming all the other similar teams. Rather than making all the other teams more like that team, they broke up the team and spread its members around like some kind of FTE pixie dust. Most of that team left the org within the next 24 months, and none of the teams they joined got better. Some got worse. Applying the “promote your best IC to be a manager” mindset applies to teams as well, and it’s a damn shame, because in both cases it usually results in the opposite of what anybody desires.
- Xcelerate 3y agoAs a data scientist, this is one of those fuzzy subjective human problems that I just don’t think is going to be solved at all anytime soon. My proposal is that we allow managers and reports to shift around much more fluidly to find a good “fit”. How to implement that? No idea.
- reaperman 3y agoIt really depends on the company. At large corporations with entrenched moats, government institutions, or quasi-government entities like large private universities, this definitely makes sense! At startups with <50 people, there may not be a runway for all that. I've been summarily let go after 1-2 months at a startup for lack of culture fit, and it made sense to me -- they simply didn't have the resources available to mentor me and help me get my head in the right place.
- neilv 3y agoHow about minimize how much anyone gets rated, and try to hire for, and structure incentives for, people motivated by: * success of the company, * success/happiness of the team, and * societally beneficial service (I assume this works best if your company also genuinely has those values. But if it does, no sense using the disproven methodology of sociopathic companies.)
- AdrianB1 3y agoIf engineers are motivated by the success of the company, you get a Boeing situation, where company is doing well and planes are bad. Happiness of the team ... the guy bringing tequila shots and cupcakes to the office will be rated the best (we had a colleague that was baking cupcakes weekly, very popular, but not contributing otherwise to the company's success). Societal service? All companies are doing it, in some way or another, otherwise they would not have any clients and would close.
- michaelmrose 3y agoAt modest size this is a self solving problem because companies full of cupcake makers and eaters simply go out of business. Boeing spent many years being led by the kind of folks who were motivated by the success of the company successfully and the wheels fell off—now literally—when it was lead by folks interested in short term profitability. People have an inherent need to be useful to matter. Mr cupcake decided to matter by making friends with food but if his performance was objectively that bad it should be fairly easy to set performance goals of some kind and term him when he doesn't meet them. This seems to be how a lot of functional places work best. Motivated people working together with relatively soft guidelines that turn into hard limits to get rid of dead weight.
- marcosdumay 3y agoI don't think engineers doing engineering can ever get themselves in a Boeing situation. It's just completely antagonistic to a maker culture. There are many ways engineers can create a dysfunctional culture, but this one is for management only.
- 3y ago
- vi_jedi 3y agoThis needs a (2015). In the 9 years since this was written I haven't seen that many attempts to do something different and these ideas are entrenched. Even newer companies that have the opportunity to do be different tend to fall back on these approaches once they hit the size that people management becomes a role. My own experience has been that performance reviews really just end up being alignment reviews, which is it's own sort of useful. I think this is possibly the most accurate signal one can expect out of these processes.
- deleted 3y ago[deleted]
- at_a_remove 3y agoAnd don't forget that brilliant "HR says we can't assign any fives, and only a handful of fours" thumb on the scale.
- dzdt 3y agoThe article is talking about ratings for existing employees, but everything said applies equally well to interview scores for potential employees. If you have candidates being scored by different interviewers, more of the variation is due to the interviewers than the candidates.
- gomezteresa5898 3y ago[dead]
- linda4151 3y agoHello everyone, do you have this feeling your partner isn’t being faithful? Are you being blackmailed by some low life? Do you want to keep track of what your kids are doing? Or do you just need to change a grade to save your GPA? Spyrecovery36 team of Hackers offer a set of Professional hacking services. Hire a hacker for Facebook, Email, Web, Database or Phone hacking. They are a team of highly skilled and professional hackers that are able to hack any social accounts, email accounts, phone, website or database. 100% guarantee of privacy and confidentiality for our customers. Mail: spyrecovery36 @ gm ail co m