5 ms·

> If someone came to me and said "I did a self-selecting self-reporting survey of salaries, and it says I'm underpaid", I think I'd be more likely to fire them

> If someone came to me and said "I did a self-selecting self-reporting survey of salaries, and it says I'm underpaid", I think I'd be more likely to fire them for incompetence than to give them a raise. Even if besides them being wrong in this one thing, they would actually be competent and underpaid? If the answer to the above question is no, why do you even have to elaborate. Then it doesn't actually matter at all if the employee complains rightly or wrongly! I have to conclude the answer has to be yes, then. It's perfectly understandable that you, as a manager or employer, don't want to pay people for their merit if you don't have to. But it's not something you should be proud of.

cperciva 11y ago
Drawing conclusions from self-selected self-reported data (without a careful process for validation and debiasing) is a big red flag for (a) being ignorant of basic statistics, and (b) not even being aware of your ignorance of basic statistics. I mean, "why do we use random samples?" is covered in the first lecture of a 1st year stats course. I expect developers to be familiar with basic statistics.
- dlitz 11y ago
  Wow. Colin, are you just trying to win an argument online or are you really that flippant when it comes to firing an employee? Keep in mind that we're talking about people's livelihoods, here---it's the sort of thing that can stress people out to the point of mental breakdown, even if you're not actively threatening them. Firstly, this isn't a job interview. If someone's been working with you for long enough that they want to discuss a salary adjustment, then shouldn't your assessment of the value that they bring to the table be more informed than looking for "red flags" on the spot? Secondly, developers are generally not familiar with basic statistics. If it's important to your business that they be better-than-average at stats, then budget some time and money for training. Thirdly, in the real world, we make decisions from incomplete and biased information all the time. Sometimes, those decisions turn out to be wrong, but on the whole, it still works out better than pretending that we have no information at all. If you're in management and you have access to more complete information, then the smart thing for an employee to do is to have a conversation with you. It makes no sense to fire someone for that. I have a lot of respect for your technical work, but the management style you're describing so far sounds authoritarian and abusive. I really hope that's not how you'd actually treat your reports, in practice.
  - lawnchair_larry 11y ago
    He has asperger's.
  - cperciva 11y ago
    are you really that flippant when it comes to firing an employee? I don't know; I've never had to fire anyone yet. I'll admit that when writing at 1AM I may have let some hyperbole slip into my comments. It's quite possible that my reaction would be "great, your assignment for the next week is to do a rigorous statistical analysis of this data". On the other hand, as someone else commented, I said "more likely"; and that's probably true. I can't imagine a situation where I'd accept a naive analysis of self-selected self-reporting data as having any substance at all, so I would certainly not give someone a raise based on that. discuss a salary adjustment Discussing salary is fine. My point is that bad statistics is worse than nothing in this context.
    - asgard1024 11y ago
      > My point is that bad statistics is worse than nothing in this context. Actually, even this point is wrong, IMHO. I can easily imagine situations where you get just one data point and correctly infer from it that you're underpaid. One example is that someone who obviously less competent than you will reveal to you higher salary than you have. Another example is that someone has access to information will tell you that your salary is low, and you have good reason to trust that person. I understand that you as an employer naturally don't think that you are or ever will be biased in this way. But it still may happen in the wild. In any case, I think it was already said, if you still think that bad statistics is worse than none, you should make the information publicly available. It's the similar thing as with rumors.
      - cperciva 11y ago
        I can easily imagine situations where you get just one data point and correctly infer from it that you're underpaid. One data point isn't even bad statistics. It's anecdote. Maybe the other person is being vastly overpaid.
- anu_gupta 11y ago
  The Conversation "Hey Coliin, I've been chatting with a couple of people, and I've got a concern about my salary" "You're fired" The End
  - vacri 11y ago
    So here is another example of not understanding the basics of statistics. The statement was 'more likely to fire than give a raise'. This does not mean that these are the only two options available. 'More likely' just means the chance is higher overall, whether it's 99% likely to fire or 2% likely to fire. It's also not what was said - it wasn't "chatting led to a concern", it was "self-reported, self-selecting study says X". That stuff really is a red flag to folks who have a decent grounding in statistics.
    - anu_gupta 11y ago
      Literal much? Brilliant mansplaining though!
      - vacri 11y ago
        Why are you complaining that my comment is taking yours seriously, when yours is taking colin's seriously? Nice hypocrisy there. I was actually referencing your underlying understanding for your ridicule, not the joke itself.
        anu_gupta 11y ago
        Can you point out where Colin indicated (clearly too subtly for me) that his repeated comments on this subject were a joke and should not be taken seriously. Once you do that, I'll try and explain (mansplain if necessary) how a play of 2 lines and 1 act is probably not a very serious response.
      - oldmanjay 11y ago
        It's a fairly reliable signal that once you start to use words like "mansplaining" you're well past having a useful point to make in a reasonable discussion.
- deleted 11y ago
  [deleted]
- wdewind 11y ago
  > a careful process for validation and debiasing What might something like this look like?
  - cperciva 11y ago
    Generally speaking, you would want to take a random sample of employees, verify that the information they self-reported was correct, and perform a regression analysis of response rate vs. other characteristics (age, seniority, gender, salary, ethnicity, etc.). In this particular context, the biggest threats to data integrity are probably: 1. People lie, and men probably lie about their salaries more than women, 2. Willingness to divulge salary is highly culture-dependent, and in societies where salary is seen as a sign of social success, can be highly salary-dependent (and probably more so for men than for women). If you're interested in learning more I suggest looking for information about how national census data is processed; they have lots of experience dealing with these sorts of issues (albeit with different factors influencing response rates and data reliability).
    - wdewind 11y ago
      Interesting, thanks!

cperciva 11y ago

Drawing conclusions from self-selected self-reported data (without a careful process for validation and debiasing) is a big red flag for (a) being ignorant of basic statistics, and (b) not even being aware of your ignorance of basic statistics. I mean, "why do we use random samples?" is covered in the first lecture of a 1st year stats course. I expect developers to be familiar with basic statistics.

dlitz 11y ago
Wow. Colin, are you just trying to win an argument online or are you really that flippant when it comes to firing an employee? Keep in mind that we're talking about people's livelihoods, here---it's the sort of thing that can stress people out to the point of mental breakdown, even if you're not actively threatening them. Firstly, this isn't a job interview. If someone's been working with you for long enough that they want to discuss a salary adjustment, then shouldn't your assessment of the value that they bring to the table be more informed than looking for "red flags" on the spot? Secondly, developers are generally not familiar with basic statistics. If it's important to your business that they be better-than-average at stats, then budget some time and money for training. Thirdly, in the real world, we make decisions from incomplete and biased information all the time. Sometimes, those decisions turn out to be wrong, but on the whole, it still works out better than pretending that we have no information at all. If you're in management and you have access to more complete information, then the smart thing for an employee to do is to have a conversation with you. It makes no sense to fire someone for that. I have a lot of respect for your technical work, but the management style you're describing so far sounds authoritarian and abusive. I really hope that's not how you'd actually treat your reports, in practice.
- lawnchair_larry 11y ago
  He has asperger's.
- cperciva 11y ago
  are you really that flippant when it comes to firing an employee? I don't know; I've never had to fire anyone yet. I'll admit that when writing at 1AM I may have let some hyperbole slip into my comments. It's quite possible that my reaction would be "great, your assignment for the next week is to do a rigorous statistical analysis of this data". On the other hand, as someone else commented, I said "more likely"; and that's probably true. I can't imagine a situation where I'd accept a naive analysis of self-selected self-reporting data as having any substance at all, so I would certainly not give someone a raise based on that. discuss a salary adjustment Discussing salary is fine. My point is that bad statistics is worse than nothing in this context.
  - asgard1024 11y ago
    > My point is that bad statistics is worse than nothing in this context. Actually, even this point is wrong, IMHO. I can easily imagine situations where you get just one data point and correctly infer from it that you're underpaid. One example is that someone who obviously less competent than you will reveal to you higher salary than you have. Another example is that someone has access to information will tell you that your salary is low, and you have good reason to trust that person. I understand that you as an employer naturally don't think that you are or ever will be biased in this way. But it still may happen in the wild. In any case, I think it was already said, if you still think that bad statistics is worse than none, you should make the information publicly available. It's the similar thing as with rumors.
    - cperciva 11y ago
      I can easily imagine situations where you get just one data point and correctly infer from it that you're underpaid. One data point isn't even bad statistics. It's anecdote. Maybe the other person is being vastly overpaid.
anu_gupta 11y ago
The Conversation "Hey Coliin, I've been chatting with a couple of people, and I've got a concern about my salary" "You're fired" The End
- vacri 11y ago
  So here is another example of not understanding the basics of statistics. The statement was 'more likely to fire than give a raise'. This does not mean that these are the only two options available. 'More likely' just means the chance is higher overall, whether it's 99% likely to fire or 2% likely to fire. It's also not what was said - it wasn't "chatting led to a concern", it was "self-reported, self-selecting study says X". That stuff really is a red flag to folks who have a decent grounding in statistics.
  - anu_gupta 11y ago
    Literal much? Brilliant mansplaining though!
    - vacri 11y ago
      Why are you complaining that my comment is taking yours seriously, when yours is taking colin's seriously? Nice hypocrisy there. I was actually referencing your underlying understanding for your ridicule, not the joke itself.
      - anu_gupta 11y ago
        Can you point out where Colin indicated (clearly too subtly for me) that his repeated comments on this subject were a joke and should not be taken seriously. Once you do that, I'll try and explain (mansplain if necessary) how a play of 2 lines and 1 act is probably not a very serious response.
    - oldmanjay 11y ago
      It's a fairly reliable signal that once you start to use words like "mansplaining" you're well past having a useful point to make in a reasonable discussion.
deleted 11y ago
[deleted]
wdewind 11y ago
> a careful process for validation and debiasing What might something like this look like?
- cperciva 11y ago
  Generally speaking, you would want to take a random sample of employees, verify that the information they self-reported was correct, and perform a regression analysis of response rate vs. other characteristics (age, seniority, gender, salary, ethnicity, etc.). In this particular context, the biggest threats to data integrity are probably: 1. People lie, and men probably lie about their salaries more than women, 2. Willingness to divulge salary is highly culture-dependent, and in societies where salary is seen as a sign of social success, can be highly salary-dependent (and probably more so for men than for women). If you're interested in learning more I suggest looking for information about how national census data is processed; they have lots of experience dealing with these sorts of issues (albeit with different factors influencing response rates and data reliability).
  - wdewind 11y ago
    Interesting, thanks!