12 ms·

voluntary response data is flawed I agree, but I hesitate to call this worthless. I'd plaster warning signs all over the place alarming people to think through

voluntary response data is flawed I agree, but I hesitate to call this worthless. I'd plaster warning signs all over the place alarming people to think through their reasoning and consider all the biases that would have taken place in the poll. (come to think of it, if the poll data incites that kind of skeptical thinking in some of its readers, that in itself might make the endeavor worthwhile)

lquist 13y ago
voluntary response data is flawed I agree, but I hesitate to call this worthless. I really don't mean to be rude, but that might just be because you don't understand statistics.
- dylangs1030 13y ago
  To expand on this, flawed data is something you should avoid like the plague. It's like introducing anecdotal evidence into an otherwise rigorous double-blind study. True, it can happen to give some truthful information, but if it's flawed you often don't know how flawed it is or how deep those flaws run in the variables. It's like contamination. Exceptions are simply a case of a broken clock being right twice a day.
  - HerbertKornfeld 13y ago
    I don't think anyone has bothered to try explaining what is wrong with this data (in this thread). The link discusses there are some response-bias models. For instance, maybe people always lie +5k. You can figure that out. Maybe you assume it's really a function of f(x)*base_salary and do something structured based on their salary as the bias. It's, of course, perfectly fine to interpret this kind of survey as the response of those in the population that decided to take it. In this case, it's readers of hacker news who filled it in. I certainly wouldn't do that, I only registered to post this comment... anyway. You could also try to validate this data against any other survey data, or by cross-linking LinkedIn data with mortgage data for a true dataset. "And that's data science, bro."
dylangs1030 13y ago
Flawed data is sometimes worse than no data at all. Data with sampling bias, too small a sample size, lack of double-blind rigor, etc. Strictly speaking, "surveys" are only really used by the softer sciences, like sociology and psychology, and both fields agree it's the least rigorous (albeit easiest) method of accumulating data. Corollary to this, a little bit of flawed data can sometimes give a good idea in the general direction, but more often than not it's worthless because it shows false correlations, leads to erroneous conclusions, stimulates debate in the wrong area, questions prior evidence, causes one to hypothesize irrelevant answers, etc.
- jleader 13y ago
  Boy, talk about "the perfect is the enemy of the good"! Is salary negotiation a branch of particle physics now? Yes, this survey is much less valid than if we could magically collect all salary data for all developers in the area. But I don't have that magic wand, and this survey is far more useful to me than its closest competitor, which is "a buddy moved to the Bay area a few years ago and told me what he'd been offered".
kevinpet 13y ago
If you had absolutely no idea for what kind of salaries people got around here, then this data would be useful. Is it $25k or $250k? This can answer that question. But what people are really interested in is "is $125k high or low" and this data is useless for answering that question. It is worse than useless because it may lead you to believe something that isn't true and think you have a valid reason for believing it. It's worse to know something that isn't true than to correctly know what you don't know.
- skrebbel 13y ago
  This. Now, I don't actually understand statistics at all, so please correct me if I'm missing something, but as a European I find this poll pretty telling, and a good indication of what to expect if I'd consider moving to the Valley. Clearly, at least the maker of the poll seems to assume that I can expect to earn at least $80k-ish. I earn a bit over half that now. This is a very big difference. Additionally, given that most people in this poll earn well over $100k, and I'm a pretty experienced programmer, this poll teaches me that asking for, say $120k, at an SV company, isn't an odd request. People won't frown or laugh at me. They might not want to pay me that, and just as well I might be undercharging, but it's not out of the question. Can I draw these conclusions? If not, why not?
  - zobzu 13y ago
    You get paid a little lower if you're going on visa, so its more like 100-110kish. I spose the 300k ppl are the ceos! (yeh, i wish haha)
    - skrebbel 13y ago
      Could you explain that? Why would companies pay less to people on visas?
      - joshAg 13y ago
        because h1b visas require you to stay employed (you have like 30 days before they deport you) and they require you to keep the same job title throughout the whole greencard process or your waiting period starts over.
smoyer 13y ago
I wouldn't call it worthless either ... and there are enough responses that you can see a peak without graphing it. So my guess is that people are responding truthfully but the bigger issue is that we don't know how many years of experience each of the respondents has.

lquist 13y ago

voluntary response data is flawed I agree, but I hesitate to call this worthless. I really don't mean to be rude, but that might just be because you don't understand statistics.

dylangs1030 13y ago
To expand on this, flawed data is something you should avoid like the plague. It's like introducing anecdotal evidence into an otherwise rigorous double-blind study. True, it can happen to give some truthful information, but if it's flawed you often don't know how flawed it is or how deep those flaws run in the variables. It's like contamination. Exceptions are simply a case of a broken clock being right twice a day.
- HerbertKornfeld 13y ago
  I don't think anyone has bothered to try explaining what is wrong with this data (in this thread). The link discusses there are some response-bias models. For instance, maybe people always lie +5k. You can figure that out. Maybe you assume it's really a function of f(x)*base_salary and do something structured based on their salary as the bias. It's, of course, perfectly fine to interpret this kind of survey as the response of those in the population that decided to take it. In this case, it's readers of hacker news who filled it in. I certainly wouldn't do that, I only registered to post this comment... anyway. You could also try to validate this data against any other survey data, or by cross-linking LinkedIn data with mortgage data for a true dataset. "And that's data science, bro."

dylangs1030 13y ago

Flawed data is sometimes worse than no data at all. Data with sampling bias, too small a sample size, lack of double-blind rigor, etc. Strictly speaking, "surveys" are only really used by the softer sciences, like sociology and psychology, and both fields agree it's the least rigorous (albeit easiest) method of accumulating data. Corollary to this, a little bit of flawed data can sometimes give a good idea in the general direction, but more often than not it's worthless because it shows false correlations, leads to erroneous conclusions, stimulates debate in the wrong area, questions prior evidence, causes one to hypothesize irrelevant answers, etc.

jleader 13y ago
Boy, talk about "the perfect is the enemy of the good"! Is salary negotiation a branch of particle physics now? Yes, this survey is much less valid than if we could magically collect all salary data for all developers in the area. But I don't have that magic wand, and this survey is far more useful to me than its closest competitor, which is "a buddy moved to the Bay area a few years ago and told me what he'd been offered".

kevinpet 13y ago

If you had absolutely no idea for what kind of salaries people got around here, then this data would be useful. Is it $25k or $250k? This can answer that question. But what people are really interested in is "is $125k high or low" and this data is useless for answering that question. It is worse than useless because it may lead you to believe something that isn't true and think you have a valid reason for believing it. It's worse to know something that isn't true than to correctly know what you don't know.

skrebbel 13y ago
This. Now, I don't actually understand statistics at all, so please correct me if I'm missing something, but as a European I find this poll pretty telling, and a good indication of what to expect if I'd consider moving to the Valley. Clearly, at least the maker of the poll seems to assume that I can expect to earn at least $80k-ish. I earn a bit over half that now. This is a very big difference. Additionally, given that most people in this poll earn well over $100k, and I'm a pretty experienced programmer, this poll teaches me that asking for, say $120k, at an SV company, isn't an odd request. People won't frown or laugh at me. They might not want to pay me that, and just as well I might be undercharging, but it's not out of the question. Can I draw these conclusions? If not, why not?
- zobzu 13y ago
  You get paid a little lower if you're going on visa, so its more like 100-110kish. I spose the 300k ppl are the ceos! (yeh, i wish haha)
  - skrebbel 13y ago
    Could you explain that? Why would companies pay less to people on visas?
    - joshAg 13y ago
      because h1b visas require you to stay employed (you have like 30 days before they deport you) and they require you to keep the same job title throughout the whole greencard process or your waiting period starts over.

smoyer 13y ago

I wouldn't call it worthless either ... and there are enough responses that you can see a peak without graphing it. So my guess is that people are responding truthfully but the bigger issue is that we don't know how many years of experience each of the respondents has.