6 ms·

Jupyter Notebook Analysis of Salary Data Spreadsheet

91 points by talloaktrees 11y ago

talloaktrees 11y ago
github repo: https://github.com/jrenner/hacker-news-salaries-data https://github.com/jrenner/hacker-news-salaries-data
Toenex 11y ago
I may have missed it but it looks like the plot from cell 'In [64]' hasn't converted from Euros and GBP to Dollars.
slashcom 11y ago
Most interesting to me was that the gender gap was so much more pronounced in SV than outside non-SV male median: 97000 non-SV female median: 90000 (92% of male) SV male median: 137120 SV female median: 99187 (72% of male!) Unless there's some external factor here, things look pretty damning in SV...
- pedrosorio 11y ago
  Without looking at the distribution of job_title between male/female in SV and non-SV, these numbers are meaningless.
  - Xorlev 11y ago
    Not to mention, the sample sizes are quite skewed -- the quantiles might line up better with more data.
NeutronBoy 11y ago
It's interesting - the company I work for (non-US, consulting) did analysis of gender pay gaps internally and noted that on average, females get 70% of male pay. However, within the same role, the difference was just 3%. Their focus is now on flexible working and figuring out why women are excluded/not interested in particular roles.
- qznc 11y ago
  That seems to be the current state of knowledge. > Although additional research in this area is clearly needed, this study leads to the unambiguous conclusion that the differences in the compensation of men and women are the result of a multitude of factors and that the raw wage gap should not be used as the basis to justify corrective action. Indeed, there may be nothing to correct. The differences in raw wages may be almost entirely the result of the individual choices being made by both male and female workers.[21] – U.S. Department of Labor as cited by Wikipedia https://en.wikipedia.org/wiki/Gender_pay_gap https://en.wikipedia.org/wiki/Gender_pay_gap
  - pc86 11y ago
    And yet people will still throw out the ~76 cents on the dollar figure that has been disproven again and again. Even 3% may be statistically significant so I don't see why some people feel the need to be blatantly dishonest regarding the problem.
    - brianwawok 11y ago
      Yes people used horrible and biased stats all the time to prove their point. And most people believe it.
    - wodenokoto 11y ago
      > Even 3% may be statistically significant so I don't see why some people feel the need to be blatantly dishonest regarding the problem. Statistical significant doesn't mean that it is significant.
    - matthewowen 11y ago
      The issue isn't just about "choice" etc: it's also about how jobs that women predominantly do are lower in prestige and pay than jobs that men predominantly do. The change in programming from being a low prestige, low salary job done by women to being a high prestige, high salary job done by men is sometimes cited an example of that. I don't know if there are flaws in that example, but I think the overall idea makes sense. I read an interesting article about this, but I don't recall the link. Sorry.
friendzis 11y ago
What strikes me that salary~experience (both company and overall) correlation is basically nonexistent in this particular dataset, at least looking visually.
- zanybear 11y ago
  I had the same thought, but then you have to remember years of experience does not equal mastery.
torrance 11y ago
I appreciate this if only as a great example of using jupyter notebook and pandas. Thanks!
stared 11y ago
Log-log scales would really help. Otherwise - points in the low range are too dense to be useful (big differences in density make be hard to spot, or even - invisible).
wyldfire 11y ago
As best I can tell, that last graph would've been clearer as a bar graph and not a scatter plot (or at least better labels). The x axis merely indicates the location. So, "mean annual salaries by location (in kUSD)" might be a good title for the last one.
- eoinmurray92 11y ago
  Also are all the salaries in USD? I know Europe might pay less in tech - but the bottom three paying locations being Sweden, Berlin and London seems strange to me. Perhaps those locations salaries are not being converted to USD...
deleted 11y ago
[deleted]
jchendy 11y ago
It looks like all the analysis is just on base salary. It would be much more interesting if it included bonus and stock.
TheAppGuy 11y ago
I've got 99 problems, but a boss ain't one!
chucky_z 11y ago
Interesting there seems to be a shift in the 0-2 year job length position. I wonder if this is because the common trend is to jump ship at 2 yrs, so folks are going out job hunting, and getting a competitive raise at their current position with the job offer in hand?
markovbling 11y ago
Awesome! Think you should convert the currencies at current spot rates instead of just removing the currency sign though :)
metakermit 11y ago
I'm trying to collect a similar dataset for freelancer hourly rates on Reddit: https://www.reddit.com/r/freelance/comments/4bsk14/freelance_hourly_rates_a_crowdsourced_spreadsheet/ https://www.reddit.com/r/freelance/comments/4bsk14/freelance... Tried it on Hacker News as well, but it didn't get any attention: https://news.ycombinator.com/item?id=11337833 https://news.ycombinator.com/item?id=11337833

91 points by talloaktrees 11y ago

talloaktrees 11y ago

github repo: https://github.com/jrenner/hacker-news-salaries-data https://github.com/jrenner/hacker-news-salaries-data

Toenex 11y ago

I may have missed it but it looks like the plot from cell 'In [64]' hasn't converted from Euros and GBP to Dollars.

slashcom 11y ago

Most interesting to me was that the gender gap was so much more pronounced in SV than outside non-SV male median: 97000 non-SV female median: 90000 (92% of male) SV male median: 137120 SV female median: 99187 (72% of male!) Unless there's some external factor here, things look pretty damning in SV...

pedrosorio 11y ago
Without looking at the distribution of job_title between male/female in SV and non-SV, these numbers are meaningless.
- Xorlev 11y ago
  Not to mention, the sample sizes are quite skewed -- the quantiles might line up better with more data.

NeutronBoy 11y ago

It's interesting - the company I work for (non-US, consulting) did analysis of gender pay gaps internally and noted that on average, females get 70% of male pay. However, within the same role, the difference was just 3%. Their focus is now on flexible working and figuring out why women are excluded/not interested in particular roles.

qznc 11y ago
That seems to be the current state of knowledge. > Although additional research in this area is clearly needed, this study leads to the unambiguous conclusion that the differences in the compensation of men and women are the result of a multitude of factors and that the raw wage gap should not be used as the basis to justify corrective action. Indeed, there may be nothing to correct. The differences in raw wages may be almost entirely the result of the individual choices being made by both male and female workers.[21] – U.S. Department of Labor as cited by Wikipedia https://en.wikipedia.org/wiki/Gender_pay_gap https://en.wikipedia.org/wiki/Gender_pay_gap
- pc86 11y ago
  And yet people will still throw out the ~76 cents on the dollar figure that has been disproven again and again. Even 3% may be statistically significant so I don't see why some people feel the need to be blatantly dishonest regarding the problem.
  - brianwawok 11y ago
    Yes people used horrible and biased stats all the time to prove their point. And most people believe it.
  - wodenokoto 11y ago
    > Even 3% may be statistically significant so I don't see why some people feel the need to be blatantly dishonest regarding the problem. Statistical significant doesn't mean that it is significant.
  - matthewowen 11y ago
    The issue isn't just about "choice" etc: it's also about how jobs that women predominantly do are lower in prestige and pay than jobs that men predominantly do. The change in programming from being a low prestige, low salary job done by women to being a high prestige, high salary job done by men is sometimes cited an example of that. I don't know if there are flaws in that example, but I think the overall idea makes sense. I read an interesting article about this, but I don't recall the link. Sorry.

friendzis 11y ago

What strikes me that salary~experience (both company and overall) correlation is basically nonexistent in this particular dataset, at least looking visually.

zanybear 11y ago
I had the same thought, but then you have to remember years of experience does not equal mastery.

torrance 11y ago

I appreciate this if only as a great example of using jupyter notebook and pandas. Thanks!

stared 11y ago

Log-log scales would really help. Otherwise - points in the low range are too dense to be useful (big differences in density make be hard to spot, or even - invisible).

wyldfire 11y ago

As best I can tell, that last graph would've been clearer as a bar graph and not a scatter plot (or at least better labels). The x axis merely indicates the location. So, "mean annual salaries by location (in kUSD)" might be a good title for the last one.

eoinmurray92 11y ago
Also are all the salaries in USD? I know Europe might pay less in tech - but the bottom three paying locations being Sweden, Berlin and London seems strange to me. Perhaps those locations salaries are not being converted to USD...

deleted 11y ago

[deleted]

jchendy 11y ago

It looks like all the analysis is just on base salary. It would be much more interesting if it included bonus and stock.

TheAppGuy 11y ago

I've got 99 problems, but a boss ain't one!

chucky_z 11y ago

Interesting there seems to be a shift in the 0-2 year job length position. I wonder if this is because the common trend is to jump ship at 2 yrs, so folks are going out job hunting, and getting a competitive raise at their current position with the job offer in hand?

markovbling 11y ago

Awesome! Think you should convert the currencies at current spot rates instead of just removing the currency sign though :)

metakermit 11y ago

I'm trying to collect a similar dataset for freelancer hourly rates on Reddit: https://www.reddit.com/r/freelance/comments/4bsk14/freelance_hourly_rates_a_crowdsourced_spreadsheet/ https://www.reddit.com/r/freelance/comments/4bsk14/freelance... Tried it on Hacker News as well, but it didn't get any attention: https://news.ycombinator.com/item?id=11337833 https://news.ycombinator.com/item?id=11337833