3 ms·

This analysis is deeply flawed. The association between skills listed in a worker's profile and hourly wages is accidental. The actual analysis we want to see

by tom_b 15y ago

This analysis is deeply flawed. The association between skills listed in a worker's profile and hourly wages is accidental. The actual analysis we want to see should be project based, showing what projects (in which I would like to see technologies used as variables) command the highest hourly wages. I would guess that most HN'ers do believe that using a better blub makes them more effective as hackers. What we don't see FTA is a link between USING the better blub and hourly wages. Maybe Clojure developers are a subset of the top 5% of Java hackers. One possibility is they are then hired to work on projects using Java and that they might command higher hourly wages (being in the top 5% of Java hackers and assuming this is demonstrable to hiring orgs) rather than being especially valued for their Clojure skills. And, make your chart x-axis start at 0 instead of 25. :) Tufte rules.

john_horton 15y ago
1) The data are what they are - these are the rates people are charging by listed skill. It would be very interesting to look at prices by actual completed projects, but that's just another approach one can take. 2) I make it very clear that there's an association, not a causal claim. I flag the fact, make a lame joke about it and put "might* in the title. 3) There's no Tufte rule about starting charts at 0 - you make choices that make it easier for people to make comparisons. Paul Krugman said it better than anyone on this topic: http://krugman.blogs.nytimes.com/2011/09/14/axes-of-evil/ http://krugman.blogs.nytimes.com/2011/09/14/axes-of-evil/
- tom_b 15y ago
  Yes, but not rates being charged for using the listed skill. As to the dreaded axis values: I think your chart, in this context, does exaggerate the variation in hourly wage by skill. At first glance, almost all your data points are between $25 and $35. Clojure hourly wage is around $31 per hour from the interactive chart. Your lowest hourly wage for a skill, haml, is $25.90. This simply is not much of a difference and your chart should make that clear. I like your article. It is interesting, and I appreciate your efforts to take this data and provide some value out of it to the community. I think we need to be more conservative in drawing any conclusions out of this data though.
- el_presidente 15y ago
  IANAS (I am not a statistician) so this may be completely wrong. You might not need to start at zero, but you do have to start with something meaningful. Like minimum wage or the poverty line.
- achompas 15y ago
  Re #3, that's Tom's point: your decision is affecting HOW readers compare the two numbers.
  - tatsuke95 15y ago
    Funny how different people see different things. The vital piece of information in the graph is the relative size of the wage. If the author had started at 0, it would be far harder to distinguish differences between the skills, by the same principles that make pie charts mostly unusable.
    - nitrogen 15y ago
      Relative (i.e. proportional/percentage) size is only apparent when you start a chart at 0. The absolute difference between 25 and 31 is 6. If you start the chart at 25, that's graphed as an ∞% difference, when it's really only a 24% difference.
      - achompas 15y ago
        Thanks for making my point for me! The chart really misleads re: the difference between languages. Do we even know if Clojure wages are statistically different from, say, Scala rates? Other problems: 1. As hinted above, what are the standard deviations? Do we even know if Clojure is statistically different from HAML? Do Clojure rates exhibit higher variation than HAML? 2. $4 difference between pattern recognition v. machine learning, where pattern recognition is, effectively, a subset of ML? We see this with legal services v. contract drafting, and (arguably) info architecture v. interaction design (less overlap with these, to be sure). Also, while John gets credit for joking about causality, I'd be very concerned about sampling bias if he uses oDesk data to examine human capital decisions.
        john_horton 15y ago
        We'll have to disagree about the axes---I think the choice of where to start axes depends on what you're trying to illustrate. For me, absolute differences in wages was what I wanted highlight. Re: Standard error bars - you're absolutely right. I should have included them, but it was my first time using the googleVis package & didn't see an easy way to add them. I don't have the numbers right in front of me, but standard deviation for each skill was on the order of ~$7/hour and each skill had to have at least 30 obs., though many where much, much higher. If we say n=50, that's an SE of about $1/hour, so clearly one shouldn't take much stock in very small differences in wages. My goal was just to unearth some interesting data---I probably should tone down the implied recommendation lest I'm responsible for a glut of unemployable lisp hackers in a few years:)
- mootothemax 15y ago
  The data are what they are - these are the rates people are charging by listed skill Right - and what about the rates for people paying for each listed skills? That's what I'm more interested in ;-)
jimbokun 15y ago
"What we don't see FTA is a link between USING the better blub and hourly wages." Neither the data nor the article make any connection between USING Clojure and hourly wages. The connection is between LEARNING Clojure and high wages. This is commonly known as the Python paradox, cited in the article. Learning currently esoteric languages signals a motivated self learner passionate about his craft. Listing Clojure in a job listing signals a company doing work that would be interesting to a motivated self learner passionate about her craft.

by tom_b 15y ago

john_horton 15y ago

1) The data are what they are - these are the rates people are charging by listed skill. It would be very interesting to look at prices by actual completed projects, but that's just another approach one can take. 2) I make it very clear that there's an association, not a causal claim. I flag the fact, make a lame joke about it and put "might* in the title. 3) There's no Tufte rule about starting charts at 0 - you make choices that make it easier for people to make comparisons. Paul Krugman said it better than anyone on this topic: http://krugman.blogs.nytimes.com/2011/09/14/axes-of-evil/ http://krugman.blogs.nytimes.com/2011/09/14/axes-of-evil/

tom_b 15y ago
Yes, but not rates being charged for using the listed skill. As to the dreaded axis values: I think your chart, in this context, does exaggerate the variation in hourly wage by skill. At first glance, almost all your data points are between $25 and $35. Clojure hourly wage is around $31 per hour from the interactive chart. Your lowest hourly wage for a skill, haml, is $25.90. This simply is not much of a difference and your chart should make that clear. I like your article. It is interesting, and I appreciate your efforts to take this data and provide some value out of it to the community. I think we need to be more conservative in drawing any conclusions out of this data though.
el_presidente 15y ago
IANAS (I am not a statistician) so this may be completely wrong. You might not need to start at zero, but you do have to start with something meaningful. Like minimum wage or the poverty line.
achompas 15y ago
Re #3, that's Tom's point: your decision is affecting HOW readers compare the two numbers.
- tatsuke95 15y ago
  Funny how different people see different things. The vital piece of information in the graph is the relative size of the wage. If the author had started at 0, it would be far harder to distinguish differences between the skills, by the same principles that make pie charts mostly unusable.
  - nitrogen 15y ago
    Relative (i.e. proportional/percentage) size is only apparent when you start a chart at 0. The absolute difference between 25 and 31 is 6. If you start the chart at 25, that's graphed as an ∞% difference, when it's really only a 24% difference.
    - achompas 15y ago
      Thanks for making my point for me! The chart really misleads re: the difference between languages. Do we even know if Clojure wages are statistically different from, say, Scala rates? Other problems: 1. As hinted above, what are the standard deviations? Do we even know if Clojure is statistically different from HAML? Do Clojure rates exhibit higher variation than HAML? 2. $4 difference between pattern recognition v. machine learning, where pattern recognition is, effectively, a subset of ML? We see this with legal services v. contract drafting, and (arguably) info architecture v. interaction design (less overlap with these, to be sure). Also, while John gets credit for joking about causality, I'd be very concerned about sampling bias if he uses oDesk data to examine human capital decisions.
      - john_horton 15y ago
        We'll have to disagree about the axes---I think the choice of where to start axes depends on what you're trying to illustrate. For me, absolute differences in wages was what I wanted highlight. Re: Standard error bars - you're absolutely right. I should have included them, but it was my first time using the googleVis package & didn't see an easy way to add them. I don't have the numbers right in front of me, but standard deviation for each skill was on the order of ~$7/hour and each skill had to have at least 30 obs., though many where much, much higher. If we say n=50, that's an SE of about $1/hour, so clearly one shouldn't take much stock in very small differences in wages. My goal was just to unearth some interesting data---I probably should tone down the implied recommendation lest I'm responsible for a glut of unemployable lisp hackers in a few years:)
mootothemax 15y ago
The data are what they are - these are the rates people are charging by listed skill Right - and what about the rates for people paying for each listed skills? That's what I'm more interested in ;-)

jimbokun 15y ago

"What we don't see FTA is a link between USING the better blub and hourly wages." Neither the data nor the article make any connection between USING Clojure and hourly wages. The connection is between LEARNING Clojure and high wages. This is commonly known as the Python paradox, cited in the article. Learning currently esoteric languages signals a motivated self learner passionate about his craft. Listing Clojure in a job listing signals a company doing work that would be interesting to a motivated self learner passionate about her craft.