4 ms·

When you don't start your y-axis at 0, you skew the interpretation of your data. At best, this is a significant mistake, and at worst, this is intentionally mis

by pflats 14y ago

When you don't start your y-axis at 0, you skew the interpretation of your data. At best, this is a significant mistake, and at worst, this is intentionally misleading. Take this graph: http://i.imgur.com/bBzCK.png http://i.imgur.com/bBzCK.png It looks like women are rated as more than twice as smart as men. Huge difference. Except until you run the numbers. Women are rated about 4.3% "smarter" than men. Not twice as smart, like the graph implies. Not 20% smarter. Not even 5%. Please, pay attention to your graphs. They're great tools, but they can mislead as much as they can help elucidate.

aero142 14y ago
And worse than that, the y-axis range shifts. For example, the range on the Fat vs. Normal chart is much smaller than the surrounding charts, giving the illusion that the spread is much larger than it is. (site is down or I'd link).
bvi 14y ago
True, and I was having a tough time deciding which option to take (whether to start the y-axis at 0, or 5). As far as averages are concerned, the differences are in the tenths and hundredths - so starting the y-axis at 0 would have it more difficult to see any differences across the graphs.
- mhartl 14y ago
  starting the y-axis at 0 would have it more difficult to see any differences across the graphs. That's the point: the differences are small.
  - scott_s 14y ago
    I would go further: the differences probably are not significant. (Small differences may still be significant. In this case, I doubt it.)
- DevX101 14y ago
  | so starting the y-axis at 0 would have it more difficult to see any differences across the graphs Then maybe there isn't a real difference. See statistical significance.
- pflats 14y ago
  A (somewhat blunt) rule of thumb, not just for you, but for anyone in this endeavor: In the future, if you are making a graph like this, and the differences are so small that they don't show up when your y-axis starts at 0, they probably aren't important enough to talk about. If you've taken a stats class, you probably know there are times when small differences are meaningful. Any time you're analyzing your own data, though, these differences are rarely worth even considering. Taking a survey and analyzing the resulting data is very hard. There are countless nuances that can affect your results, both mathematical and psychological. Look at political polling - even the people who do it for a living regularly get things wrong.
hammock 14y ago
Gah, this data is so interesting, but is not presented very well. First change I would make is put all the red bars next to each other, and the blue bars next to each other; there is more value in comparing across enthicities/genders than there is in comparing across variables. Second change I would make it get rid of the eye-popping primary colors and choose two more neutral ones (two different shades of beige maybe?) Finally, Y-axis scale changes too frequently. Also error bars would be nice as has been mentioned. edit: Now that I think more about it, at a glance it would be nice to show a scatter plot with a dot for each of Male/Female/Ethnicity/etc in the way that you have a scatter plot on your home page. http://judg.me http://judg.me
- jessriedel 14y ago
  > Also error bars would be nice as has been mentioned. Not just nice but absolutely crucial. If you're looking at differences of 2%, you need tens of thousands of samples to say anything definitive. He's only got 1000 photos total, so I'm pretty sure most of this article is just an analysis of random noise.
  - hammock 14y ago
    You're right. Also, maybe I'm just dense but it took me a REALLY long time to wrap my head around the scoring matrix of the website, and even after a few practices, each photo I still find myself having to think painfully hard about what each quadrant means. I have to imagine that this would add even more noise to the data, if people's heads are hurting and don't know where to click. One suggestion would be to provide an intuitive guide for each of the quadrants, not just the axes, e.g. "Outgoing idiot," "Shy nerd" etc. but of course you risk biasing the responses based on the language you use. The other option would be to simplify the input, only rating one variable at a time.
    - jessriedel 14y ago
      Yep, there are two sources of variance: the variance amongst a sample of 1000 photos selected from a hypothetical pool of all people, and the variance in how the sample of 100 subjects rated any given photo compared to the "intrinsic" qualities of the photo.

by pflats 14y ago

aero142 14y ago

And worse than that, the y-axis range shifts. For example, the range on the Fat vs. Normal chart is much smaller than the surrounding charts, giving the illusion that the spread is much larger than it is. (site is down or I'd link).

bvi 14y ago

True, and I was having a tough time deciding which option to take (whether to start the y-axis at 0, or 5). As far as averages are concerned, the differences are in the tenths and hundredths - so starting the y-axis at 0 would have it more difficult to see any differences across the graphs.

mhartl 14y ago
starting the y-axis at 0 would have it more difficult to see any differences across the graphs. That's the point: the differences are small.
- scott_s 14y ago
  I would go further: the differences probably are not significant. (Small differences may still be significant. In this case, I doubt it.)
DevX101 14y ago
| so starting the y-axis at 0 would have it more difficult to see any differences across the graphs Then maybe there isn't a real difference. See statistical significance.
pflats 14y ago
A (somewhat blunt) rule of thumb, not just for you, but for anyone in this endeavor: In the future, if you are making a graph like this, and the differences are so small that they don't show up when your y-axis starts at 0, they probably aren't important enough to talk about. If you've taken a stats class, you probably know there are times when small differences are meaningful. Any time you're analyzing your own data, though, these differences are rarely worth even considering. Taking a survey and analyzing the resulting data is very hard. There are countless nuances that can affect your results, both mathematical and psychological. Look at political polling - even the people who do it for a living regularly get things wrong.

hammock 14y ago

Gah, this data is so interesting, but is not presented very well. First change I would make is put all the red bars next to each other, and the blue bars next to each other; there is more value in comparing across enthicities/genders than there is in comparing across variables. Second change I would make it get rid of the eye-popping primary colors and choose two more neutral ones (two different shades of beige maybe?) Finally, Y-axis scale changes too frequently. Also error bars would be nice as has been mentioned. edit: Now that I think more about it, at a glance it would be nice to show a scatter plot with a dot for each of Male/Female/Ethnicity/etc in the way that you have a scatter plot on your home page. http://judg.me http://judg.me

jessriedel 14y ago
> Also error bars would be nice as has been mentioned. Not just nice but absolutely crucial. If you're looking at differences of 2%, you need tens of thousands of samples to say anything definitive. He's only got 1000 photos total, so I'm pretty sure most of this article is just an analysis of random noise.
- hammock 14y ago
  You're right. Also, maybe I'm just dense but it took me a REALLY long time to wrap my head around the scoring matrix of the website, and even after a few practices, each photo I still find myself having to think painfully hard about what each quadrant means. I have to imagine that this would add even more noise to the data, if people's heads are hurting and don't know where to click. One suggestion would be to provide an intuitive guide for each of the quadrants, not just the axes, e.g. "Outgoing idiot," "Shy nerd" etc. but of course you risk biasing the responses based on the language you use. The other option would be to simplify the input, only rating one variable at a time.
  - jessriedel 14y ago
    Yep, there are two sources of variance: the variance amongst a sample of 1000 photos selected from a hypothetical pool of all people, and the variance in how the sample of 100 subjects rated any given photo compared to the "intrinsic" qualities of the photo.