4 ms·

There's a larger statistical significance issue here. Check the different scales + the degree of certainly is even worse when the you're talking about minute ch

There's a larger statistical significance issue here. Check the different scales + the degree of certainly is even worse when the you're talking about minute changes to a fraction of a percentage, year-over-year. Made a quick file to illustrate: https://www.dropbox.com/s/uq0apjt69zadiii/YC%20Blog%20Resized%20graphs%20examples.pdf?dl=0 https://www.dropbox.com/s/uq0apjt69zadiii/YC%20Blog%20Resize...

minimaxir 10y ago
The axis scales are fine as long as the comparisons are apples-to-apples. Since the charts were split between startups and nonstatups, I believe it's fair. The "statistical significance" issue with using small percentages is fine as long as the sample size is sufficiently large. YC receives thousands of applications each year, which is enough... ...although in 2016, there's only been 1 round of applications so far, whereas other years have 2 rounds of data. Therefore the sample size might be smaller, which can create doubt in the "Slack is 850% better" conclusion. Again, I do not know the exact sample size to verify.
- noahmbarr 10y ago
  I get persuasive writing, but calling "Slack: King of the Enterprise Tools" because it went from 0% in 2014 to 1.2% in 2016 is absolutely an over statement. Apples-to-apples on the chart? I'd bet a Tubman that 70%+ of the readers didn't catch the scale differences as they went chart-to-chart, following the writer down his/her path. Everyone draws the line somewhere on this stuff, and I'm just surprised where Priceonomics (who's stuff is normally AWESOME) and YC placed their collective line. Then again, maybe I'm all alone out here with this concern. ---------- *If that wasn't bad enough, did they search for other potential "enterprise tools" or just the ones that are trending today? Curious if Yammer, MS Excel, MS Office, Jira, charted in an "significant" way?
cik2e 10y ago
With a couple of thousand applications per year, each one of those small percentages is going to have a 95% bound in the neighborhood of +/-0.5%. The closer a fraction is to 0 or 1, the more significant it is per given denominator. The bounds on some of these will certainly overlap y/y but the large swings over a few years are very much statistically different. Also, if this analysis is on the full population of YC apps, there's no statistical error if you take it for what it is: trends within YC apps. Not every metric needs to be extrapolated to the maximum feasible population to be interesting. Finally, the scales on the two graphs you resized shown changes of a factor of 5-7, not counting the first data points, otherwise even more. I'd argue that your scales are the ones that are misleading in this scenario. You'd be right if they had that scale for an 82% to 94% growth, vs. the 2% to 14% or .2% to 1% that they are illusrating.

minimaxir 10y ago

The axis scales are fine as long as the comparisons are apples-to-apples. Since the charts were split between startups and nonstatups, I believe it's fair. The "statistical significance" issue with using small percentages is fine as long as the sample size is sufficiently large. YC receives thousands of applications each year, which is enough... ...although in 2016, there's only been 1 round of applications so far, whereas other years have 2 rounds of data. Therefore the sample size might be smaller, which can create doubt in the "Slack is 850% better" conclusion. Again, I do not know the exact sample size to verify.

noahmbarr 10y ago
I get persuasive writing, but calling "Slack: King of the Enterprise Tools" because it went from 0% in 2014 to 1.2% in 2016 is absolutely an over statement. Apples-to-apples on the chart? I'd bet a Tubman that 70%+ of the readers didn't catch the scale differences as they went chart-to-chart, following the writer down his/her path. Everyone draws the line somewhere on this stuff, and I'm just surprised where Priceonomics (who's stuff is normally AWESOME) and YC placed their collective line. Then again, maybe I'm all alone out here with this concern. ---------- *If that wasn't bad enough, did they search for other potential "enterprise tools" or just the ones that are trending today? Curious if Yammer, MS Excel, MS Office, Jira, charted in an "significant" way?

cik2e 10y ago

With a couple of thousand applications per year, each one of those small percentages is going to have a 95% bound in the neighborhood of +/-0.5%. The closer a fraction is to 0 or 1, the more significant it is per given denominator. The bounds on some of these will certainly overlap y/y but the large swings over a few years are very much statistically different. Also, if this analysis is on the full population of YC apps, there's no statistical error if you take it for what it is: trends within YC apps. Not every metric needs to be extrapolated to the maximum feasible population to be interesting. Finally, the scales on the two graphs you resized shown changes of a factor of 5-7, not counting the first data points, otherwise even more. I'd argue that your scales are the ones that are misleading in this scenario. You'd be right if they had that scale for an 82% to 94% growth, vs. the 2% to 14% or .2% to 1% that they are illusrating.