4 ms·

Simply removing "outliers" just because you don't like them isn't a rigorous approach.

by pcrh 11y ago

erikb 11y ago
But maybe lying outside of the range of the others might be a good reason? I'm no expert. Just a guess.
- pcrh 11y ago
  Do you actually know what the "true" range is? The data presented is a sample, the "true" distribution might be different from what is shown, and if you start 'cleaning" the data based on subjective judgements, you might never discover the true range.
  - minimaxir 11y ago
    > The data presented is a sample, the "true" distribution might be different from what is shown, and if you start 'cleaning" the data based on subjective judgements, you might never discover the true range. Note that if the sample is large enough and selected uniformly, the need for the analysis on the entire population is unnecessary. In this case, it does't apply, though.
    - pcrh 11y ago
      Why would it not apply in this case? The sample isn't particularly large, and we don't know the extent of possible selection biases. For example one obvious bias is that the sample is restricted to data sent through Docsend, thus likely does not represent a uniform sampling of all pitches. In any case, there is no prior reason to exclude outliers; any large enough representative sampling of a Gaussian distribution would include "outliers".
      - minimaxir 11y ago
        That's what I meant; curse my ambigious "it."
  - erikb 11y ago
    The thing is that the three dots that lay "outside the range were the other dots are" (not the "true" range which we of course don't know) have a higher impact on the conclusion than the amount of them justifies. That's why taking them out should not harm but help. This is not the same as taking datasets out of a medical study because they got sick by the new medicine. E.g., if you look at the left diagram. The upward swing in the red curve is probably the result of all the left points (199 points). But the downward trend is as far as I can see the result of 1 point on the right. So can we trust that downward trend as much as the upward trend? Probably not. Additionally removing the two dots at the 150 meetings point might result in the curve not having such a strong incline, but one that is backed by 196 points. I really don't have more than basic education in statistics, but to me it looks like a slight but constant upward trend until 100 investors contacted and after that we simply don't have enough data. If you think about what the diagram talks about getting a constant upward line or even more likely a square root shaped line is very likely. Getting a bell curve is unlikely, just from thinking about the topic. What should happen that contacting more than X investors would result in less meetings? In the second diagram removing the two dots at $4.5m and the two dots above 200 investors met leaves us with 196 data points that show a fairly equal spread and that the amount of investors contacted might not influence that much how much money you get in the end. That's also good because it's expected to find that company evaluation depends on company value (customers, market, team, etc) and not on how many investors you contact. You make the data more conclusive by removing data points that have a big impact on your results without the backup of other data points. That's what handling outliers is.
    - pcrh 11y ago
      As to the investors contacted vs meetings plot, the red line is evidently bogus. It is likely caused by trying to fit the data to some polynomial for which there is no a priori justification. A straight-line fit would look more reasonable and also be less affected by one "outlier" who contacted 300 investors. I would agree that cutting-off curve fitting at 100-150 investors would look prettier; the thing to do in that case would be either to not attempt to fit a curve and just show the data, or alternately to fit only the earlier part (so the red line stops half-way across the x-axis), but show all the data. The only reason for doing a curve fit on this kind of data is to show that the data fit some prior model, showing that the model is likely correct, or to show where some statistical value lies.

Simply removing "outliers" just because you don't like them isn't a rigorous approach.

by pcrh 11y ago

Simply removing "outliers" just because you don't like them isn't a rigorous approach.

erikb 11y ago

But maybe lying outside of the range of the others might be a good reason? I'm no expert. Just a guess.

pcrh 11y ago
Do you actually know what the "true" range is? The data presented is a sample, the "true" distribution might be different from what is shown, and if you start 'cleaning" the data based on subjective judgements, you might never discover the true range.
- minimaxir 11y ago
  > The data presented is a sample, the "true" distribution might be different from what is shown, and if you start 'cleaning" the data based on subjective judgements, you might never discover the true range. Note that if the sample is large enough and selected uniformly, the need for the analysis on the entire population is unnecessary. In this case, it does't apply, though.
  - pcrh 11y ago
    Why would it not apply in this case? The sample isn't particularly large, and we don't know the extent of possible selection biases. For example one obvious bias is that the sample is restricted to data sent through Docsend, thus likely does not represent a uniform sampling of all pitches. In any case, there is no prior reason to exclude outliers; any large enough representative sampling of a Gaussian distribution would include "outliers".
    - minimaxir 11y ago
      That's what I meant; curse my ambigious "it."
- erikb 11y ago
  The thing is that the three dots that lay "outside the range were the other dots are" (not the "true" range which we of course don't know) have a higher impact on the conclusion than the amount of them justifies. That's why taking them out should not harm but help. This is not the same as taking datasets out of a medical study because they got sick by the new medicine. E.g., if you look at the left diagram. The upward swing in the red curve is probably the result of all the left points (199 points). But the downward trend is as far as I can see the result of 1 point on the right. So can we trust that downward trend as much as the upward trend? Probably not. Additionally removing the two dots at the 150 meetings point might result in the curve not having such a strong incline, but one that is backed by 196 points. I really don't have more than basic education in statistics, but to me it looks like a slight but constant upward trend until 100 investors contacted and after that we simply don't have enough data. If you think about what the diagram talks about getting a constant upward line or even more likely a square root shaped line is very likely. Getting a bell curve is unlikely, just from thinking about the topic. What should happen that contacting more than X investors would result in less meetings? In the second diagram removing the two dots at $4.5m and the two dots above 200 investors met leaves us with 196 data points that show a fairly equal spread and that the amount of investors contacted might not influence that much how much money you get in the end. That's also good because it's expected to find that company evaluation depends on company value (customers, market, team, etc) and not on how many investors you contact. You make the data more conclusive by removing data points that have a big impact on your results without the backup of other data points. That's what handling outliers is.
  - pcrh 11y ago
    As to the investors contacted vs meetings plot, the red line is evidently bogus. It is likely caused by trying to fit the data to some polynomial for which there is no a priori justification. A straight-line fit would look more reasonable and also be less affected by one "outlier" who contacted 300 investors. I would agree that cutting-off curve fitting at 100-150 investors would look prettier; the thing to do in that case would be either to not attempt to fit a curve and just show the data, or alternately to fit only the earlier part (so the red line stops half-way across the x-axis), but show all the data. The only reason for doing a curve fit on this kind of data is to show that the data fit some prior model, showing that the model is likely correct, or to show where some statistical value lies.