5 ms·

> Americans drive an aggregate of 3 trillion miles, while Tesla drivers have done 100 million. That's well over an order of magnitude difference. The data on 1

by squidi 13y ago

> Americans drive an aggregate of 3 trillion miles, while Tesla drivers have done 100 million. That's well over an order of magnitude difference. The data on 100 million is probably enough to compare with the 3 trillion miles. The populations don't need to be equal to compare them, just big enough that they are random and distributed enough.

nonchalance 13y ago
To put it in perspective, the disparity is equivalent to polling 10K americans and extrapolating to all of america (which, for better or for worse, is what most pollsters do).
- ProCynic 13y ago
  If the sample is truly random, it doesn't matter what the population size is. Of course, truly random samples are hard to get.
  - adw 13y ago
    That depends on the population variance.
  - deleted 13y ago
    [deleted]
- akavi 13y ago
  Except pollsters put in a lot of effort in making sure that the sample is representative of the larger population. Which is not true here, as the GP correctly notes.
  - mikeash 13y ago
    Do they really? My only interaction with pollsters has been either having them call me or solicit me on the street, and both approaches have a tremendous inherent bias. As far as I know, this is how the big national agencies do things.
    - PeterisP 13y ago
      There are all kinds of adjustments done afterwards to correct for various factors. For example, you know the age distributions in USA; and if you find out your phone calls are getting twice as many seniors than the proportion should be[1], then you throw away a random parts of them so that they don't skew final 'data' towards the typical opinions of seniors. [1] Assuming that you're not measuring average age or measuring 'who is at home', but if you want to see, say, the average political opinion of total USA population, which tends to correlate with age.
      - mikeash 13y ago
        It still seems impossible to correct for everything. Sure, you could correct for age as you describe, but I imagine that landline phone ownership correlates with political opinions in all sorts of other ways too. Furthermore, how do you gather the data needed to correct the polling numbers without being able to accurately poll people in the first place? Seems like a complete chicken-and-egg problem.
        PeterisP 13y ago
        "how do you gather the data needed to correct the polling numbers" -> you use the census. You need some info about the total population, you get it periodically and it doesn't change that much; you don't need to repeat it for every survey.
        mikeash 13y ago
        The census doesn't tell you about most of the godzillion factors that link landline phone ownership with political opinions.
- Zikes 13y ago
  I often see polls of 1K Americans and extrapolating out from there. An example: http://www.webpronews.com/americans-think-cloud-computing-comes-from-actual-clouds-2012-08 http://www.webpronews.com/americans-think-cloud-computing-co... Of course, the refrain I often hear is that as long as you pick the RIGHT 1,000 Americans, it's as good as polling all 319 million.
  - gnaritas 13y ago
    > Of course, the refrain I often hear is that as long as you pick the RIGHT 1,000 Americans, it's as good as polling all 319 million. If it's a proper random sample, then it's far better than sampling all 319 million because it's 95% accurate with about a 3% margin of error and vastly cheaper and actually practical; you can't poll 319 million people.
- mikeash 13y ago
  It's counterintuitive, but the sample size needed for a good measurement doesn't much depend on the size of the overall population. What matters is getting a properly random sample. This is where pollsters fall down, because their "random" sample tends to be heavily biased toward the sort of person who has a landline telephone and doesn't hang up on pollsters. Polling 10,000 Americans would be vast overkill, in any case.
- deleted 13y ago
  [deleted]
- gnaritas 13y ago
  You say that as if 10k is too small a sample size to extrapolate accurately with, but actually that's a huge sample size that if done properly would be extremely accurate. You don't need to poll anywhere near 10k people to accurately predict all Americans views. You can sample less than 2000 people and get 99% accuracy with a 3% margin of error for a population of 325 million. Increasing the sample size to 10k simply reduces the margin of error to 1.29%, hardly worth the extra sampling of 8k people.
- auctiontheory 13y ago
  If you haven't read Asimov's Election Day, I highly recommend it.
broostoryco 13y ago
We are comparing tail events with low probabilities.
gkoberger 13y ago
When trying to get popular opinion for, say, an election -- sure. However, there was one fire over 100 million miles. The problem isn't the 100 million, it's the 1 fire. This wasn't a controlled experiment, either -- they just stopped the clock as soon as the first fire happened, and multiplied. A week ago, they could have said "You have exactly a 0% chance of your Tesla catching on fire" and have been right by this logic. To think of it another way -- let's say you get lucky and get a hole-in-one your 10th time golfing. Does that mean you'll have 10 hole-in-ones if you golf 100 times? Doubtful. EDIT: Also, don't forget that Elon is mixing numbers. There's no fire if someone doesn't run over something. All these numbers show is that the average driver is more likely to run over something. Of course Tesla drivers run over fewer things -- there are no 16 year old kids texting while driving a Tesla... yet.
- revelation 13y ago
  Doesn't stopping the clock after 1 fire just penalize them, not benefit them? That said, I also think the 1 fire is the problem here. Just think about how that relation changes with 2 fires.
  - jamesaguilar 13y ago
    It probably penalizes them, assuming the no-fires-for-100m-miles was not a fluke. There's no way to know for sure without knowing the true distribution of fires per mile.
  - BWStearns 13y ago
    The chance goes from 0.000000010 to 0.000000020 If I had the cash I would still purchase a Tesla after the second fire as well.
    - sgustard 13y ago
      Thanks for converting that to "fires per mile".
    - moocowduckquack 13y ago
thedufer 13y ago
The relevant metric here seems to be 'miles between fire events'. On this metric, we have exactly 1 data point for Tesla. I would hardly call that "probably enough". Of course, I'm exaggerating in the other direction. What we really should be calculating is the odds that Teslas burst into flames less often than the average car, given that the average car does so every 20 million miles and the first such event in a Tesla was at the 100 million-mile mark. We're still failing to account for the fact that the average Tesla is newer and probably better-kept than the average car, but it would at least be a reasonable start. I don't know enough statistics to perform this calculation, but I would like to see how it is done.
- jurjenh 13y ago
  An easy starting point would be to get this kind of data on similarly aged cars - cars sold over the last 2-3 years, then see what stats you can get there. I'd guess that you'd find some models that have never caught fire and some that have done so a lot more than tesla's, but that's pure speculation on my part.
- PeterisP 13y ago
  We have enough events, because in this case it is appropriate to model the actual random event as 'million miles driven' with a chance of fire happening or not happening. Gasoline cars have a mean of 0.05 fires per million miles, and given the current Tesla data, the mean is 0.01 fires per million miles. I'm not taking out a calculator, but it would come out to an extremely low (0.0001%) the 'true' fire chance is the 0.05 gas car rate or higher; the 95% confidence interval should be 0.01 +- 0.02 or tighter, so still twice better than gas cars. For an exaggerated example, if Tesla had driven a billion miles and had 0 fires, you shouldn't say that there's not enough data - you definitely would have enough data to say that the chance of fire is below the gas-car rate of 5 fires per 100 million miles.
  - thedufer 13y ago
    Why is million miles driven a better way to model it than billion miles driven, for example? If you happen to choose that, there clearly isn't anywhere near enough data. I'm honestly curious how one models this type of thing statistically, and I am not convinced enough of its obviousness to just accept numbers that someone throws around.

by squidi 13y ago

nonchalance 13y ago

To put it in perspective, the disparity is equivalent to polling 10K americans and extrapolating to all of america (which, for better or for worse, is what most pollsters do).

ProCynic 13y ago
If the sample is truly random, it doesn't matter what the population size is. Of course, truly random samples are hard to get.
- adw 13y ago
  That depends on the population variance.
- deleted 13y ago
  [deleted]
akavi 13y ago
Except pollsters put in a lot of effort in making sure that the sample is representative of the larger population. Which is not true here, as the GP correctly notes.
- mikeash 13y ago
  Do they really? My only interaction with pollsters has been either having them call me or solicit me on the street, and both approaches have a tremendous inherent bias. As far as I know, this is how the big national agencies do things.
  - PeterisP 13y ago
    There are all kinds of adjustments done afterwards to correct for various factors. For example, you know the age distributions in USA; and if you find out your phone calls are getting twice as many seniors than the proportion should be[1], then you throw away a random parts of them so that they don't skew final 'data' towards the typical opinions of seniors. [1] Assuming that you're not measuring average age or measuring 'who is at home', but if you want to see, say, the average political opinion of total USA population, which tends to correlate with age.
    - mikeash 13y ago
      It still seems impossible to correct for everything. Sure, you could correct for age as you describe, but I imagine that landline phone ownership correlates with political opinions in all sorts of other ways too. Furthermore, how do you gather the data needed to correct the polling numbers without being able to accurately poll people in the first place? Seems like a complete chicken-and-egg problem.
      - PeterisP 13y ago
        "how do you gather the data needed to correct the polling numbers" -> you use the census. You need some info about the total population, you get it periodically and it doesn't change that much; you don't need to repeat it for every survey.
        mikeash 13y ago
        The census doesn't tell you about most of the godzillion factors that link landline phone ownership with political opinions.
Zikes 13y ago
I often see polls of 1K Americans and extrapolating out from there. An example: http://www.webpronews.com/americans-think-cloud-computing-comes-from-actual-clouds-2012-08 http://www.webpronews.com/americans-think-cloud-computing-co... Of course, the refrain I often hear is that as long as you pick the RIGHT 1,000 Americans, it's as good as polling all 319 million.
- gnaritas 13y ago
  > Of course, the refrain I often hear is that as long as you pick the RIGHT 1,000 Americans, it's as good as polling all 319 million. If it's a proper random sample, then it's far better than sampling all 319 million because it's 95% accurate with about a 3% margin of error and vastly cheaper and actually practical; you can't poll 319 million people.
mikeash 13y ago
It's counterintuitive, but the sample size needed for a good measurement doesn't much depend on the size of the overall population. What matters is getting a properly random sample. This is where pollsters fall down, because their "random" sample tends to be heavily biased toward the sort of person who has a landline telephone and doesn't hang up on pollsters. Polling 10,000 Americans would be vast overkill, in any case.
deleted 13y ago
[deleted]
gnaritas 13y ago
You say that as if 10k is too small a sample size to extrapolate accurately with, but actually that's a huge sample size that if done properly would be extremely accurate. You don't need to poll anywhere near 10k people to accurately predict all Americans views. You can sample less than 2000 people and get 99% accuracy with a 3% margin of error for a population of 325 million. Increasing the sample size to 10k simply reduces the margin of error to 1.29%, hardly worth the extra sampling of 8k people.
auctiontheory 13y ago
If you haven't read Asimov's Election Day, I highly recommend it.

broostoryco 13y ago

We are comparing tail events with low probabilities.

gkoberger 13y ago

When trying to get popular opinion for, say, an election -- sure. However, there was one fire over 100 million miles. The problem isn't the 100 million, it's the 1 fire. This wasn't a controlled experiment, either -- they just stopped the clock as soon as the first fire happened, and multiplied. A week ago, they could have said "You have exactly a 0% chance of your Tesla catching on fire" and have been right by this logic. To think of it another way -- let's say you get lucky and get a hole-in-one your 10th time golfing. Does that mean you'll have 10 hole-in-ones if you golf 100 times? Doubtful. EDIT: Also, don't forget that Elon is mixing numbers. There's no fire if someone doesn't run over something. All these numbers show is that the average driver is more likely to run over something. Of course Tesla drivers run over fewer things -- there are no 16 year old kids texting while driving a Tesla... yet.

revelation 13y ago
Doesn't stopping the clock after 1 fire just penalize them, not benefit them? That said, I also think the 1 fire is the problem here. Just think about how that relation changes with 2 fires.
- jamesaguilar 13y ago
  It probably penalizes them, assuming the no-fires-for-100m-miles was not a fluke. There's no way to know for sure without knowing the true distribution of fires per mile.
- BWStearns 13y ago
  The chance goes from 0.000000010 to 0.000000020 If I had the cash I would still purchase a Tesla after the second fire as well.
  - sgustard 13y ago
    Thanks for converting that to "fires per mile".
  - moocowduckquack 13y ago

thedufer 13y ago

The relevant metric here seems to be 'miles between fire events'. On this metric, we have exactly 1 data point for Tesla. I would hardly call that "probably enough". Of course, I'm exaggerating in the other direction. What we really should be calculating is the odds that Teslas burst into flames less often than the average car, given that the average car does so every 20 million miles and the first such event in a Tesla was at the 100 million-mile mark. We're still failing to account for the fact that the average Tesla is newer and probably better-kept than the average car, but it would at least be a reasonable start. I don't know enough statistics to perform this calculation, but I would like to see how it is done.

jurjenh 13y ago
An easy starting point would be to get this kind of data on similarly aged cars - cars sold over the last 2-3 years, then see what stats you can get there. I'd guess that you'd find some models that have never caught fire and some that have done so a lot more than tesla's, but that's pure speculation on my part.
PeterisP 13y ago
We have enough events, because in this case it is appropriate to model the actual random event as 'million miles driven' with a chance of fire happening or not happening. Gasoline cars have a mean of 0.05 fires per million miles, and given the current Tesla data, the mean is 0.01 fires per million miles. I'm not taking out a calculator, but it would come out to an extremely low (0.0001%) the 'true' fire chance is the 0.05 gas car rate or higher; the 95% confidence interval should be 0.01 +- 0.02 or tighter, so still twice better than gas cars. For an exaggerated example, if Tesla had driven a billion miles and had 0 fires, you shouldn't say that there's not enough data - you definitely would have enough data to say that the chance of fire is below the gas-car rate of 5 fires per 100 million miles.
- thedufer 13y ago
  Why is million miles driven a better way to model it than billion miles driven, for example? If you happen to choose that, there clearly isn't anywhere near enough data. I'm honestly curious how one models this type of thing statistically, and I am not convinced enough of its obviousness to just accept numbers that someone throws around.