3 ms·

I don't think 100 is terribly reasonable. For a company with the size and resources of Twitter, it should be trivial to check several thousand. I'm not sure wh

by HWR_14 5y ago

I don't think 100 is terribly reasonable. For a company with the size and resources of Twitter, it should be trivial to check several thousand. I'm not sure why the standard deviation worse case is 5%. Presumably the accuracy of the bot decision plays into it. But I would imagine a couple thousand would get a far more accurate answer.

RubberSoul 5y ago
The goal is to estimate the proportion of accounts that are bots. Let that number equal p. The variance of the proportion is p(1-p). The highest that can be is (0.5)0.5. Then, the standard deviation of that is the square root, which is 0.5. Now, we want to know the standard error for our estimate of the bot proportion. That is sqrt(p(1-p)/n). Suppose 50% of accounts are bots (I assume that would be very high), then our estimate of p would be 0.5 and our standard error with a sample of 100 would be 0.05. Hence, our 95% confidence interval is roughly 0.4–0.6 in the worst case (with a sample of 100). If the proportion is under 0.1 (let's assume 0.05), then the standard error would be sqrt(0.05(1-0.05)/100) = 0.022. Our 95% confidence interval in this case would be roughly 0.01–0.09. These seem like large ranges to me. Hence, I would expect them to use a larger sample too.
azakai 5y ago
The worse case is 5% because "is a bot" is a boolean variable. That has a standard deviation of 0.5 in the very worst case (50% on value 0, 50% on value 1, so the mean is 0.5, and the deviation can't be more than 0.5). And the standard deviation of a random sample scales like 1 over the square root of the sample size, so 0.5 divided by 10 => 0.05 (5%). For presidential elections it's common to see samples sizes like 1,000 or so, which have standard deviations of around 1.5%. That's better than 5%, and it makes sense since elections are often won by just a few %. But here, IIUC, the goal was to see if bots are a big part of Twitter or not, and so the answer "there are fewer than 5% bots" is enough. That is, we don't care if the % of bots is 3.5% or 4.7%.
- ineedasername 5y ago
  The standard deviation of 50 one's and 50 zero's is 0.50 What is your 5% value referencing? https://www.calculator.net/standard-deviation-calculator.html?numberinputs=1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0&ctype=s&x=68&y=14 https://www.calculator.net/standard-deviation-calculator.htm...
  - native_samples 5y ago
    Didn't azakai just say that? > That has a standard deviation of 0.5 in the very worst case followed by > the standard deviation of a random sample scales like 1 over the square root of the sample size, so 0.5 divided by 10 => 0.05 (5%).

by HWR_14 5y ago

RubberSoul 5y ago

The goal is to estimate the proportion of accounts that are bots. Let that number equal p. The variance of the proportion is p(1-p). The highest that can be is (0.5)0.5. Then, the standard deviation of that is the square root, which is 0.5. Now, we want to know the standard error for our estimate of the bot proportion. That is sqrt(p(1-p)/n). Suppose 50% of accounts are bots (I assume that would be very high), then our estimate of p would be 0.5 and our standard error with a sample of 100 would be 0.05. Hence, our 95% confidence interval is roughly 0.4–0.6 in the worst case (with a sample of 100). If the proportion is under 0.1 (let's assume 0.05), then the standard error would be sqrt(0.05(1-0.05)/100) = 0.022. Our 95% confidence interval in this case would be roughly 0.01–0.09. These seem like large ranges to me. Hence, I would expect them to use a larger sample too.

azakai 5y ago

The worse case is 5% because "is a bot" is a boolean variable. That has a standard deviation of 0.5 in the very worst case (50% on value 0, 50% on value 1, so the mean is 0.5, and the deviation can't be more than 0.5). And the standard deviation of a random sample scales like 1 over the square root of the sample size, so 0.5 divided by 10 => 0.05 (5%). For presidential elections it's common to see samples sizes like 1,000 or so, which have standard deviations of around 1.5%. That's better than 5%, and it makes sense since elections are often won by just a few %. But here, IIUC, the goal was to see if bots are a big part of Twitter or not, and so the answer "there are fewer than 5% bots" is enough. That is, we don't care if the % of bots is 3.5% or 4.7%.

ineedasername 5y ago
The standard deviation of 50 one's and 50 zero's is 0.50 What is your 5% value referencing? https://www.calculator.net/standard-deviation-calculator.html?numberinputs=1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0%2C1%2C0&ctype=s&x=68&y=14 https://www.calculator.net/standard-deviation-calculator.htm...
- native_samples 5y ago
  Didn't azakai just say that? > That has a standard deviation of 0.5 in the very worst case followed by > the standard deviation of a random sample scales like 1 over the square root of the sample size, so 0.5 divided by 10 => 0.05 (5%).