3 ms·

> Second, you would be surprised at the amount of information even small sample sizes can provide when there is great uncertainty about the true data distributi

by aab0 11y ago

> Second, you would be surprised at the amount of information even small sample sizes can provide when there is great uncertainty about the true data distribution. This isn't about sampling error, this is about inverting conditionals while ignoring base rates. This is like saying 'one third of surveyed Somalis in America report living in Minnesota, therefore, one-third of Minnesota is Somalian'. That conclusion doesn't follow, whether you surveyed 10 Somalis or 1000. That is, there is no reason to expect P(Vancouver purchase|Chinese) = P(Chinese|Vancouver purchase) = 0.33. If that number happens to be right, it is by coincidence; it only works if you are lucky enough that the relative sizes of Chinese buyers and Vancouver sales are just right. (Specifically, that the size of the Chinese buyer population is identical to the size of Vancouver sales; if there are a million Chinese buyers worldwide and a million Vancouver sales over that same period, and a third of Chinese buyers say they bought in Vancouver, then yes, a third of Vancouver sales will be to Chinese.) If you knew two of the three variables of global Chinese size/Vancouver sales size/P(Vancouver purchase|Chinese), even if you just had a vague but defensible prior, you could (weakly) estimate the third. But in isolation, each variable means nothing.

tgb 11y ago
> This isn't about sampling error, this is about inverting conditionals while ignoring base rates. This is like saying 'one third of surveyed Somalis in America report living in Minnesota, therefore, one-third of Minnesota is Somalian'. That conclusion doesn't follow, whether you surveyed 10 Somalis or 1000. That's not even remotely what was done in the article by my reading. More like: 33% of Somalis * total number of Somalis / total people in Minnesota.
- aab0 11y ago
  "That's not even remotely what was done in the article by my reading." That's what it was described as in the grandparent: "This was a study based on "a Financial Times multiple choice survey of 77 high net worth and affluent mainland Chinese individuals, 'admittedly not a statistically significant sample size,'” According to this survey, 9 individuals out of these 77 Chinese bought property in Vancouver." 'mainland Chinese' are a reference class much larger than 'Chinese buyers of Vancouver properties', as they are buying globally. If this description is wrong, then one should correct it - not introduce irrelevancies like how 'well with Bayesian reasoning this sorta small sample is OK!' "More like: 33% of Somalis * total number of Somalis / total people in Minnesota." Which is still wrong, since there are lots of Somalis not in Minnesota.
  - tgb 11y ago
    > 'mainland Chinese' are a reference class much larger than 'Chinese buyers of Vancouver properties', as they are buying globally. And 9/77 of mainland Chinese are buying properties in Vancouver, so you multiply that by total number of people in mainland China (that are high net worth and affluent) and then multiply that by the average property value bought by them. That gets (a rough approximation of) the property value bought by Chinese mainlanders in Vancouver, now divide that by the total property market in Vancouver to get the percent of Vancouver being bought by Chinese mainlanders. I don't know where you're getting anything else than this. The top level parent only described this as "some more mathematical shenanigans" yet you assume that therefore it has to be this nonesense of swapping P(A|B) with P(B|A). The article makes it clear that that's not the case, so I don't know who you're arguing with. > Which is still wrong, since there are lots of Somalis not in Minnesota. Yes, 67% of them in this hypothetical, as accounted for in the first factor. This process isn't inherently wrong, the debate is whether the small sample size makes the uncertainty too large to be useful.
    - aab0 11y ago
      > And 9/77 of mainland Chinese are buying properties in Vancouver, so you multiply that by total number of people in mainland China (that are high net worth and affluent) and then multiply that by the average property value bought by them. That gets (a rough approximation of) the property value bought by Chinese mainlanders in Vancouver, now divide that by the total property market in Vancouver to get the percent of Vancouver being bought by Chinese mainlanders. And now you've done the correction for the base rates which I discussed in my original: once you know the relevant populations and fractions, then yes, you can get out the right fraction. But it's highly unlikely the correct version will just happen to equal the original fraction in the survey, and no correction was included in the summary. Let me quote again what I am criticizing: "This was a study based on "a Financial Times multiple choice survey of 77 high net worth and affluent mainland Chinese individuals, 'admittedly not a statistically significant sample size,'” According to this survey, 9 individuals out of these 77 Chinese bought property in Vancouver." Note there is nothing about 'total number of people in mainland China'. Nothing about 'average property value'. Nothing about 'total property market in Vancouver'. I don't know where you're getting all this. The procedure as described is just plain wrong, nothing to do with sample sizes or Bayesian methods. > This process isn't inherently wrong, the debate is whether the small sample size makes the uncertainty too large to be useful. That's what the comment thought they were defending, but they were wrong. They misunderstood the problem entirely. It was interpreting the survey meaning totally incorrectly by inverting the conditional, the problem was not precision/sampling error.

by aab0 11y ago

tgb 11y ago

> This isn't about sampling error, this is about inverting conditionals while ignoring base rates. This is like saying 'one third of surveyed Somalis in America report living in Minnesota, therefore, one-third of Minnesota is Somalian'. That conclusion doesn't follow, whether you surveyed 10 Somalis or 1000. That's not even remotely what was done in the article by my reading. More like: 33% of Somalis * total number of Somalis / total people in Minnesota.

aab0 11y ago
"That's not even remotely what was done in the article by my reading." That's what it was described as in the grandparent: "This was a study based on "a Financial Times multiple choice survey of 77 high net worth and affluent mainland Chinese individuals, 'admittedly not a statistically significant sample size,'” According to this survey, 9 individuals out of these 77 Chinese bought property in Vancouver." 'mainland Chinese' are a reference class much larger than 'Chinese buyers of Vancouver properties', as they are buying globally. If this description is wrong, then one should correct it - not introduce irrelevancies like how 'well with Bayesian reasoning this sorta small sample is OK!' "More like: 33% of Somalis * total number of Somalis / total people in Minnesota." Which is still wrong, since there are lots of Somalis not in Minnesota.
- tgb 11y ago
  > 'mainland Chinese' are a reference class much larger than 'Chinese buyers of Vancouver properties', as they are buying globally. And 9/77 of mainland Chinese are buying properties in Vancouver, so you multiply that by total number of people in mainland China (that are high net worth and affluent) and then multiply that by the average property value bought by them. That gets (a rough approximation of) the property value bought by Chinese mainlanders in Vancouver, now divide that by the total property market in Vancouver to get the percent of Vancouver being bought by Chinese mainlanders. I don't know where you're getting anything else than this. The top level parent only described this as "some more mathematical shenanigans" yet you assume that therefore it has to be this nonesense of swapping P(A|B) with P(B|A). The article makes it clear that that's not the case, so I don't know who you're arguing with. > Which is still wrong, since there are lots of Somalis not in Minnesota. Yes, 67% of them in this hypothetical, as accounted for in the first factor. This process isn't inherently wrong, the debate is whether the small sample size makes the uncertainty too large to be useful.
  - aab0 11y ago
    > And 9/77 of mainland Chinese are buying properties in Vancouver, so you multiply that by total number of people in mainland China (that are high net worth and affluent) and then multiply that by the average property value bought by them. That gets (a rough approximation of) the property value bought by Chinese mainlanders in Vancouver, now divide that by the total property market in Vancouver to get the percent of Vancouver being bought by Chinese mainlanders. And now you've done the correction for the base rates which I discussed in my original: once you know the relevant populations and fractions, then yes, you can get out the right fraction. But it's highly unlikely the correct version will just happen to equal the original fraction in the survey, and no correction was included in the summary. Let me quote again what I am criticizing: "This was a study based on "a Financial Times multiple choice survey of 77 high net worth and affluent mainland Chinese individuals, 'admittedly not a statistically significant sample size,'” According to this survey, 9 individuals out of these 77 Chinese bought property in Vancouver." Note there is nothing about 'total number of people in mainland China'. Nothing about 'average property value'. Nothing about 'total property market in Vancouver'. I don't know where you're getting all this. The procedure as described is just plain wrong, nothing to do with sample sizes or Bayesian methods. > This process isn't inherently wrong, the debate is whether the small sample size makes the uncertainty too large to be useful. That's what the comment thought they were defending, but they were wrong. They misunderstood the problem entirely. It was interpreting the survey meaning totally incorrectly by inverting the conditional, the problem was not precision/sampling error.