4 ms·

Any statisticians want to chime in here? How many wallets should be dropped in each city before you can reasonably rank them? Or, is it statistically significan

Any statisticians want to chime in here? How many wallets should be dropped in each city before you can reasonably rank them? Or, is it statistically significant that Lisbon was 1/12, while Helsinki was 11/12? In other words, even with the small sample sizes, are those two values far enough apart to state the difference with 95% confidence? On another note, that site sprang 20 separate trackers/beacons/etc. in Ghostery. I know at least one was Disqus, but 7 were different beacons, and the slideshow itself didn't work correctly without disabling it.

olalonde 13y ago
I guess the rate would also vary widely depending on which part of the city or time of the day you drop the wallet.
- danielbarla 13y ago
  Also, quite possibly: day of month (contrast a week before payday vs just after), temporary local events and conditions (high unemployment one year vs the previous), and if the wallet contained a tourist's details: the level of 2nd language knowledge predominant in the country (e.g. the old lady in one of the first slides probably didn't speak English, meaning a barrier to contact or an opportunity for a second individual to pocket the wallet). Overall though, if you randomised the drops nicely, increased sample size and repeated them a few times, it might be an interesting study.
nmc 13y ago
I believe the main interest is not the statistical study itself, but rather the spontaneous explanations people give for returning a lost wallet.
- mcv 13y ago
  A lot of people said they returned it because of the family photo in it. That seems to confirm something I heard years ago: that wallets with photos of children were more likely to be returned. So if you don't have kids, put a photo of a niece or nephew, or even a complete stranger's kid in there.
andrewcooke 13y ago
assuming everything else were controlled for (and i have no idea how you do that) then you'd expect the "noise" in num-wallets-returned to be about sqrt(num-wallets-returned). in other words, in a city where the "real" rate is 4 returned, you expect to see that number varying by about 2 each time, so maybe 4, 2, 7, 3, 1... so the results here really don't mean much.
danielbarla 13y ago
Well, I'm not a statistician, but since nobody seems to be answering, I'll give it a shot. Choosing the statistical method is slightly tricky; we have binary data (which is a bit different to the methods most people are used to). Guidance from measuringusability.com [1] tells us that our scenario is very similar to their "example 2", essentially where we have two groups with simple totals (1/12 and 11/12). Their advice is to use a N-1 two proportion test [2] for this. Plugging in the values 1/12 and 11/12 here, we get a 99.994% probability that the distributions are different. This is because although the sample sizes are relatively small, 1/12 and 11/12 are very far apart. Slightly surprising, nevertheless. A more reasonable comparison would be 5/12 vs 8/12, and in this case the same test yields a 77.108% probability that they are different (i.e. not particularly significant). [1] http://www.measuringusability.com/blog/what-test.php http://www.measuringusability.com/blog/what-test.php [2] http://www.measuringusability.com/ab-calc.php http://www.measuringusability.com/ab-calc.php

olalonde 13y ago

I guess the rate would also vary widely depending on which part of the city or time of the day you drop the wallet.

danielbarla 13y ago
Also, quite possibly: day of month (contrast a week before payday vs just after), temporary local events and conditions (high unemployment one year vs the previous), and if the wallet contained a tourist's details: the level of 2nd language knowledge predominant in the country (e.g. the old lady in one of the first slides probably didn't speak English, meaning a barrier to contact or an opportunity for a second individual to pocket the wallet). Overall though, if you randomised the drops nicely, increased sample size and repeated them a few times, it might be an interesting study.

nmc 13y ago

I believe the main interest is not the statistical study itself, but rather the spontaneous explanations people give for returning a lost wallet.

mcv 13y ago
A lot of people said they returned it because of the family photo in it. That seems to confirm something I heard years ago: that wallets with photos of children were more likely to be returned. So if you don't have kids, put a photo of a niece or nephew, or even a complete stranger's kid in there.

andrewcooke 13y ago

assuming everything else were controlled for (and i have no idea how you do that) then you'd expect the "noise" in num-wallets-returned to be about sqrt(num-wallets-returned). in other words, in a city where the "real" rate is 4 returned, you expect to see that number varying by about 2 each time, so maybe 4, 2, 7, 3, 1... so the results here really don't mean much.

danielbarla 13y ago

Well, I'm not a statistician, but since nobody seems to be answering, I'll give it a shot. Choosing the statistical method is slightly tricky; we have binary data (which is a bit different to the methods most people are used to). Guidance from measuringusability.com [1] tells us that our scenario is very similar to their "example 2", essentially where we have two groups with simple totals (1/12 and 11/12). Their advice is to use a N-1 two proportion test [2] for this. Plugging in the values 1/12 and 11/12 here, we get a 99.994% probability that the distributions are different. This is because although the sample sizes are relatively small, 1/12 and 11/12 are very far apart. Slightly surprising, nevertheless. A more reasonable comparison would be 5/12 vs 8/12, and in this case the same test yields a 77.108% probability that they are different (i.e. not particularly significant). [1] http://www.measuringusability.com/blog/what-test.php http://www.measuringusability.com/blog/what-test.php [2] http://www.measuringusability.com/ab-calc.php http://www.measuringusability.com/ab-calc.php