4 ms·

I am not a gifted statistician, so I just made a monte carlo simulation to test. This is not too scientific because in reality different jobs have different num

I am not a gifted statistician, so I just made a monte carlo simulation to test. This is not too scientific because in reality different jobs have different numbers of people, and a thousand other variables, but anyway, here is the code and results: from random import random, randint p_white = .81 n_occupations = 100 n_workforce = 150000000 occupations = [[] for i in xrange(n_occupations)] for i in xrange(n_workforce): occupation = randint(0,n_occupations-1) race = "white" if random() < p_white else "other" occupations[occupation].append(race) percentages = map(lambda x:x.count("white")/float(len(x)), occupations) print sorted(percentages) [0.8093594044124566, 0.8093716670190719, 0.8093837105400035, 0.8093941116532061, 0.8094080377696309, 0.809459598174709, 0.8094606757892058, 0.8094949877814243, 0.8095051091912103, 0.8095619154480623, 0.8095855138023271, 0.8096009184415862, 0.8096130265928078, 0.8096471477224493, 0.8096569834673563, 0.8096622916149537, 0.8096635533608949, 0.8096712436008832, 0.8096812597976153, 0.8096856369286581, 0.8096963532163073, 0.8097067434989362, 0.809721568598737, 0.8097215751436514, 0.8097328871910747, 0.8097572581628716, 0.809766392402407, 0.8097706403995997, 0.8097853662894858, 0.8097907743218179, 0.8098200848588418, 0.8098433273181519, 0.8098759161986114, 0.8098761662131717, 0.8098826202953597, 0.8098902227694705, 0.8098932712388633, 0.8099027774814941, 0.8099132209630405, 0.8099239110036907, 0.8099288512271164, 0.809935064025657, 0.8099377895998511, 0.8099442967886306, 0.8099576910748759, 0.8099684818490649, 0.8099723061883299, 0.8099772955347885, 0.8099790775887737, 0.8099951683577414, 0.8099964813792994, 0.8100114768250255, 0.8100264485053597, 0.8100326219727435, 0.8100426685467571, 0.8100538851385188, 0.8100624027564277, 0.8100772456940655, 0.8100799987206602, 0.8100878460787697, 0.8101051239951382, 0.8101238836646114, 0.8101259241335592, 0.8101299639808958, 0.8101333990942324, 0.8101444590699302, 0.810176363997584, 0.8101767343321185, 0.8101835910494692, 0.810185024612142, 0.8101881998783053, 0.8102189975744003, 0.8102348016041733, 0.810247294219311, 0.8102501899417512, 0.8102612850637286, 0.8102728907675896, 0.8102876025226937, 0.8102915035431886, 0.8103243498099062, 0.8103428626649917, 0.8103475670573029, 0.8103618541773706, 0.8103623158464366, 0.8103870909678774, 0.8104014543756358, 0.8104238667513951, 0.8104306535452401, 0.8104537503309478, 0.81046437557304, 0.8104696732444567, 0.8104846773483934, 0.8104918990629446, 0.810527481205214, 0.8105324346430826, 0.8105520454920258, 0.8106940187094273, 0.8107141451872244, 0.8107156884282282, 0.8108683817976586] As we can see by the results, the min and max are around a tenth of a percent off from the expected value. Even if the job distribution is skewed (certain occupations have a larger workforce), if N is sufficiently large, numbers such as +15% should be completely unrealistic. Please correct me if I have made a grave error that I did not mention (I know that this is very very hand wavy, unscientific).

jacques_chester 13y ago
No, that fits with the remarks made by others. I wasn't sure whether the null hypothesis is approached by looking at the likelihood of sample variation from the general population of individuals, or whether by calculating how many job titles in the population of job titles would have such outlying numbers. The consensus seems to be that most of these are not by chance. Roll on the debate.
yummyfajitas 13y ago
I think you are making a grave error - near as I can tell, each of your simulated occupations has about 1.5 million people in it. Of the few professions listed that I checked, the biggest (farmer) has only 758k people in it: Veterinarian 61k, farmer 758k, mining machine operators 21k, Speech language pathologist 123k. The outcome doesn't change much (you go from 81.1% to 81.4% max), I'm just being a stats geek. http://www.bls.gov/ooh/Healthcare/Veterinarians.htm http://www.bls.gov/ooh/Healthcare/Veterinarians.htm http://www.bls.gov/ooh/farming-fishing-and-forestry/agricultural-workers.htm http://www.bls.gov/ooh/farming-fishing-and-forestry/agricult... http://www.bls.gov/ooh/About/Data-for-Occupations-Not-Covered-in-Detail.htm http://www.bls.gov/ooh/About/Data-for-Occupations-Not-Covere... http://www.bls.gov/ooh/Healthcare/Speech-language-pathologists.htm http://www.bls.gov/ooh/Healthcare/Speech-language-pathologis... Geography probably plays a big role in some of these. I expect that most veterinarians and farmers are located in farming and ranching states for example, and Iowa/Idaho are not exactly known for racial diversity.
- coldarchon 13y ago
  I think the next error is hypergeometric distribution, because parents and real-life social networks do affect the success rate in achieving a job for skilled workers. Of course farmers give their farm to their offspring etc.
- gamegoblin 13y ago
  Yes, I was aware of this, but in 5 minutes of googling I couldn't find a good answer to the total number of possible professions, so I went with an even 100. As I said, very hand-wavy and not too scientific. But yes, even for a workforce of 150,000 the results don't get close to breaching into the 90's.

jacques_chester 13y ago

No, that fits with the remarks made by others. I wasn't sure whether the null hypothesis is approached by looking at the likelihood of sample variation from the general population of individuals, or whether by calculating how many job titles in the population of job titles would have such outlying numbers. The consensus seems to be that most of these are not by chance. Roll on the debate.

yummyfajitas 13y ago

I think you are making a grave error - near as I can tell, each of your simulated occupations has about 1.5 million people in it. Of the few professions listed that I checked, the biggest (farmer) has only 758k people in it: Veterinarian 61k, farmer 758k, mining machine operators 21k, Speech language pathologist 123k. The outcome doesn't change much (you go from 81.1% to 81.4% max), I'm just being a stats geek. http://www.bls.gov/ooh/Healthcare/Veterinarians.htm http://www.bls.gov/ooh/Healthcare/Veterinarians.htm http://www.bls.gov/ooh/farming-fishing-and-forestry/agricultural-workers.htm http://www.bls.gov/ooh/farming-fishing-and-forestry/agricult... http://www.bls.gov/ooh/About/Data-for-Occupations-Not-Covered-in-Detail.htm http://www.bls.gov/ooh/About/Data-for-Occupations-Not-Covere... http://www.bls.gov/ooh/Healthcare/Speech-language-pathologists.htm http://www.bls.gov/ooh/Healthcare/Speech-language-pathologis... Geography probably plays a big role in some of these. I expect that most veterinarians and farmers are located in farming and ranching states for example, and Iowa/Idaho are not exactly known for racial diversity.

coldarchon 13y ago
I think the next error is hypergeometric distribution, because parents and real-life social networks do affect the success rate in achieving a job for skilled workers. Of course farmers give their farm to their offspring etc.
gamegoblin 13y ago
Yes, I was aware of this, but in 5 minutes of googling I couldn't find a good answer to the total number of possible professions, so I went with an even 100. As I said, very hand-wavy and not too scientific. But yes, even for a workforce of 150,000 the results don't get close to breaching into the 90's.