7 ms·
Before we jump to theories, has anyone eliminated the null hypothesis? That is: assuming a totally random distribution of individuals into professions, purely
by jacques_chester 13y ago
Before we jump to theories, has anyone eliminated the null hypothesis?
That is: assuming a totally random distribution of individuals into professions, purely by chance, some professions will have curious ratios. The first thing a statistician aims to do is to test the probability that the observed effect is due to chance.
A second thing to test is whether the ratios are stable across time.
This comment represents 20% of your RDA for nitpicky HN comments.
- daemin 13y agoYeah I was thinking something like that, because as the article said 81% of all workers are white - simplified it means 8/10 are white - and these 33 jobs (out of how many?) have 9/10 ratio - does that really mean discrimination or is this story just a filler hit piece against perceived injustices that don't actually exist?
- llamataboot 13y ago#1) This article was not a "hit piece", more a "isn't this curious?" piece #2) There's already enough statistics out there that show injustice on a much larger scale - pretty much pick any statistic - life expectancy, wealth, employment, incarceration, etc and you'll see rank racial disparities -- in many cases racial disparities that haven't closed over the past 50 years.
- daemin 13y agoWithout knowing how many jobs there are total it doesn't sound fully like a "isn't this curious" piece. At best I'd put it with the "50 weird things about the 80's" type of piece. At worst it could detrimentally affect those professions in the name of progress.
- capnrefsmmat 13y agoI think I'll try testing this statistically, even just as a cute example for Statistics Done Wrong. Thanks for the idea. (If I get interesting results I'll post back here in a few days) edit: I can't replicate the results in their charts using this BLS table: http://www.bls.gov/cps/cpsaat11.htm http://www.bls.gov/cps/cpsaat11.htm For example, for veterinarians I get 90.7% white, not 96.5%. Am I missing something?
- TylerE 13y agoThe statistics done wrong bit is where they don't control for socioeconomic status.
- sanskritabelt 13y agoEven ignoring those two chumps on the radio, the socioeconomic status of auto body repair (90.7% white) is not very high.
- TylerE 13y agoI don't think that's really true. That's generally something that requires at least tech school/community college. It's not manual labor.
- icambron 13y agoFor the individual occupations, presumably they all have enough participants to avoid a high chance of any one of them being that far off (for example, there are 100,000 veterinarians [1]) [Edit: and see sanskritabelt's comment below that seems to match my intuition.] Then the question is how many different occupations are being considered here, and it doesn't seem like it would be high enough to make numbers like 96% very likely. So if we were measuring the whole population, it seems--just eyeballing-- pretty unlikely. But of course we're not measuring the whole population. The data from which this chart was actually derived might well be caused by random chance. The article's failure to cite a more specific source than the BLS generally is a bit annoying here. Edit: I guess the source is [2], but I'm still unclear on what was measured. [1] https://www.avma.org/KB/Resources/Statistics/Pages/Market-research-statistics-US-veterinarians.aspx https://www.avma.org/KB/Resources/Statistics/Pages/Market-re... [2] http://www.bls.gov/cps/cpsaat11.htm http://www.bls.gov/cps/cpsaat11.htm
- QuasarLogic 13y agopg. 25 in this pdf (http://www.bls.gov/cps/cpsrs.pdf http://www.bls.gov/cps/cpsrs.pdf). Found the source here: http://www.bls.gov/cps/cps_over.htm#methodology http://www.bls.gov/cps/cps_over.htm#methodology
- sanskritabelt 13y agoA profession with 10K workers should randomly expect to be about 80-82% white. From here: http://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval http://en.wikipedia.org/wiki/Binomial_proportion_confidence_... p=0.81 n=10000 and z = 2 gives a 99% confidence interval
- sanskritabelt 13y agoAnd 90% white is firmly in the "basically impossible by random chance" category.
- daemin 13y agoI guess that depends on a uniform distribution of the location of the jobs to the racial characteristics in said location. For instance I would imagine there would be more vets in an area with more animals, perhaps a country or town border region. At the same time those areas might have a higher concentration of white people than the average (with other races preferring a more urban environment).
- sanskritabelt 13y agoThey need EMTs just about everywhere (92% white).
- lifeisstillgood 13y agobut they need EMTs who live nearby, just about everywhere - and it does not take much for neighbourhoods and states to clear into distinct patterns by house-purchase preferences (game theoretic economist whose name cannot remember won a prize for this) basically all that is needed is a prefernce for a particular neighbourhood attribute (walk to shops, less black people, more black people, close to hiking routes) and for that preference to be available at that persons clearing price. so, a dribbling knuckle dragging KKK member would pay a lot to avoid living in Harlem (or rather would sacrifice opportunity by staying in Fuckwatsr Alabama.) Take a nice black professional couple - happy to live in well integrated area of Manhattan - till along comes baby. at this point a slightly larger house nearer good schools is desirable - and preferably closer to grandma. Alternatively there is a nice area of Queens (?) that is predominately white but has good schools. Now all it takes is a preference or a feeling of being more comfortable near grandma and near other young mothers "just like me". Since grandma will likely be living in an area where there was traditionally a larger percentage of black families the comfortable preference is near grandma. and who would argue with an expectant mother wanting to be comfortable. At a certain point the cost of being near grandma will exceed the amount they can afford without realising they are making an explicit racist choice. which is I think my preferred definition - not emotional preference but when a conscious decision is made with race as a factor. All this shakes out to mean that levels of preference few if any of us would consider racist, lead to segregated neighbourhoods. luckily other preferences intrude masking the effect. So to help with the discussion, this implies that if everyone was perfectly color blind we would expect to see a 81% ratio. if not what is the level, the error bar, that we accept as emotional preference that does not cross the line ? And how do we tell that difference if the starting population is not 81% in that area ? and if the school district in the area is really bad / good? (nb nationally funded schools systems I would expect see nationally less segregated societies than the USA for this reason) anyway my 2 cents
- gamegoblin 13y agoI am not a gifted statistician, so I just made a monte carlo simulation to test. This is not too scientific because in reality different jobs have different numbers of people, and a thousand other variables, but anyway, here is the code and results: from random import random, randint p_white = .81 n_occupations = 100 n_workforce = 150000000 occupations = [[] for i in xrange(n_occupations)] for i in xrange(n_workforce): occupation = randint(0,n_occupations-1) race = "white" if random() < p_white else "other" occupations[occupation].append(race) percentages = map(lambda x:x.count("white")/float(len(x)), occupations) print sorted(percentages) [0.8093594044124566, 0.8093716670190719, 0.8093837105400035, 0.8093941116532061, 0.8094080377696309, 0.809459598174709, 0.8094606757892058, 0.8094949877814243, 0.8095051091912103, 0.8095619154480623, 0.8095855138023271, 0.8096009184415862, 0.8096130265928078, 0.8096471477224493, 0.8096569834673563, 0.8096622916149537, 0.8096635533608949, 0.8096712436008832, 0.8096812597976153, 0.8096856369286581, 0.8096963532163073, 0.8097067434989362, 0.809721568598737, 0.8097215751436514, 0.8097328871910747, 0.8097572581628716, 0.809766392402407, 0.8097706403995997, 0.8097853662894858, 0.8097907743218179, 0.8098200848588418, 0.8098433273181519, 0.8098759161986114, 0.8098761662131717, 0.8098826202953597, 0.8098902227694705, 0.8098932712388633, 0.8099027774814941, 0.8099132209630405, 0.8099239110036907, 0.8099288512271164, 0.809935064025657, 0.8099377895998511, 0.8099442967886306, 0.8099576910748759, 0.8099684818490649, 0.8099723061883299, 0.8099772955347885, 0.8099790775887737, 0.8099951683577414, 0.8099964813792994, 0.8100114768250255, 0.8100264485053597, 0.8100326219727435, 0.8100426685467571, 0.8100538851385188, 0.8100624027564277, 0.8100772456940655, 0.8100799987206602, 0.8100878460787697, 0.8101051239951382, 0.8101238836646114, 0.8101259241335592, 0.8101299639808958, 0.8101333990942324, 0.8101444590699302, 0.810176363997584, 0.8101767343321185, 0.8101835910494692, 0.810185024612142, 0.8101881998783053, 0.8102189975744003, 0.8102348016041733, 0.810247294219311, 0.8102501899417512, 0.8102612850637286, 0.8102728907675896, 0.8102876025226937, 0.8102915035431886, 0.8103243498099062, 0.8103428626649917, 0.8103475670573029, 0.8103618541773706, 0.8103623158464366, 0.8103870909678774, 0.8104014543756358, 0.8104238667513951, 0.8104306535452401, 0.8104537503309478, 0.81046437557304, 0.8104696732444567, 0.8104846773483934, 0.8104918990629446, 0.810527481205214, 0.8105324346430826, 0.8105520454920258, 0.8106940187094273, 0.8107141451872244, 0.8107156884282282, 0.8108683817976586] As we can see by the results, the min and max are around a tenth of a percent off from the expected value. Even if the job distribution is skewed (certain occupations have a larger workforce), if N is sufficiently large, numbers such as +15% should be completely unrealistic. Please correct me if I have made a grave error that I did not mention (I know that this is very very hand wavy, unscientific).