8 ms·
Cory Doctorow: Our dangerous statistical ignorance
- michael_dorfman 18y agoI agree with all of his points, but it was a very slim article-- I was kind of hoping for a more profound analysis from someone like Doctorow.
- spydez 18y agoIf you want a profound analysis, he wrote a (fiction) book recently on this subject, and put it out on the internet under a CC license. It's an enjoyable read; he pretty much distilled a few bits from the book to make that article. Here tis: http://craphound.com/littlebrother/Cory_Doctorow_-_Little_Brother.htm http://craphound.com/littlebrother/Cory_Doctorow_-_Little_Br...
- billroberts 18y agoI think it was about as in-depth as you are likely to get published in a (UK) national newspaper. One small but well-written step in the fight against the innumerate sensationalists that dominate the press and no doubt contribute to the populist over-reactions from the UK government.
- tonystubblebine 18y agoThere's a quote I like that I think came from Marvin Minsky, "the US needs a Department of Homeland Arithmetic." We're protecting ourselves from all the wrong risks.
- aggieben 18y agoUh, wouldn't this actually be called something like "the US Department of Education"? It's been a rousing success, too - it's allowed our Ministry of Truth to claim things like "comrades! Our production of students that pass the standardized test is up 85% this quarter!". It's all about families, man. Good families teach people what's really important - not federal departments of anonymoustaxeatingbureaucrat.
- sabat 18y agoAnd when a large proportion of families (broken, dysfunctional, otherwise unable) fail to teach kids what's important: what then?
- graywh 18y agoPeople didn't worry about that (i.e. formal education systems) until only recently, but I guess what's "important" has gotten more complex and the rate of "dysfunction" has increased.
- aggieben 18y agoI don't know many good answers to that question - but in my bones I know that the federal government isn't one of them.
- helveticaman 18y agoWhat now?
- byrneseyeview 18y agoI don't understand why Marvin Minsky hates America so much that he wants to put the government in charge of assessing such risks. The government is an incredibly complex device for distilling a tiny amount of individual ignorance from each voter into a moonshine of batshittery.
- SwellJoe 18y agoI'm not exactly sure what a "moonshine of batshittery" is, but you've got my vote.
- fnord23 18y agoI'm guess it's from the phrase 'batshit crazy'. But yeah, he's got my vote too...
- rw 18y agoIt's presumptuous to say that voters effect any real change in government policy.
- eru 18y agoPerhaps you have more effect by voting (i.e. switching) TV channels. The public opinion seems to be very important for government policy - and popular shows can be and are seen as a barometer for it, I guess.
- michaelneale 18y agoA book I want to read on that exact subject: http://miami.indymedia.org/news/2008/05/11008.php http://miami.indymedia.org/news/2008/05/11008.php
- gojomo 18y agoWe could fund the Department of Homeland Arithmetic with lottery revenues!
- mynameishere 18y agoPeople who are quite aware of the mathematics behind gambling still do it. Gambling exploits instincts rather than ignorance.
- dhimes 18y agoIf I had an intuitive grasp of statistics, I may not have a start-up!
- SwellJoe 18y agoWhy not? It's a one-in-four (or one-in-three, depending on who you believe) chance of being rich in a few years--that means that in the worst case scenario, if you keep at it, starting over when you fail, in about 10-15 years you're very nearly guaranteed to be rich. And you almost certainly won't starve in the process. Of course, that assumes a reasonable level of intelligence, education, and drive. Startups are about the best game going, as far as I can tell--I wouldn't be playing if the game was rigged against me (more than a little, anyway...sure, small companies have higher relative regulatory burden, but on the whole the technology game is actually rigged in favor of new companies, from a growth perspective).
- epi0Bauqu 18y agoWhere do you get this 1/4 or 1/3 from?
- SwellJoe 18y agoThe Internet! Where do you get your dubious information from? Seriously, though, there have been a few studies of various degrees of reliability that indicate that new technology business failure rate over five years is quite a bit smaller than the old "9 in 10 startups fail" wives tale would have us believe. I wouldn't put significant weight on any particular piece of data, but it seems to be pretty consistently in that range whenever people who I would trust to know the numbers (VCs, angels, journalists covering the field, successful and famously unsuccessful entrepreneurs) talk about it. And, among my own peers here in the valley who started during WFP07, about 1/7 of them are already rich (by some definition of rich). There were 21 groups, and I believe 3 have had exits, and I'm certain that Octopart, Weebly, Buxfer, Heysan, and Virtualmin have not come fully to fruition yet. Tsumobi might even surprise folks, as they're still slaving away in their secret underground lab in the Balkans (Josh may have actually said, "Boston", it was hard to hear at the Startup School reception due to the size of the crowd). YC certainly makes a notable improvement in the outcomes of their startups, but it's not magical, so I don't think it's a crazy idea to look at YC startups as at least somewhat representative of tech startups in general--where "tech startup" means, to me, folks who actually file the paperwork, build something, and get it into the hands of users...until you've done that, you're just another dork with a big idea (and those probably fail at a much higher rate than 9 in 10). Anyway, we're only a year and a half into the experiment with the WFP07 group, and I expect the numbers will probably end up in the 1/3 to 1/2 range.
- jpeterson 18y ago"Here's how that works: imagine that you've got a disease that strikes one in a million people, and a test for the disease that's 99% accurate. You administer the test to a million people, and it will be positive for around 10,000 of them – because for every hundred people, it will be wrong once (that's what 99% accurate means). Yet, statistically, we know that there's only one infected person in the entire sample. That means that your "99% accurate" test is wrong 9,999 times out of 10,000!" No, it means that the "99% accurate" test is wrong 9,999 times out of 1,000,000. It would be clear to anyone when stated that way. What's counterintuitive is the author's statement of the result, not the result itself.
- graywh 18y agohttp://en.wikipedia.org/wiki/Bayes'_theorem#Example_.232:_Drug_testing http://en.wikipedia.org/wiki/Bayes'_theorem#Example_.232:_Dr... Basically, an extremely low rate of incidence (in Cory's example, 1 in a million) will render a "highly accurate" test meaningless.
- tel 18y agoA better wording is that your chance of having the disease if given a positive result from the test is 1/10000 (0.1%). That's a huge increase from a 0.0001% chance of having the disease, but it's still not flat out terrifying. Repeat testing can weed through the false positives at a speed proportional to its accuracy.
- graywh 18y agoNot necessarily. If you get the same results every time, repeated testing provides nothing.
- DougBTX 18y agoIt depends on what causes the false positives. If the test is picking up something in the person being tested, then yes, you'll get the same result every time and repeated testing proves nothing. But you can still repeat using other tests. If the test gives false positives purely at random, then repeated testing will help. Say the test is wrong 50% of the time, and you do the test five times. If you get the same results every time, then you can be 100-(50/100)^5*100 = 97% sure of the results.
- andrewparker 18y agoRiding on the subway earlier today, a guy got on an starting preaching how "the lord saves" and "you must embrace Jesus." I wish math could create equivalent evangelism. I'd love to live in a world where a guy gets on the subway and starts preaching the Pythagorean theorem?
- graywh 18y agoThat one's not the problem. Perhaps Bayes' theorem?
- henning 18y agoa more basic problem is just plain ignorance. many if not most citizens of the USA do not understand the basic organization and functions of their government. most of them cannot name a single sitting supreme court justice, have never read the constitution, do not know how a bill becomes a law. and these people vote.
- redorb 18y agogood case against data mining... and the "if it saved one life, would it be worth it?" argument doesn't change my mind.
- ken 18y agoStatistician Peter Donnelly on the same subject: http://youtube.com/watch?v=kLmzxmRcUTo http://youtube.com/watch?v=kLmzxmRcUTo
- niels_olson 18y agoAs someone who deals in diagnostic tests all day, nobody's diagnosed on one test. There's a similar problem I worked out on the odds of getting into medical school, I'm sure this is elementary stats and has someone's name on it, but I'll call it the acceptance problem: how many medical schools do you have to apply to in order to have 90% chance of getting into medical school? For any school there is a acceptance quotient: Q = (acceptances sent out)/(number of applications received) For any given student applying to some schools 1 thru n, the goal is getting at least once acceptance, and applying to more schools, mathematically, can't possibly hurt in the closed case (neglecting social engineering, time spent on applications, etc), so the chance of acceptance, Ca, approaches 1 with every new application in the following fashion: Ca = (1 - (1 - Q1)(1 - Q2)(1 - Q3)...(1 - Qn))100 There's a visual and some worked out examples here: http://nielsolson.us/MedSchool/ http://nielsolson.us/MedSchool/ Similarly, if the sensitivity of a test is 90%, that means the test identifies 9 of every 10 people with the diagnosis. If I administer n different tests each with a sensitivity of S, then the chances of accurately diagnosing the disease, Cd, goes up* with each additional positive but never gets to 1. Cd = (1 - (1 - S1)(1 - S2)(1 - S3) . . . (1 - Sn))100% So lets say you are doing, say, genetic testing, and any one gene is 1% sensitive for the disease. If you tested 300 genes you could be no more than 95% certain of the diagnosis. (1 - (1 - .99)^300)100 = 95.09591...% Now, if your genetic tests were 5% accurate, you're panel could be no more than 95% accurate with 59 tests. If your test was 50% accurate, you're panel could consist of 6 tests and be no more than 95% accurate. Of course, if some of the tests are negative, things get more complicated. One of the problems with these data sets is that we have no idea how predictive they are. You can't even calculate the predictive power of the database. There simply haven't been enough events. Then we get into surrogate measures (how many were positive on tests 1 - n and were found to have razor blades in their homes, etc). The claim that these databases can't be effective isn't true. They could be. P might also equal NP. Whether the hypothesis is strictly true or not, the vague but real set of 'practical concerns' suggest that the truth of the hypothesis is sufficiently difficult to test as to render the null hypothesis the de facto assumption until proven otherwise.
- aston 18y agoThe assumption of independence underlying the math you're doing is probably outright false (albeit mathematically simple). As one example, imagine the case where all n schools use the exact same admissions criteria. Unless you're applying with a random application to each school, your math is shot; you will either get into every school or none of them. I won't even go into the genetic independence issue.
- breck 18y ago"But the fact is that attacks by strangers are so rare as to be practically nonexistent. If your child is assaulted, the perpetrator is almost certainly a relative." Maybe that statement is true in today's world. But for tens of thousands of years, while our brains evolved, I would guess attacks from strangers were a lot more common.
- tel 18y agoThe causation implied here is unlikely at best. More likely might be that until the last century no one had to deal with numbers large enough to need these kinds of statistics.
- deleted 18y ago[deleted]
- seshadripv 18y agoIt's not just the probability of an occurence that leads us to behave in an unreasonable manner. The other important thing is the impact of that 'rare occurence'. Most critics don't seem to take that in to account at all...
- __ 18y agoSomething doesn't seem right about Doctorow's example of attacks on children. He writes, "But the fact is that attacks by strangers are so rare as to be practically nonexistent. If your child is assaulted, the perpetrator is almost certainly a relative (most likely a parent)." I'm sure that's true, but it doesn't answer the real question. For most parents, the question is: given my child's particular environment, what is the greatest threat to him? Doctorow is saying: given that your child was attacked (and no additional information), the attacker is more likely than not to be a relative. That seems backwards.
- tel 18y agoHe's stating that the evidence is P(Was Relative | Child Attacked) is relatively high, especially compared to P(Was Total Stranger | Child Attacked). This means that it should, if you use Bayes' Theorem correctly in everyday life, shift your suspicion away from the random photographer. Of course, since P(Child Attacked) is so very low it's still not a huge deal. It's not really backwards. That sort of inversion is exactly part of Bayes' Theorem.
- Retric 18y agoIf you are related to X people then the odds that one of them attacks your child is greater than that anyone else on the planet. Picking numbers from thin air: Let's say are related to 30 people know 200 and you encounter 10,000 random strangers and you have a 1 in 100 chance of an attack. Well the odds a specific random stranger attacks your child given the above assumptions is less than 1/2,000,000. The odds a specific person they know attacks your child would be less than 1/20,000 and the odds that a given relative attacks your child would be more than 1/6,000. Now who would you focus on? (What about a predator that has a 50% chance of attacking someone well he is still under 1/6000 because there are so many other people for them to attack.) PS: Given my child's particular environment you should focus on relatives and people you know.
- JacobAldridge 18y agoI believe the worst damage to statistics is specious reasoning. My favourite chocolate promotion is Mars' '1 in 6 Wins a Free Bar' (on currently here in Oz). If I buy 6 bars, most people would assume I would win once. In fact, I have only a 2/3 chance of winning a free one 1 - (5^6/6^6) Buy 12 bars, and there's still a more than 10% chance I won't have won yet...most of the chocolate-buying government-voting lottery-praying public would be stunned.
- graywh 18y agoAnd as N approaches infinity, the probability of winning after N tries with a p=1/N approaches 1 - 1/e. (Proof left to the reader as an exercise.)
- aston 18y agoAlthough the average number of bars you'd need to buy before you win is indeed six. I think almost no one would put the odds of actually winning one by buying six at 100%.
- JacobAldridge 18y ago"Any 6 bars may not contain a free bar" You're probably right, and I hope that you are. Still, funny to note that Mars has even put this disclaimer on the bottom of their promo site! http://www.marsfreebars.com.au/ http://www.marsfreebars.com.au/