18 ms·

You Should Downvote Contrarian Anecdotes

219 points by tylerhobbs 14y ago

lhnz 14y ago
The simplification in the title is dangerous without a 'when' attached. Mindlessly downvoting contrarian anecdotes will lead to a hive-mind. Beware. Opinions overwhelm all other forms of material in a discussion. Anecdotes are actually one up from opinions as they are concrete. You should weight them like so: Statistical Evidence + Logic > Statistical Evidence > "Common Knowledge/Wisdom" > Anecdote + Logic > Anecdote > Opinion + Logic > Opinion
maigret 14y ago
Or maybe just not upvote? Good argumentation, still can get in the way of why HN is here. It's not just a place to make formal argumentation, but to exchange experiences, or get some opinions from others. In some case the opinion is wished from experts (how can I get funded etc), in others it is about how a product is liked by general users. So please take a piece of salt in applying that ;)
pseudonym 14y ago
In my experience, some of the best advice I've ever received has been from anecdotes that run counter to the text of an article. ...I kid. It's actually an interesting idea. I don't know about scrapping them entirely, but I think a lot of sites (say, Reddit) could benefit from moving the anecdotes elsewhere. Partially because of this, and partially because anecdotes in any form tend to derail the discussion pretty darn quick.
- necubi 14y ago
  This is what makes r/AskScience so good. "Free of anecdotes" is a rule for posts, and it is vigorously enforced.
drivebyacct2 14y ago
Is this not based in the assumption that the article to which the anecdotes are offered as replies is omniscient? Which may in and of itself be fallacious.
- VMG 14y ago
  The assumption is only that the article is based on a scientific study, not that it is omniscient.
  - r00fus 14y ago
    A study that rigidly follows the scientific principles, but is intellectually dishonest (ie, sponsored by interested party of studied subject, etc) is often not useful to the reader of the study results. Because you can follow the letter of the principles, and still flout the spirit. One should, when listening to a study, question the funding. Likewise, dissenting opinions must also be examined in what interested parties have a hand in their creation.
    - Zakharov 14y ago
      Yes, examining the biases and methodologies of a study is the correct way to present a contrarian opinion. Providing an anecdote is much less useful.
    - slurgfest 14y ago
      Sponsorship does not imply intellectual dishonesty. Intellectual dishonesty is a matter of how the argument is made. Sponsorship is a heuristic you could use to quickly filter papers but used as a counter-argument, it is simply ad hominem. You are describing a method for judging arguments without thinking critically about the arguments themselves or examining their basis in evidence and that is not any kind of science. One does not find any measure of objectivity by averaging between opinions, only by holding arguments to the yardstick of rationality and evidence.
- drivebyacct2 14y ago
  Thanks, you were great too.
deleted 14y ago
[deleted]
- deleted 14y ago
  [deleted]
rsanchez1 14y ago
The author didn't pick good examples of contrarian anecdotes found on HN. They are contrary to a study on coffee consumption and dementia. The study, however, is only a sample of 124 people from Tampa and Miami. The study is just as likely to have found an isolated effect as the posters with family members who consumed a lot of coffee and still developed alzheimers. You would be better served by this: https://sites.google.com/site/mccormickphilosophy/home/critical-thinking-syllabus/critical-thinking-schedule/understanding-statistical-arguments https://sites.google.com/site/mccormickphilosophy/home/criti... The author must be a frequent coffee drinker who didn't like that some people had a different experience with coffee than some other people had, and felt compelled to write that post. It's not the contrarian anecdotes that left me with the sense that the research findings weren't conclusive. It's the fact that a population of 124 people in two cities is not at all representative of the target population which numbers over 40 million according to the census bureau. http://quickfacts.census.gov/qfd/states/00000.html http://quickfacts.census.gov/qfd/states/00000.html Maybe when the study researchers conduct a larger study I'll believe them.
- tylerhobbs 14y ago
  Great comment. I am most certainly not claiming that you shouldn't question research findings on other grounds, such as sampling bias or statistical significance, as you have done here; those types of points are extremely valuable.
- scalable 14y ago
  N=124 can definitely give statistically significant results. Consider the following thought experiment. You have a coin that you think might be loaded. You flip it 124 times. You get 119 tails. It is probably loaded. Lets say you get 66 tails. Could definitely be due to chance. Moral: The larger effect you study, the less sample size you can get away with. It might be that people from Tampa and Miami are different in some important way from other people. However, if you want to level this criticism, maybe you should give a reason for suspecting this? "The study is just as likely to have found an isolated effect as the posters with family members who consumed a lot of coffee and still developed alzheimers." No, no, triple no.
verroq 14y ago
Rebuttals to articles (ones that are voted up anyway) in the comment section are almost always well reasoned refutations. This article is addressing a non-problem.
its_so_on 14y ago
Forget what you and your friends experience. If we manage to get a piece of research sponsored, you'd better believe it! Sorry, I'll go on the experiences of myself and people I trust over research/article spinning.
- its_so_on 14y ago
  if my friends and I have problems with your product, it doesn't matter how reliable your research spins it as, I won't believe you. Forget research: make something my friends and I, or people I hear about, don't have problems with. That's all I pay attention to, and it works a lot better than the alternative. (paying attention to whatever you "prove" at a statistically significant level - nevermind how many commissioned studies you don't publish, thereby completely invalidating that statistical significance - and ignoring anecdotes simply doesn't work, for me or anyone else.) This article is literally asking for the right to lie (under the guises of 'research') and asking us to mod down anyone who calls them out on it. It really takes some face to say "Ignore what you experience - and vote down the experiences of others - and trust our data instead." Next you'll sell me the most reliable cloud on the planet. All the responses on the article say they've had nothing but problems and downtime. But, I should just ignore these, right?
Udo 14y ago
No. Contrarian anecdotes are good. They may turn out to be without merit, but then again so may the article itself. Having a real discussion is a good thing. Also, I would like to propose a logarithmic scale for weighting such things. Say, if the article in question found something extraordinarily significant with 100 out of 100 samples resulting in A, then it's still not rational to weigh a contrarian viewpoint resulting in B with 1/101 - it should maybe be closer to 1/3 or something. Consensus culture and worship authority are not desirable in my opinion. Arguments should be weighed on their merits and it's appropriate to explore other viewpoints or explanations even if they turn out to be dead ends most of the time.
- loup-vaillant 14y ago
  Let's take an example, since they seem to work best: vaccines. Vaccines protect you from the risk of contracting particular diseases, some of which are crippling, lethal, or incurable. Plus, most are extremely effective: once you take your shots, you are effectively immune. That's good. There is a downside however. Sometimes, vaccines have side effects. Most side effects are quite benign, but if you're unlucky enough, they can be crippling, lethal, or incurable. That's bad. From a medical point of view, vaccines are a net good (let's leave aside logistic considerations, or the effort required to go to the doctor). When you look at the stats, you stand a much better chance at life and health if you take the shot. Even for relatively minor illnesses like the flu. Now, let's say someone post a heartbreaking comment about how her 9 year old daughter died of a vaccine shot, with all the gory details about the suffering, how she couldn't participate in her school's festival, the size of the coffin… I'm quite sure there are stories of the kind. Given the sheer amount of readers here, maybe one of you will more or less directly relate to that. My apologies to those who do. Nevertheless, what makes a good story doesn't necessarily make good evidence. When you know of reliable statistics, and you read a contrarian anecdote, you should shift your belief in the direction of the anecdote by a precise amount, which is almost always tiny. What your brain will actually do behind you back however, is shifting your belief by a significant amount, often crossing the "reasonable doubt" line. That's not rational, but that's what will happen. Nameless statistics feel abstract, remote. An anecdote on the other hand feels concrete, real, close. Worse, you can spend far more time reading about the salient anecdotes than learning about the end results of reliable, but boring, scientific studies. Another example: you don't win the lottery. Period. You don't know of any close family of friend that ever did. But maybe one of you readers do. Maybe that one could comment and say "Hey, but my cousin did win the lottery!". Would that prove me wrong? Not at all. It's just that when the sample size is huge enough, even the tiniest chance can actualize.
liquidcool 14y ago
I've noticed that many of my biases, typically for or against some name brand or technology, even if not anecdotal, are old. I now try to remember not only how I came to this opinion, but when. I guess that's one of the consequences of aging.
- philwelch 14y ago
  My dad once received an inappropriate form letter from Texaco in the 1970's and never went to a Texaco station since.
hasenj 14y ago
You should question everything. This includes studies published and labeled "scientific". Contrarian anecdotes are important (but they too should be questioned). Nothing should be above questioning; even prevailing wisdom.
- flyinRyan 14y ago
  >even prevailing wisdom. Especially prevailing wisdom, since it is least likely to be questioned normally.
dennisgorelik 14y ago
If best contrarian comments are weak - I assume that original article is probably correct. If I don't see any contrarian comments, then I suspect groupthink in discussion.
D_Alex 14y ago
I totally get where the author is coming from, but I am not reeeally sure if that is good advice. Anecdotal evidence has its place, eg: proof by counterexample, challenging false, but widely held beliefs, or alerting people to new circumstances. A: "Murrumbidgee River is fun and safe for children to swim in! No one came to any harm there for the last 100 years!" B: "Dunno... my dog was eaten by a crocodile there last Saturday..." B's dog is a sample of one, but maybe worth paying attention to.
- jessedhillon 14y ago
  In your example, A is not relating a statistical or experimental finding, but instead dispensing commonsense knowledge which happens to be objectively false. It's not what the OP is talking about.
  - JoeAltmaier 14y ago
    Commonsense, like ulcers are caused by diet and anxiety, or that salt raises blood pressure? I think a good point is made: statistical evidence is also misleading - it deliberately ignores (averages out) the extreme cases. The results are a distribution; statistics folds that into one number. Anecdotes fill out the distribution.
    - carbocation 14y ago
      Ulcers are not generally caused by diet and anxiety, but rather by bacterial infection. Anecdote is a much more compelling refutation of "common sense" than it is of empirical statistical evidence. Statistical evidence is not misleading; it's simply the case that if you are seeking outliers, as in your example, then looking at measures of central tendency won't contain what you're looking for. Anecdote has no role in "fill[ing] out the distribution."
- TeMPOraL 14y ago
  You don't challenge "false, but widely held beliefs" with anecdotes - those beliefs actually is exactly what anecdote becomes when it starts being repeated. Alerting people to new circumstances is a good use, but that's pretty much all. Proof by counterexample works well only in mathematical logic, which is not how the real world works[1]. In reality you enter the domain of probability theory, and there counterexamples work exactly as the article author says - as an evidence that needs to be properly weighted. And these are those weights that people vastly overestimate in case of anegdotes. [1] - by this I mean, mathematical logic is not how we model, comprehend and operate reality.
pgsandstrom 14y ago
Are people joking when they defend anecdotal stories with "once this anecdotal story proved to be true and the study was faulty"? The proposition that the article makes is not that anecdotal stories must be false, but that they might influence the reader more than they should. Thus they should not be encouraged.
- JoeAltmaier 14y ago
  Yeah, thanks for deciding how gullible I am, and please do sanitize what I get to read for my own good. That's how open dialog works, right?
  - pgsandstrom 14y ago
    What do you think the voting system on HN does? It promotes good content, thus also demotes bad content. The author is saying "Here is a reason that content X is bad. If you agree with me, downvote it".
    - JoeAltmaier 14y ago
      Doesn't make sense. If the content is 'bad', moderator could expunge it. Or writer could be counseled on giving better posts. Or any number of things. The voting system on HN sometimes demotes bad content. Far more often (discussed widely elsewhere) its used to assert agreement or disagreement. When used in this sense, its a sort of instant-poll. In the case of anecdotal argument, it forms an ad-hoc 'scientific' experiment. The HN population weighs in using their own experience. Those who's memory (or ok, memory-triggered emotion response) aligns with the author may upvote etc. To 'bad content' is uniformely suppressed instead, this social experiment is lost, and the community loses. The merit of instant-polls is debatable, but other such polls are supported here. In fact the test group on HN may exceed the original 'scientific paper' group by an order of magnitude. Its statistical significance may exceed a graduate student's narrow study.
      - carbocation 14y ago
rlpb 14y ago
A contrarian anecdote may help expose an article being false or misleading in itself. For example: a review written by the manufacturer itself, or astroturfing. First we must determine the validity of the article existing in itself and any motivations or biases in it. For this, contrarian anecdotes are useful. This article seems to miss this point. Once we can accept the article at face value, then we may hit the biases described.
- scalable 14y ago
  I would say it is far easier to astroturf a contrarian anecdote.
the_bear 14y ago
If a post is about a topic that can and has been thoroughly researched and a reasonable conclusion has been drawn, then sure, a contrarian anecdote should probably be disregarded. But honestly, how often does that apply? Most of the conversations on HackerNews aren't about things that have right and wrong answers. As a matter of fact, many of the most popular posts are nothing more than anecdotes themselves. So why exactly should an anecdote in the comments be downvoted?
retrogradeorbit 14y ago
In a world were 47 of 53 landmark cancer research papers were found to not be replicable... where one scientist admitted (after the meta-researchers tried 50 times to replicate the result and failed) that they had run the experiment 6 times and got the result once and reported that because it made a better "story"... where the act of suppressing refutations of other's bogus research is "culturally commonplace"... Something is rotten in the core of science. These days I give the research the same weight I give the anecdote... and that is no weight at all. Science has been supplanted by money and politics... At least anecdotes admit they're anecdotes! http://news.yahoo.com/cancer-science-many-discoveries-dont-hold-174216262.html http://news.yahoo.com/cancer-science-many-discoveries-dont-h...
- Peaker 14y ago
  Surely you could find some scientists or scientific institutions you can trust? Or maybe you can trust those experiments that do get replicated successfully?
  - JohnHaugeland 14y ago
    Given that the context is "downvote anything that Reddit or HN disagree with," it seems that the point is "don't follow flocks while voting; make your own decisions or don't vote."
- bad_user 14y ago
  The problem with much of modern medicine is that much of it is based on flawed and biased statistical studies. Whether this is done because medical personnel don't have training in statistics, or because such studies generate funding, I don't know, but something is definitely rotten. Let's take anything involving nutrition. Some challenges are: (1) people lie, (2) such studies can't be double-blind so placebo kicks in, (3) the statistical significance of short-term studies is zero, (4) you can't control all the variables, unless you lock those people in a cage and (5) most conclusions of such studies have the potential to confuse the cause and the effect. But not all of science is like that. Just medicine.
keithpeter 14y ago
"But in the presence of statistical evidence, don’t tolerate contrarian anecdotes, and don’t make them yourself, knowing the exaggerated impact they can have." When you start to design a statistical experiment, you have already made an important methodological choice. See http://en.wikipedia.org/wiki/Positivism http://en.wikipedia.org/wiki/Positivism http://en.wikipedia.org/wiki/Grounded_Theory http://en.wikipedia.org/wiki/Grounded_Theory I've noticed that in my own field which is education, there appears to be a fondness for sophisticated statistics, even though no manager ever allocated students to teachers on a double blind random basis. An excellent example is the way the UK Education ministry has decided that 'phonics' is the way to teach reading. http://www.bbc.co.uk/news/education-18462214 http://www.bbc.co.uk/news/education-18462214 I think that this general tendency might be an example the 'white coat syndrome' in action; belief that using formal statistical techniques might increase the meaningfulness of the results. I suppose that is a form of cargo cult. This is hacker news, a forum aimed at people with novel business proposals and new software to try out. Should you be trying to find 'the Truth' or should you be building some grounded theory that tells you what to do next, provisionally, now, today?
nathan_long 14y ago
I downvoted some contrarian anecdotes once, and it ruined the thread.
- hexagonal 14y ago
  I downvoted some contrarian anecdotes once, and my "downvote" button was disabled.
JohnHaugeland 14y ago
This is an extremely toxic position, which will lead to novices (who outnumber the experienced significantly) mathematically abusing voting systems to promote whatever this month's fad tool golden hammer is. The herd is rarely correct.
chris_wot 14y ago
Have their been stories with seemingly solid statistical data, where anecdotal evidence shows that the statistical evidence is wrong?
brudgers 14y ago
By this logic, HN'ers should downvote any anecdote describing successfully exiting a startup because statistics show that the vast majority of startups fail and these anecdotes describing success unduly influence the thinking of entrepreneurs.
- Symmetry 14y ago
  Only if its offered as a counterexample to the proposition that most startups fail.
  - brudgers 14y ago
    One might suggest that that such a proposition is an underlying premise of HN.
b1daly 14y ago
Well I think Stalin summed up the perils of totally ignoring anecdotes pretty well: "One death is a tragedy; one million is a statistic." This topic sort of came up the other day in the thread about the girl losing the iPad software she needs to talk (Silencing Maya). http://news.ycombinator.com/item?id=4103344 http://news.ycombinator.com/item?id=4103344 Some commenters thought the story should be ignored as a data point about the societal value of patents. I disagree in that case because I don't believe economics or social sciences have anywhere near the amount of rigor and theory to make a good claim on whether the patent system is a net benefit to society. Deciding such a thing is a very old, and classic problem in philosophy. The OP seems to implicitly relying on a variant of utilitarianism, which IMO is wholefully inadequate to rely on for moral decisions. Since science is so hard, there is a lot of bad science out there. What gets reported in the wider media has all sorts of weird selection bias, never mind what gets picked for publication in journals. Anecdotes are human stories and they are deeply connected to why we care. Statiscal tools can be used in ways that justify harm people in the name of greater good. I agree that scientific medicine provides tools for "real"'medicine that other methods don't. I just think we should remember to real peoples stories, especially if they are true. Anyone reading this probably gets that one can't exactly counter balance the effect of anecdotes within themselves. But it can still be countered some, I think it's worth it to hear stories (that are relaxant to the topic). Now what is the basis of my beliefs? Mostly intuition, not rationality. But I don't think it's possible to undermine my ideas with some kind of experimentally bases argument. There's just way to many variables!
- slurgfest 14y ago
  I don't understand the part about Stalin's advice. What was Stalin saying would happen if you ignored anecdotes? How do you figure he was talking about the value of anecdotes?
  - gwern 14y ago
    Funny thing, I always took this quote as Stalin cynically remarking on exactly OP's point: people overweight anecdotes by thousands to millions of times their actual worth as Bayesian evidence.
CookWithMe 14y ago
I'm all in when it comes to call bullshit on something. But because we (hackers) are used to boolean decisions, we are looking for counter-examples. But these are just non-arguments when the finding is "In 9 out of 10 cases, ...". The anecdote is the 1 out of 10 case. Let's try to question anything, and when it is suspicious - by all means, say so! But, I guess what the author is trying to say (beyond the whole Downvote discussion) is that we should try to reason and make a proper argument. Especially one that is not already addressed in the research.
- TeMPOraL 14y ago
  > But these are just non-arguments when the finding is "In 9 out of 10 cases, ...". The anecdote is the 1 out of 10 case. The whole point of the article is that people won't judge that anecdote as only 0.1 relevant, but much much more (I'd say close to 0.9 if they personally prefer what anecdote says to what research says). It's a general human flaw that's very visible in day-to-day interactions with people.
sehugg 14y ago
Anecdotes have their place, especially in complex domains. For example, I have a friend who uses medical marijuana successfully to control MS symptoms. That is a sample size of one, but it certainly colors my view of the debate. I have a much deeper view into the decision making process of a MS sufferer now than I would otherwise, one that goes beyond spasticity, pain, and cognitive scores.
kbronson 14y ago
I'm going to downvote this article because contradicts everything said here.
DanielBMarkham 14y ago
I had to vote this up because of the meta-nature of the article. If anybody makes a comment disagreeing with it while sharing an anecdote they only play into's the author's theme! It's like accusing somebody of being in denial: there's nothing they can say in return that doesn't somehow support the idea that they are in denial. Love it. Having said that, this is another really bad article in what seems like an endless series of bad articles on HN. Here are a few of the more obvious flaws: - There is no universal truth as the author seems to imply. Simply because some publication or source you may like has performed some sort of statistical study doesn't have a lot of meaning on it's own. Yes, 100% of the people who eat bananas are dead within 120 years of their consumption. No, that does not mean bananas are bad. The study or scientific reporting is simply the beginning of a much longer conversation society has over many decades that leads us to higher-fidelity models. - The purpose of a social site is to behave socially. While places like HN have (or used to have) a lot of different guidelines for the types of behaviors that are encouraged or not, being social means sharing stories, anecdotes. We are not robots. - The idea that people are unable to sort out personal anecdotes from other forms of information. The follow-up idea that since they are not able to do this, we should prevent ourselves from sharing such stories. This is bad, bad, bad, bad, bad, bad .... bad(n). We are humans. We share stories. Anybody who says "people are so stupid" can justify just about damn near anything as long as they keep emphasizing the stupidity and danger some people's actions represent. Every now and then, gasp(!), published research is either wrong or doesn't show anything near what the reporter claims. Anecdotes don't help with this, of course, but they serve to remind us that even well-known scientists working at the highest quality standards available are still just sharing with us a very specialized form on anecdote. We did these things in this way and this is what we observed. Here is how you also can observe this. The really "good" part of the story they are sharing is talking about hidden assumptions, population variance, reproducibility, and so forth. Anecdotes don't do this, but they help us brainstorm ways in which we can improve the discussion, take the next experiment to an even better place. I'm very uncomfortable with the line of reasoning that goes somewhat like this: people are broken in some way, therefore we must somehow control what they read, say, or think for their own good. To me the beauty of western civilization is that really broken people can do these amazing and awesome things. The fact that we're deeply flawed is the magic. Science and human advancement work because of our flaws, not in spite of them. This is a very important thing to understand! Setting up some ideal of perfection, no matter how well-intended, and then mucking around with the way societal interaction works in some effort to improve on things is heading down a very dark path that has a very unhappy ending. This attitude seems rife in the technology community, however, perhaps because we are such analytical people. I don't want my fellow man to be irrational and distrustful of science and knowledge. But I'll take that any day over silencing contrarian articles and dissent. We've done the math on this: wrong people who share emotional stories and persuade crowds about all sorts of illogical things are a price that a dynamic community pays for progress.
hexagonal 14y ago
For example, if a close friend goes on and on about how the Ford he bought was a piece of crap, detailing how the transmission failed at 30k miles and the rear-view mirror fell off, you’ll be wary about buying a Ford in the future, even if Consumer Reports rates them highly. Personal anecdote: I've got a '93 Ford Explorer. It has 237,000 miles on it, and has yet to have a single major component failure. I've been waiting for the goddamn thing to die for the last 70,000 miles, so I can buy a car that does better than 13 miles to the gallon, but no dice. Don't buy Ford trucks. They're too reliable.
Spooky23 14y ago
I guess it depends on the story. If HN is supposed to be about startup culture and technology, the anecdote is essential to being vibrant. What is a startup if not a story whose premise is unproven? Now if you're telling me that your sister stuck a magnet in her ear and cured her cancer, I'm not going to give that credence without some real data. Most things on HN are not that, and a good story often compels us to think. So FWIW, I'm not downvoting contrarians, and I'm not downvoting anecdotes for being anecdotes.
motters 14y ago
Without contrary viewpoints there is always a danger of falling into overconfidence and group-think.
alan_cx 14y ago
Please forgive me if I have fundamentally misunderstood this, but is this article suggesting that anecdotes are some how comparable to scientific research data? If one wants to "attack" an anecdote, then a contrarian anecdote is the weapon. If one wants to attack scientific data, you need contrarian scientific data. At least, I hope that is right. There for, to mix the two is like attacking a tank with a wooden stick. Surely, anecdotes are used as the premise of scientific research. Lots of people tell stories. There seems to be something interesting going on. Then you do the research and produce the data. If the data is conclusive, then your past the anecdote. If later contrarian anecdotes appear, and they seem significant, off you go to scientific research again. I know Im wrong somewhere. But where?
Goladus 14y ago
Contrarian anecdotes can be very useful (and logically sound) when they are contrary to a generalization. If someone asserts "f(x) = 5 for all x in A", "f(a) = 3 and a in A" is a logical contradiction proving the assertion false. And, frankly, if we're talking about public discussion of scientific papers, inappropriate generalization is as big a problem as contrarian anecdotes, if not bigger. Scientific papers often cover highly specific observations that are useful primarily to other researchers, and often even then not for many years. People then try to apply that specific knowledge to practical day to day situations.
- Symmetry 14y ago
  Have you ever seen that sort of generalization in an article on Hacker News? Even when you have situations like, e.g. nobody has ever recorded a case of a person with a certain genotype being infected by a norovirus, you don't see papers claiming that its impossible.
  - Goladus 14y ago
    Yes. Realize that often the generalizations are a lot more subtle than your example, and of course anecdotes aren't always the best or only counter-arguments you'll see, even if they are valid. Start looking and I am sure you will find examples.
vannevar 14y ago
You shouldn't downvote to censor relevant information, period. You should only downvote if the material violates site guidelines. Downvoting to disagree is lazy, substituting easy censorship for thoughtful response. Instead, post your counter-argument or upvote an existing opinion that already expresses your position.
- Symmetry 14y ago
  Censoring relevant information is, of course, never good but the authors contention was that the information isn't actually relevant. Its always good that people who are wrong get a cogent reply, but beyond that adding a "me too" post doesn't have any more benefit than a downvote does. I used to just upvote post I agreed with, but since those are no longer public the only way to clue readers in to what the consensus is is to downvote.
  - vannevar 14y ago
    I'm not sure I'd agree he was arguing for the irrelevance of anecdotal evidence, only that it has a disproportionate impact when considered with statistical evidence. Rather than censoring the anecodotes, the better solution is to simply point out the danger of overweighting them. The purpose of a site like HN is not to arrive at some kind of imaginary consensus, it's to inform and engage people in meaningful discussion. There is no winning side to be on, and no ultimate arbiter of truth. By downvoting as you do, you rob the site of content in exchange for an illusory sense of victory.
grandalf 14y ago
The idea that there was an impending credit crisis was a "contrarian anecdote" in 2008. The idea that the US Government conducted illegal eavesdropping/wiretapping operations against US Citizens was a "contrarian anecdote" in 2003. This blog post condescendingly claims that HN readers are not sophisticated enough to balance out the sources that are input to their rational decision making process. In reality, contrarian opinions sometimes turn out to be correct, and mainstream opinions sometimes turn out to be wrong. Often, the benefit of a contrarian opinion is that it causes people to ask more questions, which is rarely a bad thing. People are sheeplike enough without having to be encouraged to follow the herd! I suggest everyone find a few contrarian theories and imagine what it would be like if you rearranged your life as if you expected them to be totally true. Most people are unwilling to go that far, and yes, in that way contrarian stories can weaken rational processes.
- ubernostrum 14y ago
  The idea that there was an impending credit crisis was a "contrarian anecdote" in 2008. As the saying goes: they laughed at Galileo. They laughed at Einstein. And they laughed at Bozo the Clown.
- Symmetry 14y ago
  Contrarian opinion is often correct, but contrarian anecdote in the face of statistical evidence seldom is. The article was about the later, not the former.
dclowd9901 14y ago
Contrarian anecdotes are an ill-conceived method of contributing to a conversation, and doing so in some sort of attention-getting way. They appear to all but undermine the entire premise of the original content, and yet, are based on absolutely nothing but one's individual experience with the matter. I see this kind of stuff all the time on Reddit, and it's become so numbing that you know, no matter how well you construct an argument or how many facts you cite, someone will always come out of the wood works and tell you how you're wrong because it didn't happen that way for them. It wouldn't be so bad if people didn't take that as a genuine counter-argument.
TimPC 14y ago
As a caveat, you should only do this in response to data based articles. Articles that are themselves anecdotal, should receive anecdotes that are contrarian. I think the more general idea is comments should be held to similar standards of evidence that the article presents.
TimPC 14y ago
As a caveat, you should only do this in response to data based articles. Articles that are themselves anecdotal, should receive anecdotes that are contrarian. I think the more general idea is comments should be held to similar standards of evidence that the article presents.
tokenadult 14y ago
This is an interesting first submission by a Hacker News participant who joined the community 551 days ago. The core idea in the submitted blog post (by the submitter here) is "Contrarian anecdotes like these are particularly common http://news.ycombinator.com/item?id=4076643 http://news.ycombinator.com/item?id=4076643 http://news.ycombinator.com/item?id=4076066 http://news.ycombinator.com/item?id=4076066 in medical discussions, even in fairly rational communities like HN. I find this particularly insidious (though the commenters mean no harm), because it can ultimately sway readers from taking advantage of statistically backed evidence for or against medical cures. Most topics aren’t as serious as medicine, but the type of harm done is the same, only on a lesser scale." The basic problem, as the interesting comments here illustrate, is that human thinking has biases that ratchet discussions in certain directions even if disagreement and debate is vigorous. The general issue of human cognitive biases was well discussed in Keith R. Stanovich's book What Intelligence Tests Miss: The Psychology of Rational Thought. http://yalepress.yale.edu/yupbooks/book.asp?isbn=9780300164626 http://yalepress.yale.edu/yupbooks/book.asp?isbn=97803001646... http://www.amazon.com/What-Intelligence-Tests-Miss-Psychology/dp/0300164629/ http://www.amazon.com/What-Intelligence-Tests-Miss-Psycholog... The author is an experienced cognitive science researcher and author of a previous book How to Think Straight about Psychology. He writes about aspects of human cognition that are not tapped by IQ tests. He is part of the mainstream of psychology in feeling comfortable with calling what is estimated by IQ tests "intelligence," but he disagrees that there are no other important aspects of human cognition. Rather, Stanovich says, there are many aspects of human cognition that can be summed up as "rationality" that explain why high-IQ people (he would say "intelligent people") do stupid things. Stanovich names a new concept, "dysrationalia," and explores the boundaries of that concept at the beginning of his book. His shows a welcome convergence in the point of view of the best writers on IQ testing, as James R. Flynn's recent book What Is Intelligence? supports these conclusions from a different direction with different evidence. Stanovich develops a theoretical framework, based on the latest cognitive science, and illustrated by diagrams in his book, of the autonomous mind (rapid problem-solving modules with simple procedures evolutionarily developed or developed by practice), the algorithmic mind (roughly what IQ tests probe, characterized by fluid intelligence), and the reflective mind (habits of thinking and tools for rational cognition). He uses this framework to show how cognition tapped by IQ tests ("intelligence") interacts with various cognitive errors to produce dysrationalia. He describes several kinds of dysrationalia in detailed chapters in his book, referring to cases of human thinkers performing as cognitive misers, which is the default for all human beings, and posing many interesting problems that have been used in research to demonstrate cognitive errors. For many kinds of errors in cognition, as Stanovich points out with multiple citations to peer-reviewed published research, the performance of high-IQ individuals is no better at all than the performance of low-IQ individuals. The default behavior of being a cognitive miser applies to everyone, as it is strongly selected for by evolution. In some cases, an experimenter can prompt a test subject on effective strategies to minimize cognitive errors, and in some of those cases prompted high-IQ individuals perform better than control groups. Stanovich concludes with dismay in a sentence he writes in bold print: "Intelligent people perform better only when you tell them what to do!" Stanovich gives you the reader the chance to put your own cognition to the test. Many famous cognitive tests that have been presented to thousands of subjects in dozens of studies are included in the book. Read along, and try those cognitive tests on yourself. Stanovich comments that if the many cognitive tasks found in cognitive research were included in the item content of IQ tests, we would change the rank-ordering of many test-takers, and some persons now called intelligent would be called average, while some other people who are now called average would be called highly intelligent. Stanovich then goes on to discuss the term "mindware" coined by David Perkins and illustrates two kinds of "mindware" problems. Some--most--people have little knowledge of correct reasoning processes, which Stanovich calls having "mindware gaps," and thus make many errors of reasoning. And most people have quite a lot of "contaminated mindware," ideas and beliefs that lead to repeated irrational behavior. High IQ does nothing to protect thinkers from contaminated mindware. Indeed, some forms of contaminated mindware appeal to high-IQ individuals by the complicated structure of the false belief system. He includes information about a survey of a high-IQ society that found widespread belief in false concepts from pseudoscience among the society members. Near the end of the book, Stanovich revises his diagram of a cognitive model of the relationship between intelligence and rationality, and mentions the problem of serial associative cognition with focal bias, a form of thinking that requires fluid intelligence but that nonetheless is irrational. So there are some errors of cognition that are not helped at all by higher IQ. In his last chapter, Stanovich raises the question of how different college admission procedures might be if they explicitly favored rationality, rather than IQ proxies such as high SAT scores, and lists some of social costs of widespread irrationality. He mentions some aspects of sound cognition that are learnable, and I encouraged my teenage son to read that section. He also makes the intriguing observation, "It is an interesting open question, for example, whether race and social class differences on measures of rationality would be found to be as large as those displayed on intelligence tests." Applying these concepts to my observation of Hacker News discussions after 1309 days since joining the community, I notice that indeed most Hacker News participants (I don't claim to be an exception) enter into discussions supposing that their own comments are rational and based on sound evidence and logic. Discussions of medical treatment issues, the main concern of the submitted blog post, are highly emotional (many of us know of sad examples of close relatives who have suffered from long illnesses or who have died young despite heroic treatment) and thus personal anecdotes have strong saliency in such discussions. The process of rationally evaluating medical treatments is the subject on entire group blogs with daily posts http://www.sciencebasedmedicine.org/index.php/about-science-based-medicine/ http://www.sciencebasedmedicine.org/index.php/about-science-... and has huge implications for public policy. Not only is safe and effective medical treatment and prevention a matter of life and death, it is a matter of hundreds of billions of dollars of personal and tax-subsidized spending around the world, so it is important to get right. Blog post author and submitter here tylerhobbs suggests disregarding an individual contrary anecdote, or a group of contrary anecdotes, as a response to a general statement about effective treatment or risk reduction established by a scientifically valid http://norvig.com/experiment-design.html http://norvig.com/experiment-design.html study. With that suggestion I must agree. Even medical practitioners themselves do have difficulty sticking to the evidence, http://www.sciencebasedmedicine.org/index.php/how-do-you-feel-about-evidence-based-medicine/ http://www.sciencebasedmedicine.org/index.php/how-do-you-fee... and it doesn't advance the discussion here to bring up a few heart-wrenching personal stories if the weight of the evidence is contrary to the cognitive miser's easy conclusion from such a story. That said, I see that the submitter here has developed an empirical understanding of what gets us going in a Hacker News discussion. Making a definite statement about what ought to be downvoted works much better in gaining comments and karma than asking an open-ended question about what should be upvoted, and I'm still curious about what kinds of comments most deserve to be upvoted. I'd like to learn from other people's advice on that issue how to promote more rational thinking here and how all of us can learn from one another about evaluating evidence for controversial claims.
horsehead 14y ago
But .... but .... anecdotes are what politicians use to get elected !!!! (e.g. Joe the Plumber, et al). If we downvote anecdotes, we won't have a justification to vote for scumbag politician with a heartfelt story to tell!! edit: here's a terrific example!!! http://www.washingtonpost.com/local/alabama-law-drives-out-illegal-immigrants-but-also-has-unexpected-consequences/2012/06/17/gJQA3Rm0jV_story.html http://www.washingtonpost.com/local/alabama-law-drives-out-i...
beagle3 14y ago
You shouldn't downvote contrarian anecdotes. You should just take them at face value for what they are. His notion that in the field of medicine you should disregard contrarian anecdotes because there's statistical evidence is horrible advice. If you actually looked at said statistical evidence, you'd realize it is very rarely strong, and often only relevant to 70% or so of the population (which is great for 2 out of 3, but useless for the third one). "Best practices" often aren't, and common sense is not at all common in medicine. A significant number of published results are plain wrong (see http://saveyourself.ca/articles/ioannidis.php http://saveyourself.ca/articles/ioannidis.php and the paper it references). A lot of medical advice is wrong, harmful or useless; The archives of seth roberts' blog are an enlightening read.
- iy56 14y ago
  >You should just take them at face value for what they are. The problem is that this is impossible for most people.
  - beagle3 14y ago
    That's true, and they also get to vote (both on HN, and in real world elections). Imagine the consequences! Or just observe them in the real world....
drostie 14y ago
If the comments below are too laborious for you to read, here is the TL;DR of them: (1) Use your personal discretion. Some anecdotes are funny. Others can be perfectly cogent rebuttals, especially when people make overbroad statements.[a] (2) The author is right in some respects: you need to be aware of the ways that stupid stories can bias you. Even being aware of this fact is sometimes not sufficient, so use downvoting to try to protect others. (3) Lots of people believe in anecdotal responses to anecdotal original-claims. [b] (4) A lot of people tried to be funny by offering anecdotes to be downvoted. Unfortunately, people haven't consistently taken the advice above, so they are not all in one place (at the bottom). That is a pity -- this would have been a fun and interesting use of downvotes. [a] Actually, the discussion is full of overbroad statements of this form like "no universal truth" and someone presenting "nothing should be above questioning" as above question. [b] I would like to formally respond that this is generally stupid -- you don't clean up a house by flinging crap at the crap. Your mileage may vary.
dicroce 14y ago
I disagree. In my experience, this article is wrong.
Shoomz 14y ago
I'm curious what some of your (the community's) thoughts are on articles like this. While the article itself has a large amount of personal bias with unsubstantiated claims, the conversation it has broached has been rather rich and enlightening. However, that is largely due to its controversial and contrarian (ironically) nature. That said, while the conversation has been thoughtful and looks to have largely refuted article, because so much of the thoughts put up here in the comments have been to downplay the original article, my question is: "Was this article worth our time or just an elaborate waste of time since the premise was bunk?"
nessus42 14y ago
As someone who has had his ability to work saved due to anecdotal evidence, I could not disagree more with the OP. Well, sure, if someone has anecdotal evidence that the theory of gravity is wrong, or that 1 + 1 = 4, by all means downvote them, but not all scientific evidence is on such solid ground. E.g., I've been prescribed medication and ended up with a side effect not listed on the drug literature. The doctor told me that the side-effect must be psychosomatic. Later scientific evidence revealed that about 50% of patients given the drug experienced the same side effect, but that it had been previously underreported. Who's to say that researchers would have even bothered to research the side effects more thoroughly if they hadn't paid attention to the fact that the anecdotal evidence contradicted the scientific evidence. Or remember when the scientific evidence seemed to indicate that a high carbohydrate, low fat diet was the healthiest choice? Should I have ignored my anecdotal evidence that that diet made me feel like crap? Science sometimes gets itself into harmful orthodoxies. See Thomas Kuhn for more info on this if you are unfamiliar. One example of this is Behaviorism in Psychology. In this field, it was scientific orthodoxy for many decades that Behaviorism was scientific and Cognitive Psychology was not because Behaviorism was based on only quantifiable, measurable data. It took Chomsky to point out the idiocy of this orthodoxy, thereby breaking the orthodoxy, allowing science to progress. Re the anecdotal evidence that saved my career, I've read here that there is no scientific evidence that ergonomic keyboards can help prevent or ameleroate RSI. I am 100% sure, however, that the Kinesis Contour keyboard saved my ability to type. All I have to do to know this for sure is listen to my body. Furthermore, I personally know about a dozen programmers who feel similiarly. I'm sure that someone will pipe in that this is almost certainly the placebo effect. If that's the case, then Kinesis makes the world's very best placebo, as placebo-like things have done precious little for me in any other area. In any case, even if it were the placebo effect, what does it hurt anyone to ignore the putative scientific evidence and try out a Kinesis keyboard for themselves to see if it provides them with relief? The idea that posts should be downvoted for recommending such ergonomic keyboards is insane.
lukev 14y ago
It's true, anecdotes are not data and have no statistical weight whatsoever. However, anecdotes can and do (and should) play a large role in influencing how we think. They can humanize a problem and create food for thought in a way that no amount of statistics ever could. For example, it's one thing to cite statistics that children of gay parents do just as well as those of straight parents (I have no idea if it's true, but it's an interesting contemporary question). But that's not likely to change the mind of a homophobe on the issue of gay adoption. On the flip side, a lucid, heartfelt anecdote from a person who had gay parents might actually help someone to understand what it's like to grow up in that environment an therefore become sympathetic. Of course, it has no bearing on the actual statistics at all, but sometimes statistics aren't the most important thing.
- tylerhobbs 14y ago
  That is an excellent point, and well expressed. I do see how anecdotes and stories can be used in a very positive way to shift opinion; there are definitely some grey regions. (How do you know you're on the right sight?)
deleted 14y ago
[deleted]

219 points by tylerhobbs 14y ago

lhnz 14y ago

The simplification in the title is dangerous without a 'when' attached. Mindlessly downvoting contrarian anecdotes will lead to a hive-mind. Beware. Opinions overwhelm all other forms of material in a discussion. Anecdotes are actually one up from opinions as they are concrete. You should weight them like so: Statistical Evidence + Logic > Statistical Evidence > "Common Knowledge/Wisdom" > Anecdote + Logic > Anecdote > Opinion + Logic > Opinion

maigret 14y ago

Or maybe just not upvote? Good argumentation, still can get in the way of why HN is here. It's not just a place to make formal argumentation, but to exchange experiences, or get some opinions from others. In some case the opinion is wished from experts (how can I get funded etc), in others it is about how a product is liked by general users. So please take a piece of salt in applying that ;)

pseudonym 14y ago

In my experience, some of the best advice I've ever received has been from anecdotes that run counter to the text of an article. ...I kid. It's actually an interesting idea. I don't know about scrapping them entirely, but I think a lot of sites (say, Reddit) could benefit from moving the anecdotes elsewhere. Partially because of this, and partially because anecdotes in any form tend to derail the discussion pretty darn quick.

necubi 14y ago
This is what makes r/AskScience so good. "Free of anecdotes" is a rule for posts, and it is vigorously enforced.

drivebyacct2 14y ago

Is this not based in the assumption that the article to which the anecdotes are offered as replies is omniscient? Which may in and of itself be fallacious.

VMG 14y ago
The assumption is only that the article is based on a scientific study, not that it is omniscient.
- r00fus 14y ago
  A study that rigidly follows the scientific principles, but is intellectually dishonest (ie, sponsored by interested party of studied subject, etc) is often not useful to the reader of the study results. Because you can follow the letter of the principles, and still flout the spirit. One should, when listening to a study, question the funding. Likewise, dissenting opinions must also be examined in what interested parties have a hand in their creation.
  - Zakharov 14y ago
    Yes, examining the biases and methodologies of a study is the correct way to present a contrarian opinion. Providing an anecdote is much less useful.
  - slurgfest 14y ago
    Sponsorship does not imply intellectual dishonesty. Intellectual dishonesty is a matter of how the argument is made. Sponsorship is a heuristic you could use to quickly filter papers but used as a counter-argument, it is simply ad hominem. You are describing a method for judging arguments without thinking critically about the arguments themselves or examining their basis in evidence and that is not any kind of science. One does not find any measure of objectivity by averaging between opinions, only by holding arguments to the yardstick of rationality and evidence.
drivebyacct2 14y ago
Thanks, you were great too.

deleted 14y ago

[deleted]

deleted 14y ago
[deleted]

rsanchez1 14y ago

The author didn't pick good examples of contrarian anecdotes found on HN. They are contrary to a study on coffee consumption and dementia. The study, however, is only a sample of 124 people from Tampa and Miami. The study is just as likely to have found an isolated effect as the posters with family members who consumed a lot of coffee and still developed alzheimers. You would be better served by this: https://sites.google.com/site/mccormickphilosophy/home/critical-thinking-syllabus/critical-thinking-schedule/understanding-statistical-arguments https://sites.google.com/site/mccormickphilosophy/home/criti... The author must be a frequent coffee drinker who didn't like that some people had a different experience with coffee than some other people had, and felt compelled to write that post. It's not the contrarian anecdotes that left me with the sense that the research findings weren't conclusive. It's the fact that a population of 124 people in two cities is not at all representative of the target population which numbers over 40 million according to the census bureau. http://quickfacts.census.gov/qfd/states/00000.html http://quickfacts.census.gov/qfd/states/00000.html Maybe when the study researchers conduct a larger study I'll believe them.

tylerhobbs 14y ago
Great comment. I am most certainly not claiming that you shouldn't question research findings on other grounds, such as sampling bias or statistical significance, as you have done here; those types of points are extremely valuable.
scalable 14y ago
N=124 can definitely give statistically significant results. Consider the following thought experiment. You have a coin that you think might be loaded. You flip it 124 times. You get 119 tails. It is probably loaded. Lets say you get 66 tails. Could definitely be due to chance. Moral: The larger effect you study, the less sample size you can get away with. It might be that people from Tampa and Miami are different in some important way from other people. However, if you want to level this criticism, maybe you should give a reason for suspecting this? "The study is just as likely to have found an isolated effect as the posters with family members who consumed a lot of coffee and still developed alzheimers." No, no, triple no.

verroq 14y ago

Rebuttals to articles (ones that are voted up anyway) in the comment section are almost always well reasoned refutations. This article is addressing a non-problem.

its_so_on 14y ago

Forget what you and your friends experience. If we manage to get a piece of research sponsored, you'd better believe it! Sorry, I'll go on the experiences of myself and people I trust over research/article spinning.

its_so_on 14y ago
if my friends and I have problems with your product, it doesn't matter how reliable your research spins it as, I won't believe you. Forget research: make something my friends and I, or people I hear about, don't have problems with. That's all I pay attention to, and it works a lot better than the alternative. (paying attention to whatever you "prove" at a statistically significant level - nevermind how many commissioned studies you don't publish, thereby completely invalidating that statistical significance - and ignoring anecdotes simply doesn't work, for me or anyone else.) This article is literally asking for the right to lie (under the guises of 'research') and asking us to mod down anyone who calls them out on it. It really takes some face to say "Ignore what you experience - and vote down the experiences of others - and trust our data instead." Next you'll sell me the most reliable cloud on the planet. All the responses on the article say they've had nothing but problems and downtime. But, I should just ignore these, right?

Udo 14y ago

No. Contrarian anecdotes are good. They may turn out to be without merit, but then again so may the article itself. Having a real discussion is a good thing. Also, I would like to propose a logarithmic scale for weighting such things. Say, if the article in question found something extraordinarily significant with 100 out of 100 samples resulting in A, then it's still not rational to weigh a contrarian viewpoint resulting in B with 1/101 - it should maybe be closer to 1/3 or something. Consensus culture and worship authority are not desirable in my opinion. Arguments should be weighed on their merits and it's appropriate to explore other viewpoints or explanations even if they turn out to be dead ends most of the time.

loup-vaillant 14y ago
Let's take an example, since they seem to work best: vaccines. Vaccines protect you from the risk of contracting particular diseases, some of which are crippling, lethal, or incurable. Plus, most are extremely effective: once you take your shots, you are effectively immune. That's good. There is a downside however. Sometimes, vaccines have side effects. Most side effects are quite benign, but if you're unlucky enough, they can be crippling, lethal, or incurable. That's bad. From a medical point of view, vaccines are a net good (let's leave aside logistic considerations, or the effort required to go to the doctor). When you look at the stats, you stand a much better chance at life and health if you take the shot. Even for relatively minor illnesses like the flu. Now, let's say someone post a heartbreaking comment about how her 9 year old daughter died of a vaccine shot, with all the gory details about the suffering, how she couldn't participate in her school's festival, the size of the coffin… I'm quite sure there are stories of the kind. Given the sheer amount of readers here, maybe one of you will more or less directly relate to that. My apologies to those who do. Nevertheless, what makes a good story doesn't necessarily make good evidence. When you know of reliable statistics, and you read a contrarian anecdote, you should shift your belief in the direction of the anecdote by a precise amount, which is almost always tiny. What your brain will actually do behind you back however, is shifting your belief by a significant amount, often crossing the "reasonable doubt" line. That's not rational, but that's what will happen. Nameless statistics feel abstract, remote. An anecdote on the other hand feels concrete, real, close. Worse, you can spend far more time reading about the salient anecdotes than learning about the end results of reliable, but boring, scientific studies. Another example: you don't win the lottery. Period. You don't know of any close family of friend that ever did. But maybe one of you readers do. Maybe that one could comment and say "Hey, but my cousin did win the lottery!". Would that prove me wrong? Not at all. It's just that when the sample size is huge enough, even the tiniest chance can actualize.

liquidcool 14y ago

I've noticed that many of my biases, typically for or against some name brand or technology, even if not anecdotal, are old. I now try to remember not only how I came to this opinion, but when. I guess that's one of the consequences of aging.

philwelch 14y ago
My dad once received an inappropriate form letter from Texaco in the 1970's and never went to a Texaco station since.

hasenj 14y ago

You should question everything. This includes studies published and labeled "scientific". Contrarian anecdotes are important (but they too should be questioned). Nothing should be above questioning; even prevailing wisdom.

flyinRyan 14y ago
>even prevailing wisdom. Especially prevailing wisdom, since it is least likely to be questioned normally.

dennisgorelik 14y ago

If best contrarian comments are weak - I assume that original article is probably correct. If I don't see any contrarian comments, then I suspect groupthink in discussion.

D_Alex 14y ago

I totally get where the author is coming from, but I am not reeeally sure if that is good advice. Anecdotal evidence has its place, eg: proof by counterexample, challenging false, but widely held beliefs, or alerting people to new circumstances. A: "Murrumbidgee River is fun and safe for children to swim in! No one came to any harm there for the last 100 years!" B: "Dunno... my dog was eaten by a crocodile there last Saturday..." B's dog is a sample of one, but maybe worth paying attention to.

jessedhillon 14y ago
In your example, A is not relating a statistical or experimental finding, but instead dispensing commonsense knowledge which happens to be objectively false. It's not what the OP is talking about.
- JoeAltmaier 14y ago
  Commonsense, like ulcers are caused by diet and anxiety, or that salt raises blood pressure? I think a good point is made: statistical evidence is also misleading - it deliberately ignores (averages out) the extreme cases. The results are a distribution; statistics folds that into one number. Anecdotes fill out the distribution.
  - carbocation 14y ago
    Ulcers are not generally caused by diet and anxiety, but rather by bacterial infection. Anecdote is a much more compelling refutation of "common sense" than it is of empirical statistical evidence. Statistical evidence is not misleading; it's simply the case that if you are seeking outliers, as in your example, then looking at measures of central tendency won't contain what you're looking for. Anecdote has no role in "fill[ing] out the distribution."
TeMPOraL 14y ago
You don't challenge "false, but widely held beliefs" with anecdotes - those beliefs actually is exactly what anecdote becomes when it starts being repeated. Alerting people to new circumstances is a good use, but that's pretty much all. Proof by counterexample works well only in mathematical logic, which is not how the real world works[1]. In reality you enter the domain of probability theory, and there counterexamples work exactly as the article author says - as an evidence that needs to be properly weighted. And these are those weights that people vastly overestimate in case of anegdotes. [1] - by this I mean, mathematical logic is not how we model, comprehend and operate reality.

pgsandstrom 14y ago

Are people joking when they defend anecdotal stories with "once this anecdotal story proved to be true and the study was faulty"? The proposition that the article makes is not that anecdotal stories must be false, but that they might influence the reader more than they should. Thus they should not be encouraged.

JoeAltmaier 14y ago
Yeah, thanks for deciding how gullible I am, and please do sanitize what I get to read for my own good. That's how open dialog works, right?
- pgsandstrom 14y ago
  What do you think the voting system on HN does? It promotes good content, thus also demotes bad content. The author is saying "Here is a reason that content X is bad. If you agree with me, downvote it".
  - JoeAltmaier 14y ago
    Doesn't make sense. If the content is 'bad', moderator could expunge it. Or writer could be counseled on giving better posts. Or any number of things. The voting system on HN sometimes demotes bad content. Far more often (discussed widely elsewhere) its used to assert agreement or disagreement. When used in this sense, its a sort of instant-poll. In the case of anecdotal argument, it forms an ad-hoc 'scientific' experiment. The HN population weighs in using their own experience. Those who's memory (or ok, memory-triggered emotion response) aligns with the author may upvote etc. To 'bad content' is uniformely suppressed instead, this social experiment is lost, and the community loses. The merit of instant-polls is debatable, but other such polls are supported here. In fact the test group on HN may exceed the original 'scientific paper' group by an order of magnitude. Its statistical significance may exceed a graduate student's narrow study.
    - carbocation 14y ago

rlpb 14y ago

A contrarian anecdote may help expose an article being false or misleading in itself. For example: a review written by the manufacturer itself, or astroturfing. First we must determine the validity of the article existing in itself and any motivations or biases in it. For this, contrarian anecdotes are useful. This article seems to miss this point. Once we can accept the article at face value, then we may hit the biases described.

scalable 14y ago
I would say it is far easier to astroturf a contrarian anecdote.

the_bear 14y ago

If a post is about a topic that can and has been thoroughly researched and a reasonable conclusion has been drawn, then sure, a contrarian anecdote should probably be disregarded. But honestly, how often does that apply? Most of the conversations on HackerNews aren't about things that have right and wrong answers. As a matter of fact, many of the most popular posts are nothing more than anecdotes themselves. So why exactly should an anecdote in the comments be downvoted?

retrogradeorbit 14y ago

In a world were 47 of 53 landmark cancer research papers were found to not be replicable... where one scientist admitted (after the meta-researchers tried 50 times to replicate the result and failed) that they had run the experiment 6 times and got the result once and reported that because it made a better "story"... where the act of suppressing refutations of other's bogus research is "culturally commonplace"... Something is rotten in the core of science. These days I give the research the same weight I give the anecdote... and that is no weight at all. Science has been supplanted by money and politics... At least anecdotes admit they're anecdotes! http://news.yahoo.com/cancer-science-many-discoveries-dont-hold-174216262.html http://news.yahoo.com/cancer-science-many-discoveries-dont-h...

Peaker 14y ago
Surely you could find some scientists or scientific institutions you can trust? Or maybe you can trust those experiments that do get replicated successfully?
- JohnHaugeland 14y ago
  Given that the context is "downvote anything that Reddit or HN disagree with," it seems that the point is "don't follow flocks while voting; make your own decisions or don't vote."
bad_user 14y ago
The problem with much of modern medicine is that much of it is based on flawed and biased statistical studies. Whether this is done because medical personnel don't have training in statistics, or because such studies generate funding, I don't know, but something is definitely rotten. Let's take anything involving nutrition. Some challenges are: (1) people lie, (2) such studies can't be double-blind so placebo kicks in, (3) the statistical significance of short-term studies is zero, (4) you can't control all the variables, unless you lock those people in a cage and (5) most conclusions of such studies have the potential to confuse the cause and the effect. But not all of science is like that. Just medicine.

keithpeter 14y ago

"But in the presence of statistical evidence, don’t tolerate contrarian anecdotes, and don’t make them yourself, knowing the exaggerated impact they can have." When you start to design a statistical experiment, you have already made an important methodological choice. See http://en.wikipedia.org/wiki/Positivism http://en.wikipedia.org/wiki/Positivism http://en.wikipedia.org/wiki/Grounded_Theory http://en.wikipedia.org/wiki/Grounded_Theory I've noticed that in my own field which is education, there appears to be a fondness for sophisticated statistics, even though no manager ever allocated students to teachers on a double blind random basis. An excellent example is the way the UK Education ministry has decided that 'phonics' is the way to teach reading. http://www.bbc.co.uk/news/education-18462214 http://www.bbc.co.uk/news/education-18462214 I think that this general tendency might be an example the 'white coat syndrome' in action; belief that using formal statistical techniques might increase the meaningfulness of the results. I suppose that is a form of cargo cult. This is hacker news, a forum aimed at people with novel business proposals and new software to try out. Should you be trying to find 'the Truth' or should you be building some grounded theory that tells you what to do next, provisionally, now, today?

nathan_long 14y ago

I downvoted some contrarian anecdotes once, and it ruined the thread.

hexagonal 14y ago
I downvoted some contrarian anecdotes once, and my "downvote" button was disabled.

JohnHaugeland 14y ago

This is an extremely toxic position, which will lead to novices (who outnumber the experienced significantly) mathematically abusing voting systems to promote whatever this month's fad tool golden hammer is. The herd is rarely correct.

chris_wot 14y ago

Have their been stories with seemingly solid statistical data, where anecdotal evidence shows that the statistical evidence is wrong?

brudgers 14y ago

By this logic, HN'ers should downvote any anecdote describing successfully exiting a startup because statistics show that the vast majority of startups fail and these anecdotes describing success unduly influence the thinking of entrepreneurs.

Symmetry 14y ago
Only if its offered as a counterexample to the proposition that most startups fail.
- brudgers 14y ago
  One might suggest that that such a proposition is an underlying premise of HN.

b1daly 14y ago

Well I think Stalin summed up the perils of totally ignoring anecdotes pretty well: "One death is a tragedy; one million is a statistic." This topic sort of came up the other day in the thread about the girl losing the iPad software she needs to talk (Silencing Maya). http://news.ycombinator.com/item?id=4103344 http://news.ycombinator.com/item?id=4103344 Some commenters thought the story should be ignored as a data point about the societal value of patents. I disagree in that case because I don't believe economics or social sciences have anywhere near the amount of rigor and theory to make a good claim on whether the patent system is a net benefit to society. Deciding such a thing is a very old, and classic problem in philosophy. The OP seems to implicitly relying on a variant of utilitarianism, which IMO is wholefully inadequate to rely on for moral decisions. Since science is so hard, there is a lot of bad science out there. What gets reported in the wider media has all sorts of weird selection bias, never mind what gets picked for publication in journals. Anecdotes are human stories and they are deeply connected to why we care. Statiscal tools can be used in ways that justify harm people in the name of greater good. I agree that scientific medicine provides tools for "real"'medicine that other methods don't. I just think we should remember to real peoples stories, especially if they are true. Anyone reading this probably gets that one can't exactly counter balance the effect of anecdotes within themselves. But it can still be countered some, I think it's worth it to hear stories (that are relaxant to the topic). Now what is the basis of my beliefs? Mostly intuition, not rationality. But I don't think it's possible to undermine my ideas with some kind of experimentally bases argument. There's just way to many variables!

slurgfest 14y ago
I don't understand the part about Stalin's advice. What was Stalin saying would happen if you ignored anecdotes? How do you figure he was talking about the value of anecdotes?
- gwern 14y ago
  Funny thing, I always took this quote as Stalin cynically remarking on exactly OP's point: people overweight anecdotes by thousands to millions of times their actual worth as Bayesian evidence.

CookWithMe 14y ago

I'm all in when it comes to call bullshit on something. But because we (hackers) are used to boolean decisions, we are looking for counter-examples. But these are just non-arguments when the finding is "In 9 out of 10 cases, ...". The anecdote is the 1 out of 10 case. Let's try to question anything, and when it is suspicious - by all means, say so! But, I guess what the author is trying to say (beyond the whole Downvote discussion) is that we should try to reason and make a proper argument. Especially one that is not already addressed in the research.

TeMPOraL 14y ago
> But these are just non-arguments when the finding is "In 9 out of 10 cases, ...". The anecdote is the 1 out of 10 case. The whole point of the article is that people won't judge that anecdote as only 0.1 relevant, but much much more (I'd say close to 0.9 if they personally prefer what anecdote says to what research says). It's a general human flaw that's very visible in day-to-day interactions with people.

sehugg 14y ago

Anecdotes have their place, especially in complex domains. For example, I have a friend who uses medical marijuana successfully to control MS symptoms. That is a sample size of one, but it certainly colors my view of the debate. I have a much deeper view into the decision making process of a MS sufferer now than I would otherwise, one that goes beyond spasticity, pain, and cognitive scores.

kbronson 14y ago

I'm going to downvote this article because contradicts everything said here.

DanielBMarkham 14y ago

I had to vote this up because of the meta-nature of the article. If anybody makes a comment disagreeing with it while sharing an anecdote they only play into's the author's theme! It's like accusing somebody of being in denial: there's nothing they can say in return that doesn't somehow support the idea that they are in denial. Love it. Having said that, this is another really bad article in what seems like an endless series of bad articles on HN. Here are a few of the more obvious flaws: - There is no universal truth as the author seems to imply. Simply because some publication or source you may like has performed some sort of statistical study doesn't have a lot of meaning on it's own. Yes, 100% of the people who eat bananas are dead within 120 years of their consumption. No, that does not mean bananas are bad. The study or scientific reporting is simply the beginning of a much longer conversation society has over many decades that leads us to higher-fidelity models. - The purpose of a social site is to behave socially. While places like HN have (or used to have) a lot of different guidelines for the types of behaviors that are encouraged or not, being social means sharing stories, anecdotes. We are not robots. - The idea that people are unable to sort out personal anecdotes from other forms of information. The follow-up idea that since they are not able to do this, we should prevent ourselves from sharing such stories. This is bad, bad, bad, bad, bad, bad .... bad(n). We are humans. We share stories. Anybody who says "people are so stupid" can justify just about damn near anything as long as they keep emphasizing the stupidity and danger some people's actions represent. Every now and then, gasp(!), published research is either wrong or doesn't show anything near what the reporter claims. Anecdotes don't help with this, of course, but they serve to remind us that even well-known scientists working at the highest quality standards available are still just sharing with us a very specialized form on anecdote. We did these things in this way and this is what we observed. Here is how you also can observe this. The really "good" part of the story they are sharing is talking about hidden assumptions, population variance, reproducibility, and so forth. Anecdotes don't do this, but they help us brainstorm ways in which we can improve the discussion, take the next experiment to an even better place. I'm very uncomfortable with the line of reasoning that goes somewhat like this: people are broken in some way, therefore we must somehow control what they read, say, or think for their own good. To me the beauty of western civilization is that really broken people can do these amazing and awesome things. The fact that we're deeply flawed is the magic. Science and human advancement work because of our flaws, not in spite of them. This is a very important thing to understand! Setting up some ideal of perfection, no matter how well-intended, and then mucking around with the way societal interaction works in some effort to improve on things is heading down a very dark path that has a very unhappy ending. This attitude seems rife in the technology community, however, perhaps because we are such analytical people. I don't want my fellow man to be irrational and distrustful of science and knowledge. But I'll take that any day over silencing contrarian articles and dissent. We've done the math on this: wrong people who share emotional stories and persuade crowds about all sorts of illogical things are a price that a dynamic community pays for progress.

hexagonal 14y ago

For example, if a close friend goes on and on about how the Ford he bought was a piece of crap, detailing how the transmission failed at 30k miles and the rear-view mirror fell off, you’ll be wary about buying a Ford in the future, even if Consumer Reports rates them highly. Personal anecdote: I've got a '93 Ford Explorer. It has 237,000 miles on it, and has yet to have a single major component failure. I've been waiting for the goddamn thing to die for the last 70,000 miles, so I can buy a car that does better than 13 miles to the gallon, but no dice. Don't buy Ford trucks. They're too reliable.

Spooky23 14y ago

I guess it depends on the story. If HN is supposed to be about startup culture and technology, the anecdote is essential to being vibrant. What is a startup if not a story whose premise is unproven? Now if you're telling me that your sister stuck a magnet in her ear and cured her cancer, I'm not going to give that credence without some real data. Most things on HN are not that, and a good story often compels us to think. So FWIW, I'm not downvoting contrarians, and I'm not downvoting anecdotes for being anecdotes.

motters 14y ago

Without contrary viewpoints there is always a danger of falling into overconfidence and group-think.

alan_cx 14y ago

Please forgive me if I have fundamentally misunderstood this, but is this article suggesting that anecdotes are some how comparable to scientific research data? If one wants to "attack" an anecdote, then a contrarian anecdote is the weapon. If one wants to attack scientific data, you need contrarian scientific data. At least, I hope that is right. There for, to mix the two is like attacking a tank with a wooden stick. Surely, anecdotes are used as the premise of scientific research. Lots of people tell stories. There seems to be something interesting going on. Then you do the research and produce the data. If the data is conclusive, then your past the anecdote. If later contrarian anecdotes appear, and they seem significant, off you go to scientific research again. I know Im wrong somewhere. But where?

Goladus 14y ago

Contrarian anecdotes can be very useful (and logically sound) when they are contrary to a generalization. If someone asserts "f(x) = 5 for all x in A", "f(a) = 3 and a in A" is a logical contradiction proving the assertion false. And, frankly, if we're talking about public discussion of scientific papers, inappropriate generalization is as big a problem as contrarian anecdotes, if not bigger. Scientific papers often cover highly specific observations that are useful primarily to other researchers, and often even then not for many years. People then try to apply that specific knowledge to practical day to day situations.

Symmetry 14y ago
Have you ever seen that sort of generalization in an article on Hacker News? Even when you have situations like, e.g. nobody has ever recorded a case of a person with a certain genotype being infected by a norovirus, you don't see papers claiming that its impossible.
- Goladus 14y ago
  Yes. Realize that often the generalizations are a lot more subtle than your example, and of course anecdotes aren't always the best or only counter-arguments you'll see, even if they are valid. Start looking and I am sure you will find examples.

vannevar 14y ago

You shouldn't downvote to censor relevant information, period. You should only downvote if the material violates site guidelines. Downvoting to disagree is lazy, substituting easy censorship for thoughtful response. Instead, post your counter-argument or upvote an existing opinion that already expresses your position.

Symmetry 14y ago
Censoring relevant information is, of course, never good but the authors contention was that the information isn't actually relevant. Its always good that people who are wrong get a cogent reply, but beyond that adding a "me too" post doesn't have any more benefit than a downvote does. I used to just upvote post I agreed with, but since those are no longer public the only way to clue readers in to what the consensus is is to downvote.
- vannevar 14y ago
  I'm not sure I'd agree he was arguing for the irrelevance of anecdotal evidence, only that it has a disproportionate impact when considered with statistical evidence. Rather than censoring the anecodotes, the better solution is to simply point out the danger of overweighting them. The purpose of a site like HN is not to arrive at some kind of imaginary consensus, it's to inform and engage people in meaningful discussion. There is no winning side to be on, and no ultimate arbiter of truth. By downvoting as you do, you rob the site of content in exchange for an illusory sense of victory.

grandalf 14y ago

The idea that there was an impending credit crisis was a "contrarian anecdote" in 2008. The idea that the US Government conducted illegal eavesdropping/wiretapping operations against US Citizens was a "contrarian anecdote" in 2003. This blog post condescendingly claims that HN readers are not sophisticated enough to balance out the sources that are input to their rational decision making process. In reality, contrarian opinions sometimes turn out to be correct, and mainstream opinions sometimes turn out to be wrong. Often, the benefit of a contrarian opinion is that it causes people to ask more questions, which is rarely a bad thing. People are sheeplike enough without having to be encouraged to follow the herd! I suggest everyone find a few contrarian theories and imagine what it would be like if you rearranged your life as if you expected them to be totally true. Most people are unwilling to go that far, and yes, in that way contrarian stories can weaken rational processes.

ubernostrum 14y ago
The idea that there was an impending credit crisis was a "contrarian anecdote" in 2008. As the saying goes: they laughed at Galileo. They laughed at Einstein. And they laughed at Bozo the Clown.
Symmetry 14y ago
Contrarian opinion is often correct, but contrarian anecdote in the face of statistical evidence seldom is. The article was about the later, not the former.

dclowd9901 14y ago

Contrarian anecdotes are an ill-conceived method of contributing to a conversation, and doing so in some sort of attention-getting way. They appear to all but undermine the entire premise of the original content, and yet, are based on absolutely nothing but one's individual experience with the matter. I see this kind of stuff all the time on Reddit, and it's become so numbing that you know, no matter how well you construct an argument or how many facts you cite, someone will always come out of the wood works and tell you how you're wrong because it didn't happen that way for them. It wouldn't be so bad if people didn't take that as a genuine counter-argument.

TimPC 14y ago

As a caveat, you should only do this in response to data based articles. Articles that are themselves anecdotal, should receive anecdotes that are contrarian. I think the more general idea is comments should be held to similar standards of evidence that the article presents.

TimPC 14y ago

tokenadult 14y ago

This is an interesting first submission by a Hacker News participant who joined the community 551 days ago. The core idea in the submitted blog post (by the submitter here) is "Contrarian anecdotes like these are particularly common http://news.ycombinator.com/item?id=4076643 http://news.ycombinator.com/item?id=4076643 http://news.ycombinator.com/item?id=4076066 http://news.ycombinator.com/item?id=4076066 in medical discussions, even in fairly rational communities like HN. I find this particularly insidious (though the commenters mean no harm), because it can ultimately sway readers from taking advantage of statistically backed evidence for or against medical cures. Most topics aren’t as serious as medicine, but the type of harm done is the same, only on a lesser scale." The basic problem, as the interesting comments here illustrate, is that human thinking has biases that ratchet discussions in certain directions even if disagreement and debate is vigorous. The general issue of human cognitive biases was well discussed in Keith R. Stanovich's book What Intelligence Tests Miss: The Psychology of Rational Thought. http://yalepress.yale.edu/yupbooks/book.asp?isbn=9780300164626 http://yalepress.yale.edu/yupbooks/book.asp?isbn=97803001646... http://www.amazon.com/What-Intelligence-Tests-Miss-Psychology/dp/0300164629/ http://www.amazon.com/What-Intelligence-Tests-Miss-Psycholog... The author is an experienced cognitive science researcher and author of a previous book How to Think Straight about Psychology. He writes about aspects of human cognition that are not tapped by IQ tests. He is part of the mainstream of psychology in feeling comfortable with calling what is estimated by IQ tests "intelligence," but he disagrees that there are no other important aspects of human cognition. Rather, Stanovich says, there are many aspects of human cognition that can be summed up as "rationality" that explain why high-IQ people (he would say "intelligent people") do stupid things. Stanovich names a new concept, "dysrationalia," and explores the boundaries of that concept at the beginning of his book. His shows a welcome convergence in the point of view of the best writers on IQ testing, as James R. Flynn's recent book What Is Intelligence? supports these conclusions from a different direction with different evidence. Stanovich develops a theoretical framework, based on the latest cognitive science, and illustrated by diagrams in his book, of the autonomous mind (rapid problem-solving modules with simple procedures evolutionarily developed or developed by practice), the algorithmic mind (roughly what IQ tests probe, characterized by fluid intelligence), and the reflective mind (habits of thinking and tools for rational cognition). He uses this framework to show how cognition tapped by IQ tests ("intelligence") interacts with various cognitive errors to produce dysrationalia. He describes several kinds of dysrationalia in detailed chapters in his book, referring to cases of human thinkers performing as cognitive misers, which is the default for all human beings, and posing many interesting problems that have been used in research to demonstrate cognitive errors. For many kinds of errors in cognition, as Stanovich points out with multiple citations to peer-reviewed published research, the performance of high-IQ individuals is no better at all than the performance of low-IQ individuals. The default behavior of being a cognitive miser applies to everyone, as it is strongly selected for by evolution. In some cases, an experimenter can prompt a test subject on effective strategies to minimize cognitive errors, and in some of those cases prompted high-IQ individuals perform better than control groups. Stanovich concludes with dismay in a sentence he writes in bold print: "Intelligent people perform better only when you tell them what to do!" Stanovich gives you the reader the chance to put your own cognition to the test. Many famous cognitive tests that have been presented to thousands of subjects in dozens of studies are included in the book. Read along, and try those cognitive tests on yourself. Stanovich comments that if the many cognitive tasks found in cognitive research were included in the item content of IQ tests, we would change the rank-ordering of many test-takers, and some persons now called intelligent would be called average, while some other people who are now called average would be called highly intelligent. Stanovich then goes on to discuss the term "mindware" coined by David Perkins and illustrates two kinds of "mindware" problems. Some--most--people have little knowledge of correct reasoning processes, which Stanovich calls having "mindware gaps," and thus make many errors of reasoning. And most people have quite a lot of "contaminated mindware," ideas and beliefs that lead to repeated irrational behavior. High IQ does nothing to protect thinkers from contaminated mindware. Indeed, some forms of contaminated mindware appeal to high-IQ individuals by the complicated structure of the false belief system. He includes information about a survey of a high-IQ society that found widespread belief in false concepts from pseudoscience among the society members. Near the end of the book, Stanovich revises his diagram of a cognitive model of the relationship between intelligence and rationality, and mentions the problem of serial associative cognition with focal bias, a form of thinking that requires fluid intelligence but that nonetheless is irrational. So there are some errors of cognition that are not helped at all by higher IQ. In his last chapter, Stanovich raises the question of how different college admission procedures might be if they explicitly favored rationality, rather than IQ proxies such as high SAT scores, and lists some of social costs of widespread irrationality. He mentions some aspects of sound cognition that are learnable, and I encouraged my teenage son to read that section. He also makes the intriguing observation, "It is an interesting open question, for example, whether race and social class differences on measures of rationality would be found to be as large as those displayed on intelligence tests." Applying these concepts to my observation of Hacker News discussions after 1309 days since joining the community, I notice that indeed most Hacker News participants (I don't claim to be an exception) enter into discussions supposing that their own comments are rational and based on sound evidence and logic. Discussions of medical treatment issues, the main concern of the submitted blog post, are highly emotional (many of us know of sad examples of close relatives who have suffered from long illnesses or who have died young despite heroic treatment) and thus personal anecdotes have strong saliency in such discussions. The process of rationally evaluating medical treatments is the subject on entire group blogs with daily posts http://www.sciencebasedmedicine.org/index.php/about-science-based-medicine/ http://www.sciencebasedmedicine.org/index.php/about-science-... and has huge implications for public policy. Not only is safe and effective medical treatment and prevention a matter of life and death, it is a matter of hundreds of billions of dollars of personal and tax-subsidized spending around the world, so it is important to get right. Blog post author and submitter here tylerhobbs suggests disregarding an individual contrary anecdote, or a group of contrary anecdotes, as a response to a general statement about effective treatment or risk reduction established by a scientifically valid http://norvig.com/experiment-design.html http://norvig.com/experiment-design.html study. With that suggestion I must agree. Even medical practitioners themselves do have difficulty sticking to the evidence, http://www.sciencebasedmedicine.org/index.php/how-do-you-feel-about-evidence-based-medicine/ http://www.sciencebasedmedicine.org/index.php/how-do-you-fee... and it doesn't advance the discussion here to bring up a few heart-wrenching personal stories if the weight of the evidence is contrary to the cognitive miser's easy conclusion from such a story. That said, I see that the submitter here has developed an empirical understanding of what gets us going in a Hacker News discussion. Making a definite statement about what ought to be downvoted works much better in gaining comments and karma than asking an open-ended question about what should be upvoted, and I'm still curious about what kinds of comments most deserve to be upvoted. I'd like to learn from other people's advice on that issue how to promote more rational thinking here and how all of us can learn from one another about evaluating evidence for controversial claims.

horsehead 14y ago

But .... but .... anecdotes are what politicians use to get elected !!!! (e.g. Joe the Plumber, et al). If we downvote anecdotes, we won't have a justification to vote for scumbag politician with a heartfelt story to tell!! edit: here's a terrific example!!! http://www.washingtonpost.com/local/alabama-law-drives-out-illegal-immigrants-but-also-has-unexpected-consequences/2012/06/17/gJQA3Rm0jV_story.html http://www.washingtonpost.com/local/alabama-law-drives-out-i...

beagle3 14y ago

You shouldn't downvote contrarian anecdotes. You should just take them at face value for what they are. His notion that in the field of medicine you should disregard contrarian anecdotes because there's statistical evidence is horrible advice. If you actually looked at said statistical evidence, you'd realize it is very rarely strong, and often only relevant to 70% or so of the population (which is great for 2 out of 3, but useless for the third one). "Best practices" often aren't, and common sense is not at all common in medicine. A significant number of published results are plain wrong (see http://saveyourself.ca/articles/ioannidis.php http://saveyourself.ca/articles/ioannidis.php and the paper it references). A lot of medical advice is wrong, harmful or useless; The archives of seth roberts' blog are an enlightening read.

iy56 14y ago
>You should just take them at face value for what they are. The problem is that this is impossible for most people.
- beagle3 14y ago
  That's true, and they also get to vote (both on HN, and in real world elections). Imagine the consequences! Or just observe them in the real world....

drostie 14y ago

If the comments below are too laborious for you to read, here is the TL;DR of them: (1) Use your personal discretion. Some anecdotes are funny. Others can be perfectly cogent rebuttals, especially when people make overbroad statements.[a] (2) The author is right in some respects: you need to be aware of the ways that stupid stories can bias you. Even being aware of this fact is sometimes not sufficient, so use downvoting to try to protect others. (3) Lots of people believe in anecdotal responses to anecdotal original-claims. [b] (4) A lot of people tried to be funny by offering anecdotes to be downvoted. Unfortunately, people haven't consistently taken the advice above, so they are not all in one place (at the bottom). That is a pity -- this would have been a fun and interesting use of downvotes. [a] Actually, the discussion is full of overbroad statements of this form like "no universal truth" and someone presenting "nothing should be above questioning" as above question. [b] I would like to formally respond that this is generally stupid -- you don't clean up a house by flinging crap at the crap. Your mileage may vary.

dicroce 14y ago

I disagree. In my experience, this article is wrong.

Shoomz 14y ago

I'm curious what some of your (the community's) thoughts are on articles like this. While the article itself has a large amount of personal bias with unsubstantiated claims, the conversation it has broached has been rather rich and enlightening. However, that is largely due to its controversial and contrarian (ironically) nature. That said, while the conversation has been thoughtful and looks to have largely refuted article, because so much of the thoughts put up here in the comments have been to downplay the original article, my question is: "Was this article worth our time or just an elaborate waste of time since the premise was bunk?"

nessus42 14y ago

As someone who has had his ability to work saved due to anecdotal evidence, I could not disagree more with the OP. Well, sure, if someone has anecdotal evidence that the theory of gravity is wrong, or that 1 + 1 = 4, by all means downvote them, but not all scientific evidence is on such solid ground. E.g., I've been prescribed medication and ended up with a side effect not listed on the drug literature. The doctor told me that the side-effect must be psychosomatic. Later scientific evidence revealed that about 50% of patients given the drug experienced the same side effect, but that it had been previously underreported. Who's to say that researchers would have even bothered to research the side effects more thoroughly if they hadn't paid attention to the fact that the anecdotal evidence contradicted the scientific evidence. Or remember when the scientific evidence seemed to indicate that a high carbohydrate, low fat diet was the healthiest choice? Should I have ignored my anecdotal evidence that that diet made me feel like crap? Science sometimes gets itself into harmful orthodoxies. See Thomas Kuhn for more info on this if you are unfamiliar. One example of this is Behaviorism in Psychology. In this field, it was scientific orthodoxy for many decades that Behaviorism was scientific and Cognitive Psychology was not because Behaviorism was based on only quantifiable, measurable data. It took Chomsky to point out the idiocy of this orthodoxy, thereby breaking the orthodoxy, allowing science to progress. Re the anecdotal evidence that saved my career, I've read here that there is no scientific evidence that ergonomic keyboards can help prevent or ameleroate RSI. I am 100% sure, however, that the Kinesis Contour keyboard saved my ability to type. All I have to do to know this for sure is listen to my body. Furthermore, I personally know about a dozen programmers who feel similiarly. I'm sure that someone will pipe in that this is almost certainly the placebo effect. If that's the case, then Kinesis makes the world's very best placebo, as placebo-like things have done precious little for me in any other area. In any case, even if it were the placebo effect, what does it hurt anyone to ignore the putative scientific evidence and try out a Kinesis keyboard for themselves to see if it provides them with relief? The idea that posts should be downvoted for recommending such ergonomic keyboards is insane.

lukev 14y ago

It's true, anecdotes are not data and have no statistical weight whatsoever. However, anecdotes can and do (and should) play a large role in influencing how we think. They can humanize a problem and create food for thought in a way that no amount of statistics ever could. For example, it's one thing to cite statistics that children of gay parents do just as well as those of straight parents (I have no idea if it's true, but it's an interesting contemporary question). But that's not likely to change the mind of a homophobe on the issue of gay adoption. On the flip side, a lucid, heartfelt anecdote from a person who had gay parents might actually help someone to understand what it's like to grow up in that environment an therefore become sympathetic. Of course, it has no bearing on the actual statistics at all, but sometimes statistics aren't the most important thing.

tylerhobbs 14y ago
That is an excellent point, and well expressed. I do see how anecdotes and stories can be used in a very positive way to shift opinion; there are definitely some grey regions. (How do you know you're on the right sight?)

deleted 14y ago

[deleted]