9 ms·
How not to be a crank: rules for not being a science-dick
- Finnucane 9y agoIf we extended these rules to the internet, Facebook and Twitter would go out of business.
- wonderbear 9y agoCurrently I'm trying to think of a way to get my father to read this without overtly telling him he needs to read this. Because that would be being a crank.
- meri_dian 9y agoOpen it on his computer and walk away
- ak39 9y agoThis is bad advice. I hope you only being funny.
- icebraining 9y agoMaybe send it in a mix of "interesting links" of various topics.
- acidburnNSA 9y agoSell it like it's going to help him get his important messages across more effectively.
- rz2k 9y agoThink deeply about your father. Reflect on times when he's been funniest, had the most fun, and why you admire him. From this perspective think about how you would want to talk about this article. Maybe ask about his opinion on some of the details in the middle. In other words rather than figuring out a strategy to get him to read it that is adversarial but somehow slips under his defenses, figure out how to get yourself in the right frame of mind to introduce ideas on how to be more effective to someone you think is important. Presumably your father is enough like you that he will be able to tell when you are treating him like someone who needs to be emotionally managed, as opposed to when you are having a two-way conversation. Maybe one marker of managing someone is when the speaker is not focusing on themselves at all. When you talk to a friend you like you are likely concerned about how you are portraying yourself as well as whether or not you are getting your way.
- exabrial 9y agoAll of these are extremely important! Two life skills: * If you are without a doubt, you are wrong If you can't fathom the other side and where they're coming from, you, are in fact, biased beyond listening to facts, and are likely wrong. * Words matter As my grandma said, "If you don't have anything nice to say don't say anything at all."
- ars 9y ago> If you can't fathom the other side and where they're coming from, you, are in fact, biased beyond listening to facts, and are likely wrong. This one especially applies to politics. If you are one of those looking for "sanity", or "common sense", and can't imagine how the other side can possibly hold some position, then this applies to you.
- Karunamon 9y agoI’m not sure that’s accurate. People hold all manner of irrational political beliefs and there is value in recognizing that irrationality.
- tensor_rank_0 9y agoit's a lot easier to dismiss political ideas that you don't like by labeling them "irrational" than it is to open your mind to the reasons why a person might rationally disagree with you.
- TheAdamAndChe 9y agoYes, but there is also the real possibility that your beliefs are irrational as well. Everyone thinks they're right, and everyone has a train of thought that has led to that belief.
- TheOtherHobbes 9y agoA key test for rationality is prediction. If you believe that banning sex education in schools will decrease teen pregnancy, and it turns out that teen pregnancy rates actually increase when teens aren't educated, then your belief about sex education isn't rational. Many - possibly most - public policy decisions are open to this kind of testing. The fact that formal testing rarely happens, and that it's consistently ignored even when it does, suggests that politics itself is inherently pre-rational. Arguably it's then irrational to expect it to be rational - but that's a deeper problem that doesn't just apply to politics.
- arkem 9y agoThis is great advice for information security professionals as well, especially vulnerability researchers and other appsec types!
- JasonFruit 9y agoA certain amount of this boils down to, "Don't be mentally ill or have an otherwise disordered mind." I've known people who were the sort of crank he's talking about, and they were generally — though not always — unhealthily isolated people who had difficulties managing their personal lives. Most had a tenuous grasp on reality, especially the motivations of other people. There are cranks who could respond to these rules, but I believe (though I am not a mental-health professional) that many would be unable to.
- captaincrowbar 9y agoWell, I don't think the article was really aimed at actual cranks (who are, more or less by definition, immune to this kind of advice). It feels more aimed at scientists, or people of a scientific turn of mind, who may have found a genuine flaw in some widely accepted result but might not have the communication skills to get it across to the scientific community.
- zaphar 9y agoI think we can let our emotions get away from us whether we suffer from mental illness or not. Many people exhibit the behavior in the article despite being of perfectly sound mind.
- JasonFruit 9y agoCertainly true as well.
- mike_hearn 9y agoI've also investigated bad science papers before, and blogged about it a few times. There's a lot out there. There's nothing special about getting a PhD that makes you politically neutral or morally superior. The temptation to abuse maths to make your personal views seem "scientific" is irresistible for too many. My last public attempt: https://blog.plan99.net/did-russian-bots-impact-brexit-ad66f08c014a https://blog.plan99.net/did-russian-bots-impact-brexit-ad66f... I feel some sympathy with fellow Brit Nick Brown after reading this story about his work: https://www.theguardian.com/science/2014/jan/19/mathematics-of-happiness-debunked-nick-brown https://www.theguardian.com/science/2014/jan/19/mathematics-... (as an aside, totally unsurprised the guy works in computing, developing an intolerance for abuse of logic is a job hazard.) But I don't know if this perspective is helpful to be honest. "Don't be a dick" is both quite an obvious principle and also rather useless when criticising people's work: some of the authors, and some of the people who want to believe the research is true, will inevitably view any criticism as being dickish. This is doubly true if you take the obvious next step and speculate as to whether the issues are mere oversights or (far more commonly, in my view) deliberate deceptions intended to further an agenda. One problem is that genuine, honest mistakes tend to get picked up already by the existing peer review process. The really bad papers that get through are usually bad in support of some wider social mission, usually leading the public to some policy goal, and they don't get struck down because the peer reviewers share the same goals. Thus pointing out mistakes has no effect, because they already knew the science was bad. The only thing that can work is pointing out to the people who they're trying to influence that something has gone wrong. There is a responsibility to be harsh in these cases. Failing to do so can simply let the issues fester and compound. From the Guardian article on Brown: There were several psychologists, versed in non-linear dynamics, who smelt something fishy about the maths in the published paper. Stephen Guastello, from Marquette University, wrote a note of mild complaint to American Psychologist, which it chose not to publish because "there wasn't enough interest in the article". Guastello feels now that he should have been more forceful in his opinions. "In retrospect," he says, "I see how I could have been more clearly negative and less supportive of what looked like an article that could move the field forward if someone would follow up with some strong empirical work." The story describes Brown's first debunking of a paper that used lots of clever looking math to reach obviously absurd conclusions about psychology. One author of the paper admitted that she had never understood the maths and the person who did create the maths (Losada) has refused to ever respond to the debunking in any way. All the reviewers were successfully intimidated by the maths and were unwilling to criticise it, allowing Losada to get away with it: John Gottman, a leading authority in the psychology of successful relationships, wrote to Losada because he couldn't follow the equations. "I thought it was something I didn't know about, because he's a smart guy, Losada. He never answered my email," he says. So the critics weren't critical enough and a paper based on pure mathematical bullshit racked up 350 citations. Vast amounts of time, money and effort was wasted. What does this say about the scientific process? By the way, in case anyone thinks computing is immune to this sort of thing, it's not. Papers about "Russian twitter bots" are a current streak of bad science in computing, for obvious reasons: the authors want to undermine political trends they don't like. The last one I read claimed to have constructed a neural network that could detect Russian bots with 99%+ accuracy. Nowhere in the paper did it give any examples of what the network detected, nor what the ground truth they were comparing to was. When I emailed the authors to ask for the data so the study could be replicated, I was told the data was not available (despite being just a bunch of tweets i.e. public data). It was literally an entire paper that looked scientific but boiled down to "we made a magic black box that without fail finds evidence of a vast conspiracy, but we can't show you. Trust us."
- cryoshon 9y agothe authors make some good points, but i have a different perspective when it comes to criticizing science. 1. criticism is easy. penetrating criticism that demands an answer beyond the evidence already presented lest the criticized appear totally defeated is hard. it's also the only kind of criticism worth doing, imo. 2. being listened to is a matter of having penetrating criticism and offering a reasonable alternative. if you destroy, you must offer a chance to rebuild afterward, or else the criticism will be too uncomfortable to respond to, and nobody will appear to listen as a result. if you are skeptical of this point (as you should be) refer to the position of string theory within the world of particle physics; its criticism of the mainstream is very difficult to recover from if accepted, and there's no real mental way forward as a result. 3. it sucks to have your work criticized in public -- it's intolerable, even. but it sometimes gets results. it also sucks to have someone be impolite to you. but i am sure we can all remember a time that someone (perhaps a teacher or parent) has corrected us impolitely when we did not deserve courtesy. you need to retain the ability to dismiss cranks out of hand once you've suitably proven that you can beat them by making a diligent argument. anything else would lead to your time being wasted rehashing the same mistakes of others in detail. 4. scientists are, very frequently, in their professional life, jerks. i am speaking from my own experiences and my own actions. the OP doesn't exactly brush on this, but sometimes being a jerk to the right people is what makes you the right friends. you don't really want to be friends with baddies anyway. this next point is super important and directly related. 5. don't be a jerk to people because they produced incorrect work. often the theories that turn out to be totally wrong are the ones that move the field forward the most by virtue of the work put into disproving them. if they're fabricating data or doing things that are methodologically unforgivable, that's a different story. 6. scientists are a stubborn lot, until they break in the face of evidence. if you are "right" and the world is "wrong," perpetually aggregating more evidence and showing it off will eventually work, especially if you make a point to summarize your findings every once in a while. this is just as true regarding criticism as it is regarding experimental results. here is a secret: if the subfield where you are directing your criticism has only made incremental progress for a long time -- which is to say, recently there have been no major breakthroughs or destructive revolutions which reshape prior understandings -- time is on the critic's side. incremental progress isn't stagnation, but it rarely adds enough evidence about fundamental assumptions to protect those assumptions from a sustained attack. if a field is making leaps and bounds forward, criticism is going to be shrugged off because there will be new understandings which are too juicy to slow down to refine. the later refining stages are where critics are the strongest.
- keithpeter 9y agoOther would-be data whistleblowers are impressed by the duo’s success in getting journals to act. Paul Brookes, of the University of Rochester Medical Center in New York, briefly used his now-defunct blog, science-fraud.org, to highlight questionable papers anonymously. Brookes, who was outed amid legal threats in 2013 after only 6 months, says he would “routinely write dozens of emails [to journal editors], and it was common to have no response at all.” Quote from the Science article linked to in OA. I think the OA is useful in laying out a less confrontational approach to raising issues.
- psyc 9y agoI was struck by how every word of this could apply to how not to do politics and advocacy on social media. (Aside from the part where they actually advised taking all the terrible behavior to social media. It seems everyone is already doing that.)
- AlexCoventry 9y agoYou'd be surprised at how far people get simply by making it unpleasant and socially destructive to express certain views.
- lemagedurage 9y ago"And death threats (and trolling, and weird online gender-flavoured violence) are not just deeply unpleasant, they’re CRIMES. If you do this, you are breaking the law AND being a shit human being, and you should either stop or walk yourself into the sun." Death threats towards people that use death threats, really? I don't like the article. Some people are more knowledgeable on a topic than others, "if you aren't in doubt you're probably wrong" doesn't apply there. Not even in general in my estimation. A better alternative to "you have to sugarcoat all your thoughts by 30% before you speak them out" would be to train people to be better at taking criticism. People get spoiled by a high baseline positivity. I'd like to bring my points objectively without sugar coating, if everyone did that there would be no problem. Now i would have to either pretend to be more nice than i am ( talking behind people's back is a consequence of this), be straight to the point and get disliked by people who are used to sugar coating or keep silent. It usually boils down to the latter. Let me work alone if i can't be my critical self. Also, this article seems to be mostly aimed at unfair criticism, let me pose the following. If the criticism is fair and reasonable, it is justified, doesn't have to be nice, and the receiver is at fault if they dislike it. If the criticism is not fair or reasonable, it is not justified, the criticizer should express uncertainty where applicable, they are at fault as the article suggests.
- shaki-dora 9y agoThe article makes the point, quite explicitly, that you should not even call a vast conspiracy a “vast conspiracy”. Not because it’s “not PC”, but simply because people on the receiving end of such diatribes have come to associate certain terms with cranks. You can disregard such advise, and simply launch more and more over-the-top screeches against the MSM trying to silence you. But you’re not going to change any minds.
- lemagedurage 9y agoWhat about using evidence instead of manipulating your point in order to convince people?
- 9y ago
- cwyers 9y agoI don't know how much I agree with the premise of this article. People aren't resistant to retracting faulty research because the people who point out faulty research sound like cranks, they do it because their incentives are to maintain the work they've done. If you look at someone like Brian Wansink[1], what started off as unearthing methodological errors seems to have found actual fraud, although I'm sure Wansink wouldn't view it that way. And this massive peer review process not only didn't discover it, it's been an impediment to correcting it. At stake are millions of dollars in federal funding and more than a few careers. Do you think anyone involved is primarily concerned with tone? And I'm not sure how effective anyone can be at combating it by working within a crooked and broken system, where extracting rents from academic institutions for publishing their own work is the main incentive for publishers, and academics are incentivized to go along because their careers depend on numbers of publications and citations. You may have a high success rate at combating individual errors, but you aren't making any headway on the bigger issues like publication bias and the overall trend away from long, important work and towards shorted studies designed to provide an eye-catching conclusion so that you can get published and cited. It's like bailing out a boat with a drinking cup. You're still sinking. 1) https://slate.com/technology/2018/02/how-brian-wansink-forgot-the-difference-between-science-and-marketing.html https://slate.com/technology/2018/02/how-brian-wansink-forgo...
- soVeryTired 9y agoNice article. For the last few years, I've had concerns that parts of economics have serious problems. But it's good to get a reminder of how to criticise well: points one and two (calm down, and don't be a dick) are particularly relevant to me. For me, the stakes feel high enough that it's easy to get worked up and spit vitriol. But I've found that I'll lose the interest of anyone I'm talking to when I do.
- mike_hearn 9y agoYou definitely don't want to come across as worked up or vitriolic. I would note that the guy who wrote this article is sometimes a tiny bit dickish though, check this article out: https://medium.com/@jamesheathers/long-hair-dont-care-5eeba266ec52 https://medium.com/@jamesheathers/long-hair-dont-care-5eeba2... Sample comments: If that title leads you to expect off-brand evo psych exploring massive behavioural changes induced through the complicated medium of hairdressing… well, you’d be right. Mmmm. It's also worth bearing in mind your audience. This article was written by two guys who by and large are going after junk science in psychology, nutrition, and so on, where the worst motivation can only be to advance the careers of the scientists - and quite possibly the errors are just the result of over-optimism or lack of statistical abilities. For that sort of work it makes sense to try and fix psychology from within, by pointing out errors. There isn't much point in trying to spread the word through the wider world that psychology is full of errors: what's the best outcome, exactly? A few people stop being invited to give TED talks? Big deal. Psychology isn't going to stop being studied, nor should it be! But in the case of economics, there are two problems: 1) Economics is absolutely riddled with not just bad individual papers but entire sub-fields that are nonsense, it's an epidemic. It probably cannot be fixed by shaming individuals who make simple errors via their own journals and hoping for the field to improve. Even when major economists like Paul Krugman or Paul Romer have gone on very serious attacks against their own field, there has been no visible impact and in Romer's case he simply lost his job. 2) Economics is routinely used to justify massive government interventions (or lack of interventions) in markets and society. Bad economics research can have a very serious impact on people's lives. In this case, it's probably better to work at a different level and try to spread the word amongst the general public that economics is not trustworthy. Thus the audience is not journals or the authors of the papers themselves. It's the public. This is not hard because most people who follow the news have already encountered major cases where consensus economic predictions were simply wrong. Polling shows that trust in economists is quite low as is. If and when the field reforms it can try to rebuild trust.
- sporkologist 9y agoCranks generally don't think they are cranks....
- cgiles 9y agoI would use Andrew Gelman and John Ioannidis as prototypes for the Good Critic and Bad Critic, respectively. Gelman does real, solid work and confines his righteous takedowns to a side hobby. And his criticisms are directed at individual, specific cases, with evidence, and he reserves most of his wrath for repeat offenders rather than one-off mistakes, which are unavoidable given enough projects. He confines criticisms to his area of expertise, which is statistics. Ioannidis' claim to fame is writing a hand-waving, philosophical argument in order to cast doubt on all research at once. Followed up shortly by a paper that implied (but didn't directly state!) that regression to the mean on replication implies error or fabrication in the original. And he has ridden this sort of tired argument into a lucrative and prestigious career and Stanford appointment. Never mind the collateral damage on funding and morale. My theory is that people pay attention to Ioannidis and wring their hands about these highly generic issues for the same reason most people pretend to be highly, highly concerned about sexual harassment/discrimination in the workplace -- they're praying that if they act concerned enough, the hammer will fall somewhere else. But indiscriminate criticism like this is highly toxic to science (or anywhere else, for that matter, and has the side effect of letting real bad actors off the hook). A further moral of the story is that, for better or worse, I don't trust and generally ignore criticisms that do not come from someone who has invested time into a field and made useful contributions to it. 'Full-time critics' strike me as generally sleazy and opportunistic, and I doubt the sincerity of their desire to 'improve' a field if all they can do is tear down others' work and not provide any of their own.
- curuinor 9y agoThe fact that Ioannidis stakes his claim to fame as attacks on medical research is no evidence that he is a crank: it's evidence that medical research is a target-rich environment.
- cgiles 9y agoNo doubt it is a target-rich environment. However, his arguments are so generic that they could apply to any field studying anything multifactorial or with high variance. Fundamentally, however, I don't believe, for example, that physicists are more scrupulous/intelligent/honest than biologists, who are more so than psychologists. The irreproducibility rates in a field have to be primarily caused by the nature of the system studied in that field rather than some inherent attributes of the researchers. So any field will have some constant level of irreproducibility which which essentially cannot be changed. But some researchers do better than others. My fundamental point is that if you want to change what can be changed, and genuinely improve a field, you have to focus the criticisms on specific bad actors rather than target an entire field at once and leverage a near-tautology (more complex systems yield more irreproducible results) into a 20-year tirade/career. Also, helpfully for people like Ioannidis, a statement like "field X has lots of errors and irreproducibility" is non-falsifiable, and indeed will be true no matter what, so he is taking no risk with his criticisms -- even though they do more damage to public perception, morale, and funding, and help less to change things than targeted criticisms would do. If I sound irritated by this sort of thing, it is because my graduate advisor dabbled in "meta-science" / "error correction" and I stoutly refused to have anything to do with those projects. I thought, and still think, that it is considerably more valuable to try to build my own solid lego tower despite the difficulties rather than knocking down other people's, despite the latter being considerably easier and a fairly simple and reliable route to a high citation count. In the end, irreproducible or erroneous papers in science just get ignored rather than retracted, and I think that's fine.