9 ms·

> and whether online critiques of past work constitute “bullying” or shaming. The PubPeer comments are just a tiny part of that debate. But it can hit a nerve.

• dmd 10y ago
  I took a class with Susan Fiske in grad school, and the following is a quote from one of her comments on a homework where I had called into question a paper that was clearly not just statistically incorrectly done but was based on urban-legend-level assumptions: "These results are from respected investigators, so it's inappropriate to question them."
  • ben_jones 10y ago
    While I trust your story, and I believe it's somewhat taboo on HN to ask for this, I feel that it would be much more valuable if evidence was provided. That is an absolutely egregious act for a professor to commit and I feel like it should be reported to someone if substantiated..
    • dmd 10y ago
      Oh, I made a huge stink about it (and this was just one example of many comments of that type). This was at Penn in 2007.
      • DiabloD3 10y ago
        An upvote for outing her for clearly bad science and bad teaching, if she actually did this. There are few things that offend me, a teacher damaging impressionable students like this is one of those.
        • jacquesm 10y ago
          I think she mostly damaged herself there, the student seems to have come out all right (but there was a risk they wouldn't have).
      • tgb 10y ago
        Not to question you, but I can't find any evidence that Fiske was ever teaching at Penn. Her CV [1] is quite exhaustive, but only lists giving colloquia at Penn. [1] http://static1.1.sqspcdn.com/static/f/1605966/27061956/1465245429350/FISKE+VITA2016.pdf?token=g9VFTcMNi1O15oD6BnFsF27s9PI%3D http://static1.1.sqspcdn.com/static/f/1605966/27061956/14652...
        • dmd 10y ago
          She was one of the people giving the Social Psychology prosem in Fall 2005, because John Sabini (who normally gave it) died: > September 21: The Self (Prentice) > September 28: Social Cognition (Fiske) > October 5: Attitudes (Cooper) > October 19: Social Inference (Pronin) > November 2: Prejudice & Stigma (Shelton) > November 16: Dissonance (Cooper) > November 30: Close Relationships (Fiske) > December 7: Social Influence (Prentice)
          • tgb 10y ago
            Thanks.
        • Lambdanaut 10y ago
          > Not to question you It's quite okay. That's exactly the kind of attitude that's being scrutinized here ;)
  • agumonkey 10y ago
    I've seen churches with more skepticism.
  • deleted 10y ago
    [deleted]
  • Paul-ish 10y ago
    Is that verbatim?
    • dmd 10y ago
      Yes.
      • harry8 10y ago
        Please photograph and publish it. (Pics or it didn't happen?) This debate is important and shouldn't be muddied with hearsay on all sides. Thanks.
        • monochromatic 10y ago
          dmd is a respected HNer, so it's inappropriate to question him.
          • lifthrasiir 10y ago
            It is inappropriate to use that fact to dismiss a valid request for the evidence. Being famous in some (possibly popular) website does not mean that one's experience has not been biased.
            • cfqycwz 10y ago
              Go five levels up from your comment and you might find a joke you're missing—unless you're in on it too and it's sailing right over my head
              • lifthrasiir 10y ago
                Oh-kay, now I understood what happened (the fifth level is invisible from the parent comment, ugh). Thank you for pointing out where I screwed up :S
            • deleted 10y ago
              [deleted]
  • jessaustin 10y ago
    Haha this reminds me of the feedback I got on a report in Comparative Religion, when I questioned some French dude's really elaborate and fanciful interpretation of the paintings at Lascaux. "Ummm... what supporting texts do we have from 17,000 years ago?" "No, you can't ask that sort of question and it's really disrespectful." FWIW, I wasn't really upset by that feedback. Senior spring; I don't care!
    • V-2 10y ago
      It immediately reminded me of "Emperor Kennedy Legend: A New Anthropological Debate" by Leszek Kołakowski : ) Highly recommended and very funny snce there's much truth to it I think. I found it online in English: http://jennis22.weebly.com/uploads/3/0/5/3/30531242/emperor_kennedy_legend.docx http://jennis22.weebly.com/uploads/3/0/5/3/30531242/emperor_...
  • law_of_the_hobo 10y ago
    Out of curiosity, do you remember what paper it was?
    • dmd 10y ago
      I'm not entirely certain but I think it was The modern synthesis of contrasting styles of thought in “east” and “west” *Nisbett, R. E., Masuda, T. (2003). Culture and point of view. Proceedings of the National Academy of Sciences, 100, September 16, 2003, 11163-11170
      • Cd00d 10y ago
        Interesting. As a physicist, I tend to put extra trust in PNAS credibility due to their history of critical editors and reviewers, and therefore am less likely to question results published there. Maybe it's the same fallacy, but at the same time we can't all be experts at everything and have to defer to trust at some point. Anyway, thanks for the contextual tidbit.
        • Obi_Juan_Kenobi 10y ago
          PNAS is notorious for inconsistent quality, and has recently had to modify their direct submission process, partly as a result. I don't know about physics, but I've seen them publish a lot of very poor work.
• csallen 10y ago
  You left out this part: > The gist of her article was that she feared too much of the criticism of past work had been ceded to social media, and that the criticism there is taking on an uncivil tone. So it seems to me, her complaint was not that research shouldn't be criticized. It was more-so that research should be criticized well, and that social media is a poor place for it, because it favors personal attacks over thoughtful criticism. (I'm not saying I agree with her. Just trying to understand her argument in the best light.)
  • emodendroket 10y ago
    Personal attacks in academia aren't really unheard of.
    • apathy 10y ago
      neither are bullshit papers/editorials/letters-to-the-editor http://frog.gatech.edu/Pubs/How-to-Publish-a-Scientific-Comment-in-123-Easy-Steps.pdf http://frog.gatech.edu/Pubs/How-to-Publish-a-Scientific-Comm... "Eminent" specialists often argue in favor of eminence (journals, tenured investigators, blah blah) over evidence (tossing an analysis up on arXiv and posting a link to it). Young turks or iconoclasts often do the reverse, since there is a power structure in place via editors, reviewers, study sections, and the like. Orthodoxy often sticks even when the evidence supporting it is quite skimpy (or absent). Funny thing is, when you pull the underlying data (in fields where this is possible), you routinely find that the reported conclusions are overblown. Not necessarily wrong, but routinely sold as more conclusive than they are (and sometimes they are in fact wrong, whether due to small sample sizes, "outlier removal", or outright fraud). In no way, shape, or form are these bad habits limited to psychology. They happen in basic biology, they happen in medicine, they happen in clinical trial reporting. It's faster and easier to oversell shitty science than it is to do good, thorough science, so with the tight competition for grant funding, you can imagine what happens next. Trust whoever you like, but verify results from everyone. There are good people out there who are fastidious; there are good people out there who are sloppy; and there are people who don't care one bit whether they're publishing absolute horse shit. The onus is on you, the reader or researcher, to do the requisite critical thinking (and, perhaps, a few analyses before you waste time running down a dead end). Keep an open mind, but don't let your brain fall out. Also, many good scientists aren't very nice people, and some very pleasant people in science are shitty scientists. It's very hard to tell a priori which is which, so look at the evidence and decide for yourself.
• kevinwang 10y ago
  The last quote, which is the most extreme response, seems like it's not entirely within the same context as the article. I got that it was talking about _human_ online critics, and especially (only?) those who have an "uncivil tone", which is something I agree with.
  • Mithaldu 10y ago
    Yeah, that's the key detail here. The article went kinda off-topic on itself, by throwing in quotes about a completely unrelated incident, which on skimming may be read as referring to statcheck, despite having nothing whatsoever to do with it.
• thaw13579 10y ago
  > That anyone in the field of psychology would even be comfortable publically espousing a non-scientific worldview like that means that psychologists are not being properly educated in the scientific method. This is a generalization about psychologists from a sample size of one psychologist, to suggest that they don't understand science?
  • c3534l 10y ago
    Hey, I did say "if" at the start of my little rant. You're right that I don't know how embedded this attitude is in psychology, but I am often dispirited by the attitude towards verification these days based on a number of different things I've read.
    • thaw13579 10y ago
      It can be dispiriting, but I think there is also a lot of good science being done that is too boring to hit the radar.
  • deleted 10y ago
    [deleted]
  • wfo 10y ago
    One ex-president of the psychologists professional association, a person who was literally deliberately selected by the community of psychologists to represent them. It would similarly be completely inappropriate to generalize the policy positions of Hillary Clinton to the Democratic party, yes?
    • deleted 10y ago
      [deleted]
    • thaw13579 10y ago
      Saying that psychologists don't understand the scientific method is similar to a saying that democrats don't understand something as basic as the separation of powers in government
• SilasX 10y ago
  Yes! Similarly, I remember a few years back being shocked at the attitude toward the replication crisis, where authors of irreproducible work were miffed that no one talked to them first. I was like, "Um, the whole point of science is that it's knowledge in the public record. If you need to 'chill' with the original researcher to get critical details, that's not science anymore." Remember, the studies might form the basis of an FDA approval. If only the original lab can produce the result, then is the FDA approval of the treatment going to be limited to only that lab as well? Kinda limits scalability... I did a search and found the blog post that quoted the researcher [1] with this scary attitude: >When researchers at Amgen, a pharmaceutical company in Thousand Oaks, California, failed to replicate many important studies in preclinical cancer research, they tried to contact the authors and exchange materials. They could confirm only 11% of the papers. I think that if more biotech companies had the patience to send someone to the original labs, perhaps the percentage of reproducibility would be much higher. [1] http://andrewgelman.com/2013/12/17/replication-backlash/ http://andrewgelman.com/2013/12/17/replication-backlash/ (thanks Google)
  • nicops 10y ago
    > If you need to a 'chill' with the original researcher to get critical details, that's not science anymore. Certainly there is always the possibility that there was some details misunderstood, something that needs to be clarified, a print error, etc. Your "that's not science anymore" statement seems highly exaggerated. People are not supposed to communicate only via papers.
    • nitrogen 10y ago
      The usefulness of a paper that doesn't stand on its own is rather limited, though.
      • pbhjpbhj 10y ago
        All papers should start with a dictionary? No clearly not, so there's always going to be some assumed knowledge - words change their meaning and have different meaning to different people so we're already on to a loser just with the medium we're using. So, the possibility of things like, say, a researcher not mentioning something that is standard practice in their lab that later is found to be a crucial part of the setup for an experiment seems high. But just like you don't want to provide a dictionary of standard terms with a paper you don't want to provide a list of the chemicals used to mop the floor, or a list of the lumen and colour temperature ratings of lights in the fume cupboards, or ... IMO if a paper is not reproducible then yes it should be published but also the original team producing the paper should be challenged to reproduce the results. It's not a fight, we're all on the same team - work with them and try to find the reason for the lack of reproducibility.
• dahart 10y ago
  > If these are attitudes typical of psychology, then I cannot say I consider psychology to be a proper social science. It's quite unscientific to draw a blanket conclusion like that about a field based on the anecdotal evidence of one named person and some unnamed "critics". Let's get some statistics before casting doubt on the entire field of psychology. I think most people are trying to get by and to be good. My personal experience in academics is that the vast, vast majority of students and teachers just want to study their art or science and contribute to their community without having to spin or sell to get funding. Some of the people that you hear about in articles get attention because they like attention, and they will happily say opinionated, contentious and unscientific things because it gets talked about. There are bad actors with high levels self interest in all fields of study and work, so really the only scientific conclusion you can come to is that not all humans are perfect.
  • ignoramceisblis 10y ago
    > If these are attitudes typical of psychology, then I cannot say I consider psychology to be a proper social science. is a proposition. He/she's not concluding psychology is not a proper social science; he/she's stating > If these are attitudes typical of psychology, then I cannot say I consider psychology to be a proper social science.
  • wfo 10y ago
    Sure "but not ALL psychologists!" is valid here (as it is in literally any discussion about anything online or elsewhere) but she is not just one person. She is the former head of a professional organization for psychologists; if we look for someone to represent them as a whole, she is the first place we should look. Would the professional organization for physics have as their head someone who believes we had it right when we said the earth was flat and the expertise of these scientists shouldn't be questioned? How many psychologists have/will publicly denounced her? HN is ready to attack every police officer who hides and defends the actions of a few bad apples as bad apples themselves (rightly so), where's the equal treatment for psuedoscientists?
    • dragonwriter 10y ago
• disgruntledphd2 10y ago
  You are entirely correct (p< 0.05). To be fair, psychology is a broad church. Within it, we have people like Jacob Cohen, Daniel Kahneman and Tversky (mathematical psychologists awarded a Nobel in Economics). But apparently we also have Susan Fiske. I think that the real difficulty with psychology is that people care too much. Its a little easier to be objective around the decay of atomic particles than about the intelligence of different genders and races. Additionally, many psychologists are not comfortable with statistics. A large proportion went into psychology to avoid maths, and are somewhat disappointed to discover that this does not work. So they follow guidelines, and paste tables from SPSS into MS Word (no automation for some reason), and stuff like this happens. They don't use cross-validation and they are too much in love with their models, and the theory that said models facilitate. I suppose this was a pretty long-winded way to say "Not all psychologists!" Cohen, J: http://www.ics.uci.edu/~sternh/courses/210/cohen94_pval.pdf http://www.ics.uci.edu/~sternh/courses/210/cohen94_pval.pdf (The World is Round (p< 0.05))
  • zeveb 10y ago
    > I think that the real difficulty with psychology is that people care too much. Yup. There are powerful social forces which act to dissuade psychologists from publishing research which opposes to feelings of their peer group. One can't speak an inconvenient truth.
  • jessaustin 10y ago
    An interesting recent complaint about Fiske doesn't entirely spare Kahneman: http://andrewgelman.com/2016/09/21/what-has-happened-down-here-is-the-winds-have-changed/ http://andrewgelman.com/2016/09/21/what-has-happened-down-he...
• lutusp 10y ago
  > If these are attitudes typical of psychology, then I cannot say I consider psychology to be a proper social science. In my experience, such attitudes are typical. Here's a 2014 paper by a Harvard psychology professor who tries to argue that null replication results shouldn't be taken seriously because the failure to replicate ipso facto indicts the study's methods: http://jasonmitchell.fas.harvard.edu/Papers/Mitchell_failed_science_2014.pdf http://jasonmitchell.fas.harvard.edu/Papers/Mitchell_failed_... Quote: "Because experiments can be undermined by a vast number of practical mistakes, the likeliest explanation for any failed replication will always be that the replicator bungled something along the way ... The field of social psychology can be improved, but not by the publication of negative findings."
  • taneq 10y ago
    Wow, so many responses that come to mind and they're all of the form 'un' + some swearing + 'scientific'.
  • ramchip 10y ago
    > Someone who publishes a replication is, in effect, saying something like,“You found an effect. I did not. One of us is the inferior scientist.” Wow... I really don't see it like that.
    • sqeaky 10y ago
      I agree with you. When Two scientists find different results only inferior scientists would take it as a personal attack. There was a story about two chemists, I might have some minor details wrong, but that shouldn't hurt this anecdote because something like this happened back when chemistry was still raw and new. The going wisdom was based on the assumption that the atoms in a molecule were the only thing that determined its properties. These two scientists were having trouble replicating each others results when working with a molecule of 2 Carbon, 2 Nitrogen, 2 Oxygen and 1 Mercury atoms. One describe it as inert and the other as explosive. Neither bothered to include synthesis steps in their papers and each thought the other was wrong. Rather than insult they swapped more and more information until eventually each confirmed that we dealing with a molecule with identical parts, but different properties. They had discovered that molecules have structure. One arrangement was Mercury Fulminate an unstable explosive used in primers and detonators today and the other was really inert and not of interest, to me at least. Any scientist who argues with the concept of "disagreement based on evidence" is arguing a foundation of science. They should be looking for evidence of reasons for that disagreement instead of quibbling over personal attacks and defense.
• cylinder 10y ago
  There go Americans throwing around the terrorism word again. https://news.ycombinator.com/item?id=12620132 https://news.ycombinator.com/item?id=12620132
• jomamaxx 10y ago
  "a proper social science" Is 'social science' even a 'proper science' ?
  • apathy 10y ago
    In general, if you have to put science in the name, it isn't.
    • OJFord 10y ago
      Computer Science?
      • apathy 10y ago
        That's long for math
        • OJFord 10y ago
          Which definitely is science; QED.
          • apathy 10y ago
            Proofs are math, science is uncertain (otherwise why bother running experiments)
            • OJFord 10y ago
              No, science is - literally - knowledge.
              • dragonwriter 10y ago
                > No, science is - literally - knowledge. No, science is a means of generating (and justifying) beliefs, it is not knowledge itself.
• empath75 10y ago
  Anybody who objects to such an anodyne project that strenuously with such disingenuous arguments immediately makes me think they're hiding something.
• deleted 10y ago
  [deleted]
• jmde 10y ago
  As someone in this field somewhat close to the article in a certain sense, I wouldn't say Susan Fiske's attitudes are typical of psychology. Maybe not rare, but not typical either. Her comments have been very controversial to say the least, and many are disturbed by them. Moreover, if you level that statement against psychology, I'm afraid you have to level it against medicine as well, given that there have been similar sentiments expressed in major medical journals as well recently. Also, without meaning to defend Fiske, her comments here are taken out of context somewhat. Her reference to "methodological terrorism" (from what I have heard through the grapevine) is more, or at least in part, about the trend toward having scientific debates outside of the peer review process, in social media. So my guess is that she might say that part of what she objects to about Statcheck is that it crawls through the papers, labels an error, and then we end up discussing it on HN rather than through peer review. What if Statcheck made an error, which it does sometimes? I don't agree with it, but I think the position I'm describing (which I think is her point in part) isn't unreasonable either. That is, it's not the checking of stats, it's the chaos and disintegration of the peer review system, and "extrascientific" discourse that's happening in science today, if you define "extrascientific" as "outside of peer reviewed journals," where your critics are attacking you on twitter, forums, and facebook, more so than in professional published outlets in a sort of mob. Again, I do not share her perspective at all (I'm in favor of a shift away from journals) but I do think here her original point was twisted a bit.
• M_Grey 10y ago
  If psychologists were educated properly to be scientists, they'd abandon psychology, which is at best an art.
  • danso 10y ago
    I agree with Dr. Srivastava that psychology is not only a science but the hardest science: https://hardsci.wordpress.com/2009/03/14/making-progress-in-the-hardest-science/ https://hardsci.wordpress.com/2009/03/14/making-progress-in-... > What are the “hard” — as in difficult — problems in science? Hard problems in science are those that are embedded in complex systems; they are hard because to study something well you often need to isolate it from outside influences. Hard problems are those that vary by local conditions — science seeks to identify general laws, and when something is locally dependent, you need to sniff out the complex interactions that make it so. And hard problems are those that are difficult to quantify — science rests upon formalization and quantification, and you need to get traction at that initial step of quantification (i.e., measurement) before you can test theories. So… by these measures, if we are going to differentiate areas of science, the continuum of scientific problems should go from “hard” to “easy,” and psychology is clearly a science that deals with hard problems. Perhaps the hardest. Reminds me of the counterintuitive about the two hard problems in computer science. Naming things, and deciding when data is obsolete, are easy mechanically. But because they have neither a deterministic nor correct solution, they end up being the hard problems you face as you build bigger projects. Or, to emphasize a line from Dr. Srivastava above: > And hard problems are those that are difficult to quantify — science rests upon formalization and quantification, and you need to get traction at that initial step of quantification (i.e., measurement) before you can test theories. It's possible for a science to be "hard" while having a lot of mediocre practitioners.
    • LunaSea 10y ago
      That quote still fails to show why psychology should be considered a science. It only speaks about the difficulty of problems in science.
    • M_Grey 10y ago
      In what way does any of that support your/Srivastava's assertion?
• Bartweiss 10y ago
  Calling Statcheck a "gotcha algorithm" descends into self-parody. It reads like outrage at the idea that anyone might expect you to be accurate about facts.
• summarite 10y ago
  Don't judge a science by looking at just a few of the people in it... Also to say i think it's a graded thing. Often such type of critique might be serious and well developed, often it might be serious but based on misunderstanding, and sometimes it might be an attempt at trolling, annoying or payback. I would guess your average peer reviewer engages more deeply with the paper than most readers and takes time to check their critique before sending it, and since there's an editor involved there might also be a better level of politeness. If now suddenly every reader is free to criticise (assuming they fall in category 1 & 2) that can end up in a barrage of critiques that an author might feel obliged to respond to in order to protect his/her reputation. And still those critiques can then be taken out of context, eg when someone in the media or from the competition tries to put you down a notch for holding the wrong views... Think of anti-vaxers going around and criticising every paper on vaccines. You're always going to find something to criticise and once it's criticised you can point to SG authoritative 'the paper is criticised by other scientists for..' Or another angle: do you want to feel obligated to defend every one of your papers for the rest of your life? This can really feel like bullying even if it's not intended as such. That's in particular if you write about contentious issues or such with strong interest groups - think vaccines, abortion, gender, GMO, lobbying, financial regulation, inequality, race, ... I think open 'peer review/critique' is a good idea, but it can take dark turns if there's no mechanism to prevent this.