6 ms·

My big eye-openers (some from postdoc) were more about the sociology of science than the day-to-day productivity: - Even the most blatantly wrong and illogical

by xab31 5y ago

My big eye-openers (some from postdoc) were more about the sociology of science than the day-to-day productivity: - Even the most blatantly wrong and illogical published work can only be displaced by another publication that explains/does the same phenomenon better; i.e., people are going to keep believing in phlogiston until someone shows them oxygen. If you simply point out inconsistencies in phlogiston theory, in person or in writing, they may well make a variety of unwanted psychological deductions about you. - Similarly, nobody actually enjoys being around critics or enduring criticism, and therefore you will observe many senior scientists partially avoiding the major downsides of being a critic by artfully concealing criticisms inside what sounds to the uninitiated like mutual affirmation sessions. You have to listen very closely and learn the lingo to pick this up. - Never question a scientific superior (other than maybe a direct mentor or very close colleague) with any other approach besides "I have a helpful suggestion about how you can maybe reach your intended destination better/faster/more precisely". Regardless of where that destination might be, such as off a cliff or into a wall. - The opinion/fact ratio you are allowed to have as a scientist is directly and very strongly correlated with seniority, H-index, and so on. - The incentive structure of scientific publication is such that there are big rewards for being right on an important question, bigger the earlier you are to the party, and little to no penalties for being wrong, so long as the error cannot be provably and directly linked to fraud. There are a variety of interesting consequences to this incentive structure.

kafkaIncarnate 5y ago
This is true here, as well. I once asked about something related to SSL/TLS (fairly politely) and was kind of mockingly escorted by some groupies to the corner since I apparently responded to an Apache developer. I was just trying to learn. Learning bad is what I learned.
zebraflask 5y ago
That seems to ring true, but - it also seems a bit defeatist, don't you think? "Never question a scientific superior?" Not parsing that concept, please elaborate.
- selimthegrim 5y ago
  >"Never question a scientific superior?" Not parsing that concept, please elaborate. If you think you've been put on a bum topic or your supervisor has put you on the scientific equivalent of a PIP with no way up or out your room for maneuvering is limited, to put it politely.
- natechols 5y ago
  I agree with most of the GP's points and I don't think of them as defeatist, but rather a call for realism when dealing with people (versus data, which have no ego to bruise). It's very hard to devise a system that rewards individual achievement without ever falling prey to classic human flaws. The good news is that science over time tends to be self-correcting, and all that requires is a commitment to shared principles and methods, combined with enough anarchy that no one individual can screw up an entire field (Trofim Lysenko being the most extreme example, but any bureaucracy can accomplish this).
- wbsss4412 5y ago
  I understand the impulse to not want to be defeatist, but sometimes it’s both easier and more productive to stop running into the same walls over and over and instead find the path around them.
- xab31 5y ago
  Very early on, I noticed that graduate students tend to be idealistic, postdocs extremely cynical, and faculty ruthlessly pragmatic perhaps to the point of occasional shortsightedness. Clearly, something about this progression is expected and normal. I'm a postdoc now, so I'm right on schedule. I think the way it ultimately works is that you have to be disillusioned from the grade-school fairy tales told to the public about how science works before you can learn to live and work in the environment that actually exists rather than the one you wish existed. > "Never question a scientific superior?" Not parsing that concept, please elaborate. tech < grad student < postdoc < junior faculty < full prof < Big Guy/Gal < Nobel Laureate < NIH Director People above you in that chain will accept limited feedback on methods to attain their chosen goals and will greatly resent questions about whether their selected goals are worthwhile/realistic/rational, or whether their gestalt vision of the field's conventional wisdom is correct.
ta988 5y ago
This reflects my experience as well.
- Ostrogodsky 5y ago
  Young people need to realize that the things we love about science: the uncompromising search for the truth, its international and no-boundaries character, the ability to bow down to evidence, the ambition of the ideas, are just a very distilled fraction (basically the highlights) of a what it is a very mundane, fragile, political human activity, full of petty and lame characters, absurd situations and pathetic developments.
  - jjdredd 5y ago
    > full of petty and lame characters, absurd situations and pathetic developments. Yeh, I wish I could understand this a few years ago. Pursuit of truth is not the primary goal of many tenure profs it seems.
  - DarylZero 5y ago
    Well there's science and then there are social institutions that undertake science. But science can exist outside the institutions.
deleted 5y ago
[deleted]
dhd415 5y ago
In addition to ringing true, this seems largely in line with Thomas Kuhn's thesis in his Structure of Scientific Revolutions [0], a book which despite its shortcomings, should be required reading for anyone in a STEM field. [0] https://en.wikipedia.org/wiki/The_Structure_of_Scientific_Revolutions https://en.wikipedia.org/wiki/The_Structure_of_Scientific_Re...
- bangkoksbest 5y ago
  Kuhn's thesis doesn't have a lot space for the sociology of science to have this kind of influence. Of the classical theses of scientific progress, this comes closer to Lakatos' thesis of research programs.[0][1] [0] https://en.wikipedia.org/wiki/Research_program https://en.wikipedia.org/wiki/Research_program [1] Lakatos, Imre. (1978) The Methodology of Scientific Research Programmes: Philosophical Papers (J. Worrall & G. Currie, Eds.). Cambridge University Press.
keeptrying 5y ago
Beautifully put. Your observations aptly apply to industry as well except for your final one regarding the incentive structure. Thank you for commenting.
jrumbut 5y ago
> The incentive structure of scientific publication is such that there are big rewards for being right on an important question, bigger the earlier you are to the party, and little to no penalties for being wrong, so long as the error cannot be provably and directly linked to fraud. This is fantastic insight and I'd like to thank you for sharing it with our group. Would you agree that the model of rewards for correctness and penalization only in the case of fraud is the core feature of science? And what separates it from business or politics where being an honest failure is worse than being dishonest but successful? Again, this is a great post, and I think you have a fantastic future in the sociology of science!
- myle 5y ago
  Are you trying to provide an example for many of the points of OP above or am I overreading this? In your message I observe, very careful criticism, uncalled praise, admission and defense of a system that excludes most criticism...
  - jrumbut 5y ago
    I'm joking a bit with the style and my limited experience leads me to agree with the OP. The middle paragraph includes my sincere response that a system where discovery is rewarded, failure forgiven, and dishonesty punished is ideally suited to the mission of science. So I was left wondering if the OP would expand on what their thoughts were about the interesting consequences.
- xab31 5y ago
  It's a good question, and I don't like posting excessively long comments and didn't have time to make it concise, so here's an attempt at an answer: https://pastebin.com/fsrTtiKY https://pastebin.com/fsrTtiKY I think science is too big a thing to have a small set of "core features", and the question of how to usefully define "honesty" in a scientific context is another big topic, but reading about "bullshit" (the term of art that has its own literature, not the colloquialism) is a good place to start thinking about it. I would suggest that fraud is one of the rarest types of dishonesty, because people who are both smart and dishonest have less risky ways to proceed, and that such people are very glad fraud exists, because it misdirects attention away from their arguably more damaging and prevalent methods. Feynman has a passage about how honesty in science is more a state of mind, which I agree with. But really, the techniques to be dishonest with low risk are the same in science, journalism, politics, and business. My field isn't sociology of science though; these are just views from the genomics trenches.
lambdatronics 5y ago
The sociology of science is so interesting. (Not the field, but the subject.) Here are some of my favorites quotes/thoughts: This one is a direct contrast to your advice (which speaks volumes about what's wrong with academia): "A good scientist, in other words, does not merely ignore conventional wisdom, but makes a special effort to break it. Scientists go looking for trouble."[0] This was written about physics at Caltech, but applies more broadly. It explains why the ability to 'manage up' is so critical for early-career success. "[...] departments are run, for better and worse, by the professors who often lack managerial experience. Worse, they are generally unaware of this shortcoming, assuming incorrectly that management is trivially easy compared to their topics of study and merits minimal effort. We have now seen the consequences of this lack of attention." [1] Academic politics is a great reason not to stay in academia: "Look for environments where competitors see themselves as playing a game, rather than fighting for survival — this prevents rankings within the hierarchy from becoming an existential problem." [2] This book has a great chapter of career advice, here's a gem: "Don't build a pyramid. Everyone seems to build one pyramid per career. A pyramid is an ambitious system that one person really cares about and that winds up working well, but then just sits in the desert because nobody else cares the same way. This happens usually just after leaving graduate school." [3] "In general, status-conscious places are miserable for everyone, and the more, the worse." [3, next page] Gatekeeping is predictable from the incentive structure: "For all the high-level talk about how we need to plug the leaks in our STEM education pipeline, not only are we not plugging the holes, we're proud of how fast the pipeline is leaking." [4] "So why am I not an academic? There are many factors, and starting Tarsnap is certainly one; but most of them can be summarized as 'academia is a lousy place to do novel research'." [5] "...whereas Newton could say, 'If I have seen a little farther than others, it is because I have stood on the shoulders of giants,' I am forced to say, 'Today we stand on each other's feet.'" [6] [0] http://www.paulgraham.com/say.html http://www.paulgraham.com/say.html [1] https://caseyhandmer.wordpress.com/2019/08/09/caltech-astrophysics-and-harassment-lessons-learned/ https://caseyhandmer.wordpress.com/2019/08/09/caltech-astrop... [2] https://www.briantimar.com/notes/mimetic/mimetic/ https://www.briantimar.com/notes/mimetic/mimetic/ [3] Phillip Hobbs, "Building Electrooptical systems: making it all work" 2nd Ed, p392 [4] https://danluu.com/teach-debugging/ https://danluu.com/teach-debugging/ [5] http://www.daemonology.net/blog/2020-09-20-On-the-use-of-a-life.html http://www.daemonology.net/blog/2020-09-20-On-the-use-of-a-l... [6] Richard Hamming 1968 Turing Award lecture, Journal of the ACM 16 (1), January 1969, p. 3–12

by xab31 5y ago

kafkaIncarnate 5y ago

This is true here, as well. I once asked about something related to SSL/TLS (fairly politely) and was kind of mockingly escorted by some groupies to the corner since I apparently responded to an Apache developer. I was just trying to learn. Learning bad is what I learned.

zebraflask 5y ago

That seems to ring true, but - it also seems a bit defeatist, don't you think? "Never question a scientific superior?" Not parsing that concept, please elaborate.

selimthegrim 5y ago
>"Never question a scientific superior?" Not parsing that concept, please elaborate. If you think you've been put on a bum topic or your supervisor has put you on the scientific equivalent of a PIP with no way up or out your room for maneuvering is limited, to put it politely.
natechols 5y ago
I agree with most of the GP's points and I don't think of them as defeatist, but rather a call for realism when dealing with people (versus data, which have no ego to bruise). It's very hard to devise a system that rewards individual achievement without ever falling prey to classic human flaws. The good news is that science over time tends to be self-correcting, and all that requires is a commitment to shared principles and methods, combined with enough anarchy that no one individual can screw up an entire field (Trofim Lysenko being the most extreme example, but any bureaucracy can accomplish this).
wbsss4412 5y ago
I understand the impulse to not want to be defeatist, but sometimes it’s both easier and more productive to stop running into the same walls over and over and instead find the path around them.
xab31 5y ago
Very early on, I noticed that graduate students tend to be idealistic, postdocs extremely cynical, and faculty ruthlessly pragmatic perhaps to the point of occasional shortsightedness. Clearly, something about this progression is expected and normal. I'm a postdoc now, so I'm right on schedule. I think the way it ultimately works is that you have to be disillusioned from the grade-school fairy tales told to the public about how science works before you can learn to live and work in the environment that actually exists rather than the one you wish existed. > "Never question a scientific superior?" Not parsing that concept, please elaborate. tech < grad student < postdoc < junior faculty < full prof < Big Guy/Gal < Nobel Laureate < NIH Director People above you in that chain will accept limited feedback on methods to attain their chosen goals and will greatly resent questions about whether their selected goals are worthwhile/realistic/rational, or whether their gestalt vision of the field's conventional wisdom is correct.

ta988 5y ago

This reflects my experience as well.

Ostrogodsky 5y ago
Young people need to realize that the things we love about science: the uncompromising search for the truth, its international and no-boundaries character, the ability to bow down to evidence, the ambition of the ideas, are just a very distilled fraction (basically the highlights) of a what it is a very mundane, fragile, political human activity, full of petty and lame characters, absurd situations and pathetic developments.
- jjdredd 5y ago
  > full of petty and lame characters, absurd situations and pathetic developments. Yeh, I wish I could understand this a few years ago. Pursuit of truth is not the primary goal of many tenure profs it seems.
- DarylZero 5y ago
  Well there's science and then there are social institutions that undertake science. But science can exist outside the institutions.

deleted 5y ago

[deleted]

dhd415 5y ago

In addition to ringing true, this seems largely in line with Thomas Kuhn's thesis in his Structure of Scientific Revolutions [0], a book which despite its shortcomings, should be required reading for anyone in a STEM field. [0] https://en.wikipedia.org/wiki/The_Structure_of_Scientific_Revolutions https://en.wikipedia.org/wiki/The_Structure_of_Scientific_Re...

bangkoksbest 5y ago
Kuhn's thesis doesn't have a lot space for the sociology of science to have this kind of influence. Of the classical theses of scientific progress, this comes closer to Lakatos' thesis of research programs.[0][1] [0] https://en.wikipedia.org/wiki/Research_program https://en.wikipedia.org/wiki/Research_program [1] Lakatos, Imre. (1978) The Methodology of Scientific Research Programmes: Philosophical Papers (J. Worrall & G. Currie, Eds.). Cambridge University Press.

keeptrying 5y ago

Beautifully put. Your observations aptly apply to industry as well except for your final one regarding the incentive structure. Thank you for commenting.

jrumbut 5y ago

> The incentive structure of scientific publication is such that there are big rewards for being right on an important question, bigger the earlier you are to the party, and little to no penalties for being wrong, so long as the error cannot be provably and directly linked to fraud. This is fantastic insight and I'd like to thank you for sharing it with our group. Would you agree that the model of rewards for correctness and penalization only in the case of fraud is the core feature of science? And what separates it from business or politics where being an honest failure is worse than being dishonest but successful? Again, this is a great post, and I think you have a fantastic future in the sociology of science!

myle 5y ago
Are you trying to provide an example for many of the points of OP above or am I overreading this? In your message I observe, very careful criticism, uncalled praise, admission and defense of a system that excludes most criticism...
- jrumbut 5y ago
  I'm joking a bit with the style and my limited experience leads me to agree with the OP. The middle paragraph includes my sincere response that a system where discovery is rewarded, failure forgiven, and dishonesty punished is ideally suited to the mission of science. So I was left wondering if the OP would expand on what their thoughts were about the interesting consequences.
xab31 5y ago
It's a good question, and I don't like posting excessively long comments and didn't have time to make it concise, so here's an attempt at an answer: https://pastebin.com/fsrTtiKY https://pastebin.com/fsrTtiKY I think science is too big a thing to have a small set of "core features", and the question of how to usefully define "honesty" in a scientific context is another big topic, but reading about "bullshit" (the term of art that has its own literature, not the colloquialism) is a good place to start thinking about it. I would suggest that fraud is one of the rarest types of dishonesty, because people who are both smart and dishonest have less risky ways to proceed, and that such people are very glad fraud exists, because it misdirects attention away from their arguably more damaging and prevalent methods. Feynman has a passage about how honesty in science is more a state of mind, which I agree with. But really, the techniques to be dishonest with low risk are the same in science, journalism, politics, and business. My field isn't sociology of science though; these are just views from the genomics trenches.

lambdatronics 5y ago

The sociology of science is so interesting. (Not the field, but the subject.) Here are some of my favorites quotes/thoughts: This one is a direct contrast to your advice (which speaks volumes about what's wrong with academia): "A good scientist, in other words, does not merely ignore conventional wisdom, but makes a special effort to break it. Scientists go looking for trouble."[0] This was written about physics at Caltech, but applies more broadly. It explains why the ability to 'manage up' is so critical for early-career success. "[...] departments are run, for better and worse, by the professors who often lack managerial experience. Worse, they are generally unaware of this shortcoming, assuming incorrectly that management is trivially easy compared to their topics of study and merits minimal effort. We have now seen the consequences of this lack of attention." [1] Academic politics is a great reason not to stay in academia: "Look for environments where competitors see themselves as playing a game, rather than fighting for survival — this prevents rankings within the hierarchy from becoming an existential problem." [2] This book has a great chapter of career advice, here's a gem: "Don't build a pyramid. Everyone seems to build one pyramid per career. A pyramid is an ambitious system that one person really cares about and that winds up working well, but then just sits in the desert because nobody else cares the same way. This happens usually just after leaving graduate school." [3] "In general, status-conscious places are miserable for everyone, and the more, the worse." [3, next page] Gatekeeping is predictable from the incentive structure: "For all the high-level talk about how we need to plug the leaks in our STEM education pipeline, not only are we not plugging the holes, we're proud of how fast the pipeline is leaking." [4] "So why am I not an academic? There are many factors, and starting Tarsnap is certainly one; but most of them can be summarized as 'academia is a lousy place to do novel research'." [5] "...whereas Newton could say, 'If I have seen a little farther than others, it is because I have stood on the shoulders of giants,' I am forced to say, 'Today we stand on each other's feet.'" [6] [0] http://www.paulgraham.com/say.html http://www.paulgraham.com/say.html [1] https://caseyhandmer.wordpress.com/2019/08/09/caltech-astrophysics-and-harassment-lessons-learned/ https://caseyhandmer.wordpress.com/2019/08/09/caltech-astrop... [2] https://www.briantimar.com/notes/mimetic/mimetic/ https://www.briantimar.com/notes/mimetic/mimetic/ [3] Phillip Hobbs, "Building Electrooptical systems: making it all work" 2nd Ed, p392 [4] https://danluu.com/teach-debugging/ https://danluu.com/teach-debugging/ [5] http://www.daemonology.net/blog/2020-09-20-On-the-use-of-a-life.html http://www.daemonology.net/blog/2020-09-20-On-the-use-of-a-l... [6] Richard Hamming 1968 Turing Award lecture, Journal of the ACM 16 (1), January 1969, p. 3–12