3 ms·

Toxic, or rigorous? To my understanding, this kind of questioning is how the search for truth tries to keep itself from veering off into bullshit, and is fundam

Toxic, or rigorous? To my understanding, this kind of questioning is how the search for truth tries to keep itself from veering off into bullshit, and is fundamental to the whole project. I work in the corporate world where we incubate a great deal of bullshit; projects, careers, and entire teams enjoy wild success on the backs of purported contributions that do not withstand a moment’s scrutiny. It makes a polite and non-confrontational environment, sure, but also comes with its own kinds of psychic injuries.

cs137 4y ago
In academia, it's open season on attacking ideas. There's nothing wrong with this; indeed, that's what we expect our intellectuals to do. Unfortunately, some cannot separate (both as critics, and when criticized) the attacking of ideas from attacking of people. In corporate, it's always an attack on a person, because ideas don't matter in corporate. In theory, ideas can matter insofar as they can drive a P&L, but in practice, corporate is not about P&L at all so much as it is about how P&L are attributed to oneself personally. This does mean people are less ready to criticize an idea (or a person) in the open, although that's mostly because the real gaming happens behind the backs of those being gamed.
jltsiren 4y ago
Toxic. Most professional environments are non-confrontational but not nice. The academic ideal is being nice but sometimes confrontational. Being nice matters, because you can't hide from the same people behind your professional role for decades. Academic research is fundamentally based on trust. Trust that the others know what they are doing. Trust that they have taken something obvious into account even if they don't mention it explicitly. Trust that they are presenting their work honestly. And so on. If you choose not to trust the other party, you can almost always poke holes into their argument, as long as you are reasonably familiar with the topic. You should always be critical of your own research and offer constructive criticism to others. Your job is not shooting down their arguments but finding what can be salvaged and guiding them towards stronger arguments. And sometimes you have to accept that their arguments are already good enough, even if you think further work could improve them. Besides, everything is bullshit anyway. Our job is just finding something interesting in it.
- closeparen 4y ago
  Don’t you think it matters whether the claims in the paper are true? Won’t other people be relying on them in the future, depending on how they’re received in the field? It would seem important for other researchers to digest the evidence offered and update their worldviews accordingly, and it further seems like this would be more likely to converge towards reality if researchers talked openly about the process rather than keeping any serious flaws they find to themselves. I’m not a scientist, so I was relying on secondhand accounts. Am I way off base about what science is, here?
  - mattkrause 4y ago
    You can be rigorous without being toxic or--sadly--toxic without being rigorous. In fact, I'd say they are almost unrelated and the same critique can be leveled either way. I don't think @jltsiren meant that you have to have unconditional trust in someone else's work. Instead, you should approach it with a bit of humility. At a talk, the speaker is usually presenting work they've spent years on, whereas most of the audience saw it for the first time fifteen minutes ago. In most cases, you can trust that they've thought a bit about the obvious, show-stopping objection that immediately popped into your head. You can--and should--verify this assumption too, but there are light-years between: - "Could you walk us through your experiment design again? In particular, is there a wash-out period to account for any potential carryover effects?" - "These results are hot garbage. I don't trust any fMRI stuff and as for the rest, I can't believe you'd run a crossover experiment for this because there are obviously carryover effects that you've glossed over in your talk."

cs137 4y ago

In academia, it's open season on attacking ideas. There's nothing wrong with this; indeed, that's what we expect our intellectuals to do. Unfortunately, some cannot separate (both as critics, and when criticized) the attacking of ideas from attacking of people. In corporate, it's always an attack on a person, because ideas don't matter in corporate. In theory, ideas can matter insofar as they can drive a P&L, but in practice, corporate is not about P&L at all so much as it is about how P&L are attributed to oneself personally. This does mean people are less ready to criticize an idea (or a person) in the open, although that's mostly because the real gaming happens behind the backs of those being gamed.

jltsiren 4y ago

Toxic. Most professional environments are non-confrontational but not nice. The academic ideal is being nice but sometimes confrontational. Being nice matters, because you can't hide from the same people behind your professional role for decades. Academic research is fundamentally based on trust. Trust that the others know what they are doing. Trust that they have taken something obvious into account even if they don't mention it explicitly. Trust that they are presenting their work honestly. And so on. If you choose not to trust the other party, you can almost always poke holes into their argument, as long as you are reasonably familiar with the topic. You should always be critical of your own research and offer constructive criticism to others. Your job is not shooting down their arguments but finding what can be salvaged and guiding them towards stronger arguments. And sometimes you have to accept that their arguments are already good enough, even if you think further work could improve them. Besides, everything is bullshit anyway. Our job is just finding something interesting in it.

closeparen 4y ago
Don’t you think it matters whether the claims in the paper are true? Won’t other people be relying on them in the future, depending on how they’re received in the field? It would seem important for other researchers to digest the evidence offered and update their worldviews accordingly, and it further seems like this would be more likely to converge towards reality if researchers talked openly about the process rather than keeping any serious flaws they find to themselves. I’m not a scientist, so I was relying on secondhand accounts. Am I way off base about what science is, here?
- mattkrause 4y ago
  You can be rigorous without being toxic or--sadly--toxic without being rigorous. In fact, I'd say they are almost unrelated and the same critique can be leveled either way. I don't think @jltsiren meant that you have to have unconditional trust in someone else's work. Instead, you should approach it with a bit of humility. At a talk, the speaker is usually presenting work they've spent years on, whereas most of the audience saw it for the first time fifteen minutes ago. In most cases, you can trust that they've thought a bit about the obvious, show-stopping objection that immediately popped into your head. You can--and should--verify this assumption too, but there are light-years between: - "Could you walk us through your experiment design again? In particular, is there a wash-out period to account for any potential carryover effects?" - "These results are hot garbage. I don't trust any fMRI stuff and as for the rest, I can't believe you'd run a crossover experiment for this because there are obviously carryover effects that you've glossed over in your talk."