4 ms·

I have some modest experience with telecommunications (engineering) research. In that field negative citations are standard practice to provide motivation for t

by pzs 11y ago

I have some modest experience with telecommunications (engineering) research. In that field negative citations are standard practice to provide motivation for the work you are about to introduce in a new paper. This does not necessarily have any negative connotation: it is merely a reflection of the fact that progress is made by improving on existing results.

kazinator 11y ago
But the thing is, those aren't disagreements. A disagreement is a claim that the cited researchers didn't find what they think they found; that essentially some of their results (perhaps all) are false. That does have a negative connotation; the "improvement" consists in uprooting some falsehood. Improving on engineering is nonconflicting. The existence of an excavator doesn't refute the shovel, and so on. A disagreement in, say, CS would be, "Foo Barley [42] mistakenly estimates an upper bound of O(n log n) on his algorithm; here we prove, in fact, a quadratic lower bound! Moreover, the algorithm calculates produces results in several cases that Foo Barley doesn't consider, presenting it as general." How much of that do we see?
- eob 11y ago
  To put this into the concrete terms of a story.. I used to be in the NLP group of a good school on the east coast. I was trying to reproduce an algorithm from a paper that won Best Paper from a good school on the west coast. A postdoc from that school said, "oh stay away from that algorithm. they basically cooked the results of that whole paper by cherry picking data. it doesn't really work at all". At that point I dropped it and never thought about it again -- there is no career value for a phd student (the engine room of research) in "outing" bad research. I was just trying to get some baselines to compare my own work against. On the flip side, getting best paper award at a top conference is the kind of thing that sets you up for faculty interviews at you choice of university. All that said, academia and the scientific process tend to work well, in aggregate and over time. I think a lot of times people with juicy or disgruntled stories speak more and get remembered more, and it casts academia in far more negative a light than reality. Most of the time, science works well.. just in a way that doesn't produce stories worth posting to boards like HN. So that's my counterbalance.
- pzs 11y ago
  Open disagreement is indeed rare according to my experience. EDIT: That may have to do something with peer reviews: they kind of prevent the publication of work that will be openly disagreed with later on.
derf_ 11y ago
When I was a grad student, I was explicitly advised to make such citations uncritical. Instead of saying, "Researchers A, B, and C failed to consider cases X, Y, and Z, which we address here," you would say, "Researchers A, B, and C solved this problem under the assumptions that X, Y, and Z were not issues. We extend their work here to handle these cases." That is, you say what they achieved, rather than what they didn't. Everyone wins with this approach. Sure, those lazy bums didn't consider cases X, Y, and Z, but then no one else had solved the problem regardless of whether or not they handled X, Y, or Z. That's how A, B, and C's results got published! You're building on their work. You look better for acknowledging it rather than trying to knock down those who came before. They are likely to be respected quite a bit better than you are. Also, researchers A, B, and C are the most likely candidates to be reviewing your paper. So there's that. This is, of course, orthogonal to the issue of unreproducible experimental results.
- pzs 11y ago
  I agree with you. You can and should point out shortcomings of work done by others in a respectful way. Even turn it around, if possible, as your example shows. Sometimes, however, there is no easy way to avoid saying what is missing. As an example, consider the case when the published research, without giving any explicit justification, does not address one aspect of a problem that the other researcher working in the same area finds important. But even in that case, you should not fail to acknowledge the work that you are building on. As another comment suggests my interpretation of 'negative citation' is probably less strict. I considered negative even the case of pointing out a shortcoming in a polite way, without questioning the validity of the work in a confrontative manner.
- refurb 11y ago
  As others have mentioned scientists tend to be pretty humble. Even when a previous paper was completely wrong (say a chemical structure was incorrect) very rarely do follow-up papers make a negative comment. They usually say "our results differ and this is why we think we're correct". I think a part of it too is that it's not hard to be fooled by your own results. Even a relatively innocuous error or failure to notice something in the date is really easy to do. I think follow-up papers are less critical since scientists can understand why the original paper was wrong.

by pzs 11y ago

kazinator 11y ago

But the thing is, those aren't disagreements. A disagreement is a claim that the cited researchers didn't find what they think they found; that essentially some of their results (perhaps all) are false. That does have a negative connotation; the "improvement" consists in uprooting some falsehood. Improving on engineering is nonconflicting. The existence of an excavator doesn't refute the shovel, and so on. A disagreement in, say, CS would be, "Foo Barley [42] mistakenly estimates an upper bound of O(n log n) on his algorithm; here we prove, in fact, a quadratic lower bound! Moreover, the algorithm calculates produces results in several cases that Foo Barley doesn't consider, presenting it as general." How much of that do we see?

eob 11y ago
To put this into the concrete terms of a story.. I used to be in the NLP group of a good school on the east coast. I was trying to reproduce an algorithm from a paper that won Best Paper from a good school on the west coast. A postdoc from that school said, "oh stay away from that algorithm. they basically cooked the results of that whole paper by cherry picking data. it doesn't really work at all". At that point I dropped it and never thought about it again -- there is no career value for a phd student (the engine room of research) in "outing" bad research. I was just trying to get some baselines to compare my own work against. On the flip side, getting best paper award at a top conference is the kind of thing that sets you up for faculty interviews at you choice of university. All that said, academia and the scientific process tend to work well, in aggregate and over time. I think a lot of times people with juicy or disgruntled stories speak more and get remembered more, and it casts academia in far more negative a light than reality. Most of the time, science works well.. just in a way that doesn't produce stories worth posting to boards like HN. So that's my counterbalance.
pzs 11y ago
Open disagreement is indeed rare according to my experience. EDIT: That may have to do something with peer reviews: they kind of prevent the publication of work that will be openly disagreed with later on.

derf_ 11y ago

When I was a grad student, I was explicitly advised to make such citations uncritical. Instead of saying, "Researchers A, B, and C failed to consider cases X, Y, and Z, which we address here," you would say, "Researchers A, B, and C solved this problem under the assumptions that X, Y, and Z were not issues. We extend their work here to handle these cases." That is, you say what they achieved, rather than what they didn't. Everyone wins with this approach. Sure, those lazy bums didn't consider cases X, Y, and Z, but then no one else had solved the problem regardless of whether or not they handled X, Y, or Z. That's how A, B, and C's results got published! You're building on their work. You look better for acknowledging it rather than trying to knock down those who came before. They are likely to be respected quite a bit better than you are. Also, researchers A, B, and C are the most likely candidates to be reviewing your paper. So there's that. This is, of course, orthogonal to the issue of unreproducible experimental results.

pzs 11y ago
I agree with you. You can and should point out shortcomings of work done by others in a respectful way. Even turn it around, if possible, as your example shows. Sometimes, however, there is no easy way to avoid saying what is missing. As an example, consider the case when the published research, without giving any explicit justification, does not address one aspect of a problem that the other researcher working in the same area finds important. But even in that case, you should not fail to acknowledge the work that you are building on. As another comment suggests my interpretation of 'negative citation' is probably less strict. I considered negative even the case of pointing out a shortcoming in a polite way, without questioning the validity of the work in a confrontative manner.
refurb 11y ago
As others have mentioned scientists tend to be pretty humble. Even when a previous paper was completely wrong (say a chemical structure was incorrect) very rarely do follow-up papers make a negative comment. They usually say "our results differ and this is why we think we're correct". I think a part of it too is that it's not hard to be fooled by your own results. Even a relatively innocuous error or failure to notice something in the date is really easy to do. I think follow-up papers are less critical since scientists can understand why the original paper was wrong.