7 ms·

Professor's apparent request to student for fake data published by mistake

112 points by apo 13y ago

sachingulaya 13y ago
Can anyone comment on how important this is to the results of the paper?
- AsymetricCom 13y ago
  Somewhere between vital and fundamental.
  - rflrob 13y ago
    If that's the case, why is it in the Supporting Information section? In my most recent paper, I used the SI for results that would be of interest to some people, but do not fundamentally affect the interpretation of the data. In the bad old days before journals were online, having a Supporting Information section wasn't even an option---either it's in the paper, or it's not. If it is crucial to the interpretation of the paper, it ought to be in a major figure/table, and the reviewers ought to have asked for it to be. Otherwise, it's position in the SI indicates to me that it's potentially useful, but not critical.
    - refurb 13y ago
      Data is often repeated in the SI section. You put all of your data in the SI and only put select data in the paper.
    - AsymetricCom 13y ago
      I understand what you're saying but more fundamental than the supporting data is the trust in the person witting it.
- refurb 13y ago
  To the scientific results of the paper? Not much. Usually you use two different methods of analysis to confirm the structure of a compound. The compound was identified by NMR, so the elemental analysis (EA) is used to back-up the identification (it's actually very sensitive and very solid evidence). It's not that big of a deal, because often intermediates are not isolated, so if this compound was carried forward to another compound and THAT was fully characterized, the science is solid. However, there are two problems. Do you believe the data for the final compounds knowing that the data from an intermediate was fudged? In the sense of damage to the credibility of the authors, it is very important.
randallsquared 13y ago
Is it possible that "make up" meant "put together" or "do", instead of "fake up"?
- devindotcom 13y ago
  This is what I thought... there are brits in the comments saying they have used this phrase in this way many times. That said, it is a little odd that it made it through 3 reviewers, none of whom objected or got clarification — or if they did, it's not on the record. (edit: which/whom)
  - mathattack 13y ago
    That it made it through the reviewers suggests the rigor of the review process. :-) Was this a major journal? I also wondered if there was something less nefarious in the explanation. Was it "Falsify some results" or "Toss up some BS about this topic". The latter is bad, but part of the academic paper business. The first is career ending.
  - anateus 13y ago
    Using the plural for "data" ("Where are they?" it says) increases the likelihood that "make up" just means "do" in that context.
- JohnDakota 13y ago
  The student is now a Postdoc in Brazil and the PI has since moved to Western Australia for another position. These moves happened in 2011 and 2012 respectively, according to their linkedin profiles. So unless they're still working on the project, it's unlikely the phrase 'just make up' means to put together.. rather it's more likely that it means to fabricate. Basic elemental analysis is very easy to fake, and very hard to demonstrate it was faked.
- vstm 13y ago
  Since the prof studied at the university of Neuchâtel and Salerno I guess it's save to assume he's not a native english speaker. So I think it's possible that it was just sloppy wording. I'm curious what his statement is going to say.
cing 13y ago
The editor of the journal has made a statement, http://www.chemistry-blog.com/2013/08/07/when-authors-forget-to-fake-an-elemental-analysis/ http://www.chemistry-blog.com/2013/08/07/when-authors-forget...
- devindotcom 13y ago
  Doesn't add much, though: "The author has explained to us that the statement pertains to a compound that was ”downgraded” from something being isolated to a proposed intermediate. Hence, we have left the ASAP manuscript on the web for now." Did the author explain whether or not he was asking a subordinate to fabricate data? That seems to be the more pressing issue here!
FD3SA 13y ago
On the matter of the academic-industrial complex, I humbly defer to Elon Musk: http://youtu.be/vDwzmJpI4io?t=9m31s http://youtu.be/vDwzmJpI4io?t=9m31s
- FD3SA 13y ago
  Due to the downvote, I'd just like to explain a bit further. Academia is designed with very unique economics which favor publishing over useful experimental results. As such, it can only be expected that rational actors will game the system by fabricating data in order to publish impactful papers, especially if the short term gains (tenure) outweigh the long term consequences (incorrect results). It is a classic case of improper incentives leading to undesired outcomes.
short_circut 13y ago
I honestly can't see how this made it through the peer review process and the editorial review process with out being noticed. Were the reviewers also not doing their due diligence?
- JohnDakota 13y ago
  Reviewer responsibilities are generally volunteer hours insofar as the peer reviewing the article isn't getting paid, or compensated in any way for the many hours/day(s) it will likely take to properly vet a piece of work. Therefore reviewers eventually blow off some articles, choosing instead to do a quick analysis and hope the other reviewers perform a more thorough analysis. Apparently, in the case of this paper, all the reviewers blew off their jobs.
- bathat 13y ago
  What John Dakota said, plus one more thing: I've noticed that papers written in the past ten years tend to be written by people who have poorer writing skills (to the point of sometimes writing nearly incomprehensible paragraphs) than papers published, say, 30 to 40 years ago. At least in my field and a few somewhat-related fields. It may be different in other fields. It's possible that some of today's reviewers also have poorer reading skills than reviewers 40 years ago, and thus don't really understand what they are reading some of the time. Combined with a lack of time, they might simply skip parts they do not understand.
cperciva 13y ago
My father -- a chemistry professor who has published in Organometallics and worked in Zurich (where this article originated) about 35 years ago -- says he thinks it's very likely the "make up" came from a Swiss-German speaker and meant "put together". It's very embarrassing that an article would get published with a clear lack of proofreading (and presumably missing data), but based on his comments I'm willing to hold off on suspecting fraud for now.
- mikeash 13y ago
  That was my thought as well. Even in English, you can use "make up" to mean "create" rather than "build something fake", as in "make up a sandwich". It's not common usage, but it's a real thing.
  - UnoriginalGuy 13y ago
    Indeed. Seems like an Americanism to use "make up" only as synonymous with "fake." See definition #1: http://en.wiktionary.org/wiki/make_up http://en.wiktionary.org/wiki/make_up Around here (UK) I hear "make up" to mean "create" all the time... If it is fake or real depends on context, and "just" isn't enough to form an opinion.
    - gojomo 13y ago
      In American dialect, at least, I can assure that the added 'up' nudges the interpretation more towards fabrication (though not definitively so). Compare: "...make an elementary analysis." [strongly implies 'do the necessary work'] "...make up an elementary analysis." [might mean 'do the work' but has some implicature of 'improvise from your imagination'] On the other hand, 'whip up' is merely an idiom for 'bring together quickly', and so "...whip up an elementary analysis" would again have no implication of faked data (as opposed to real data collected quickly).
      - emiliobumachar 13y ago
        Ah, the irony. I'm not a native English speaker, but I believe the word "fabrication" also has a less used meaning of "made, created" with no falsehood necessarily. A quick web search seemed to confirm this.
haldujai 13y ago
While this specific example may not be one of academic fraud it is important not to underestimate the ease in fabricating data, especially if it is for a project no one really cares about. You can't even apply the logic of 'if this is from a respected lab it must be authentic' because papers from such labs are rarely reviewed with care by either the editors of the journal or the PI (the professor in charge) due to the reputation of said professor. What I find far worse is the lack of rigor in many labs, little errors like overgrowing your cells or incorrectly calibrating an instrument can cause drastic phenotype changes that renders any results moot without being obvious, no one would bothers to check that the basics of lab work are correctly implemented. I have personally witnessed such errors, which when caught, were not reported as an addendum. The problem with these errors is that there exists data to back up the claims but the data is invalid, but that is extremely hard to prove. Often when a lab with a systematic error affecting their results is questioned they would simply repeat the experiments and send the raw data to the investigator and that would be that. Fabricating data on the other hand is much easier to prove as there is no underlying true data, such as instrument measurements, and the experiment will never be repeatable.
mbreese 13y ago
Here's a comment from the PI on the paper: http://blog.chembark.com/2013/08/06/a-disturbing-note-in-a-recent-si-file/#comment-52180 http://blog.chembark.com/2013/08/06/a-disturbing-note-in-a-r... Compound 14 in the SI is an intermediate and has not been fully characterized, hence does not have a number in the manuscript. Wording and numbering of the compounds in the supporting information are wrong (on different levels!). Characterized compound 14 and 15a-c of the article correspond to compounds 154, 165a, 165b and 165c of the supporting info. Anything else is being dealt with by the editors of the journal as we speak.
auctiontheory 13y ago
Scientists tend to use language in precise and idiosyncratic ways, and when you add to that the language barrier - this seems much ado about nothing. I remember a German colleague being reprimanded over his usage of "you people" in a mass email - apparently it means something different and more friendly in the German from which he mentally translated it.
sukaka 13y ago
tl;dr, "make up" data means either put together or fake.
charris5 13y ago
With over 14 years in academia in chemistry and physics labs, and in my personal experience, European scientists speak English often with greater precision than native US citizens. And, given my own experience in academia, ... "make up" probably meant literally "you need to 'find' the data to fit the conclusions of this paper, ... truth be damned." Many academic PI's (lead investigators or Professors), for many complicated reasons and under the weight of a ridiculous feedback-loop that relentlessly increases the pressure to publish, have abandoned what many perceive as the White Tower's pursuit of truth in exchange for the pursuit of career. That's not to say there aren't many noble academics pursuing the truth and publishing only verified, repeatable findings ... it is just that they seem to be unfortunately in the minority within the zeitgeist that presently drives the climate of our academic research institutions. Academic research is no longer about White Towers and truth ... Academic research has become "big business" with millions of dollars of funding at stake. Unfortunate...sad...but tragically true.

112 points by apo 13y ago

sachingulaya 13y ago

Can anyone comment on how important this is to the results of the paper?

AsymetricCom 13y ago
Somewhere between vital and fundamental.
- rflrob 13y ago
  If that's the case, why is it in the Supporting Information section? In my most recent paper, I used the SI for results that would be of interest to some people, but do not fundamentally affect the interpretation of the data. In the bad old days before journals were online, having a Supporting Information section wasn't even an option---either it's in the paper, or it's not. If it is crucial to the interpretation of the paper, it ought to be in a major figure/table, and the reviewers ought to have asked for it to be. Otherwise, it's position in the SI indicates to me that it's potentially useful, but not critical.
  - refurb 13y ago
    Data is often repeated in the SI section. You put all of your data in the SI and only put select data in the paper.
  - AsymetricCom 13y ago
    I understand what you're saying but more fundamental than the supporting data is the trust in the person witting it.
refurb 13y ago
To the scientific results of the paper? Not much. Usually you use two different methods of analysis to confirm the structure of a compound. The compound was identified by NMR, so the elemental analysis (EA) is used to back-up the identification (it's actually very sensitive and very solid evidence). It's not that big of a deal, because often intermediates are not isolated, so if this compound was carried forward to another compound and THAT was fully characterized, the science is solid. However, there are two problems. Do you believe the data for the final compounds knowing that the data from an intermediate was fudged? In the sense of damage to the credibility of the authors, it is very important.

randallsquared 13y ago

Is it possible that "make up" meant "put together" or "do", instead of "fake up"?

devindotcom 13y ago
This is what I thought... there are brits in the comments saying they have used this phrase in this way many times. That said, it is a little odd that it made it through 3 reviewers, none of whom objected or got clarification — or if they did, it's not on the record. (edit: which/whom)
- mathattack 13y ago
  That it made it through the reviewers suggests the rigor of the review process. :-) Was this a major journal? I also wondered if there was something less nefarious in the explanation. Was it "Falsify some results" or "Toss up some BS about this topic". The latter is bad, but part of the academic paper business. The first is career ending.
- anateus 13y ago
  Using the plural for "data" ("Where are they?" it says) increases the likelihood that "make up" just means "do" in that context.
JohnDakota 13y ago
The student is now a Postdoc in Brazil and the PI has since moved to Western Australia for another position. These moves happened in 2011 and 2012 respectively, according to their linkedin profiles. So unless they're still working on the project, it's unlikely the phrase 'just make up' means to put together.. rather it's more likely that it means to fabricate. Basic elemental analysis is very easy to fake, and very hard to demonstrate it was faked.
vstm 13y ago
Since the prof studied at the university of Neuchâtel and Salerno I guess it's save to assume he's not a native english speaker. So I think it's possible that it was just sloppy wording. I'm curious what his statement is going to say.

cing 13y ago

The editor of the journal has made a statement, http://www.chemistry-blog.com/2013/08/07/when-authors-forget-to-fake-an-elemental-analysis/ http://www.chemistry-blog.com/2013/08/07/when-authors-forget...

devindotcom 13y ago
Doesn't add much, though: "The author has explained to us that the statement pertains to a compound that was ”downgraded” from something being isolated to a proposed intermediate. Hence, we have left the ASAP manuscript on the web for now." Did the author explain whether or not he was asking a subordinate to fabricate data? That seems to be the more pressing issue here!

FD3SA 13y ago

On the matter of the academic-industrial complex, I humbly defer to Elon Musk: http://youtu.be/vDwzmJpI4io?t=9m31s http://youtu.be/vDwzmJpI4io?t=9m31s

FD3SA 13y ago
Due to the downvote, I'd just like to explain a bit further. Academia is designed with very unique economics which favor publishing over useful experimental results. As such, it can only be expected that rational actors will game the system by fabricating data in order to publish impactful papers, especially if the short term gains (tenure) outweigh the long term consequences (incorrect results). It is a classic case of improper incentives leading to undesired outcomes.

short_circut 13y ago

I honestly can't see how this made it through the peer review process and the editorial review process with out being noticed. Were the reviewers also not doing their due diligence?

JohnDakota 13y ago
Reviewer responsibilities are generally volunteer hours insofar as the peer reviewing the article isn't getting paid, or compensated in any way for the many hours/day(s) it will likely take to properly vet a piece of work. Therefore reviewers eventually blow off some articles, choosing instead to do a quick analysis and hope the other reviewers perform a more thorough analysis. Apparently, in the case of this paper, all the reviewers blew off their jobs.
bathat 13y ago
What John Dakota said, plus one more thing: I've noticed that papers written in the past ten years tend to be written by people who have poorer writing skills (to the point of sometimes writing nearly incomprehensible paragraphs) than papers published, say, 30 to 40 years ago. At least in my field and a few somewhat-related fields. It may be different in other fields. It's possible that some of today's reviewers also have poorer reading skills than reviewers 40 years ago, and thus don't really understand what they are reading some of the time. Combined with a lack of time, they might simply skip parts they do not understand.

cperciva 13y ago

My father -- a chemistry professor who has published in Organometallics and worked in Zurich (where this article originated) about 35 years ago -- says he thinks it's very likely the "make up" came from a Swiss-German speaker and meant "put together". It's very embarrassing that an article would get published with a clear lack of proofreading (and presumably missing data), but based on his comments I'm willing to hold off on suspecting fraud for now.

mikeash 13y ago
That was my thought as well. Even in English, you can use "make up" to mean "create" rather than "build something fake", as in "make up a sandwich". It's not common usage, but it's a real thing.
- UnoriginalGuy 13y ago
  Indeed. Seems like an Americanism to use "make up" only as synonymous with "fake." See definition #1: http://en.wiktionary.org/wiki/make_up http://en.wiktionary.org/wiki/make_up Around here (UK) I hear "make up" to mean "create" all the time... If it is fake or real depends on context, and "just" isn't enough to form an opinion.
  - gojomo 13y ago
    In American dialect, at least, I can assure that the added 'up' nudges the interpretation more towards fabrication (though not definitively so). Compare: "...make an elementary analysis." [strongly implies 'do the necessary work'] "...make up an elementary analysis." [might mean 'do the work' but has some implicature of 'improvise from your imagination'] On the other hand, 'whip up' is merely an idiom for 'bring together quickly', and so "...whip up an elementary analysis" would again have no implication of faked data (as opposed to real data collected quickly).
    - emiliobumachar 13y ago
      Ah, the irony. I'm not a native English speaker, but I believe the word "fabrication" also has a less used meaning of "made, created" with no falsehood necessarily. A quick web search seemed to confirm this.

haldujai 13y ago

While this specific example may not be one of academic fraud it is important not to underestimate the ease in fabricating data, especially if it is for a project no one really cares about. You can't even apply the logic of 'if this is from a respected lab it must be authentic' because papers from such labs are rarely reviewed with care by either the editors of the journal or the PI (the professor in charge) due to the reputation of said professor. What I find far worse is the lack of rigor in many labs, little errors like overgrowing your cells or incorrectly calibrating an instrument can cause drastic phenotype changes that renders any results moot without being obvious, no one would bothers to check that the basics of lab work are correctly implemented. I have personally witnessed such errors, which when caught, were not reported as an addendum. The problem with these errors is that there exists data to back up the claims but the data is invalid, but that is extremely hard to prove. Often when a lab with a systematic error affecting their results is questioned they would simply repeat the experiments and send the raw data to the investigator and that would be that. Fabricating data on the other hand is much easier to prove as there is no underlying true data, such as instrument measurements, and the experiment will never be repeatable.

mbreese 13y ago

Here's a comment from the PI on the paper: http://blog.chembark.com/2013/08/06/a-disturbing-note-in-a-recent-si-file/#comment-52180 http://blog.chembark.com/2013/08/06/a-disturbing-note-in-a-r... Compound 14 in the SI is an intermediate and has not been fully characterized, hence does not have a number in the manuscript. Wording and numbering of the compounds in the supporting information are wrong (on different levels!). Characterized compound 14 and 15a-c of the article correspond to compounds 154, 165a, 165b and 165c of the supporting info. Anything else is being dealt with by the editors of the journal as we speak.

auctiontheory 13y ago

Scientists tend to use language in precise and idiosyncratic ways, and when you add to that the language barrier - this seems much ado about nothing. I remember a German colleague being reprimanded over his usage of "you people" in a mass email - apparently it means something different and more friendly in the German from which he mentally translated it.

sukaka 13y ago

tl;dr, "make up" data means either put together or fake.

charris5 13y ago

With over 14 years in academia in chemistry and physics labs, and in my personal experience, European scientists speak English often with greater precision than native US citizens. And, given my own experience in academia, ... "make up" probably meant literally "you need to 'find' the data to fit the conclusions of this paper, ... truth be damned." Many academic PI's (lead investigators or Professors), for many complicated reasons and under the weight of a ridiculous feedback-loop that relentlessly increases the pressure to publish, have abandoned what many perceive as the White Tower's pursuit of truth in exchange for the pursuit of career. That's not to say there aren't many noble academics pursuing the truth and publishing only verified, repeatable findings ... it is just that they seem to be unfortunately in the minority within the zeitgeist that presently drives the climate of our academic research institutions. Academic research is no longer about White Towers and truth ... Academic research has become "big business" with millions of dollars of funding at stake. Unfortunate...sad...but tragically true.