11 ms·

While pursuing my Ph.D. in chemistry, I was asked to peer review an inorganic chemistry publication in which the author attempted to fake crystallography data.

While pursuing my Ph.D. in chemistry, I was asked to peer review an inorganic chemistry publication in which the author attempted to fake crystallography data. In this case the telltale sign was that they left the hydrogen atoms in the structure - X-ray diffraction is unable to resolve the hydrogens, you need a more sophisticated technique like neutron diffraction for that. What it turned out they had done is optimize a predicted structure of their molecule using density functional theory (DFT), then draw it like a crystal structure would be displayed (a 3D plot with error ellipsoids at each atom, with probable bonds shown) and tried to pass it off as a true crystal structure. My advisor missed this when he reviewed it first - if you're reading quickly, you just see a crystal structure and accept it for what it is. So I wonder how many like this slip through the cracks every year. Apparently a lot, if the CCDC identified ca. 1000 of them.

echelon 4y ago
Does this not immediately destroy one's scientific career and invalidate all past papers? It seems like it should. Maybe there should be a bounty on uncovering these? This shit has to stop. If publish or perish is going to continue to be a thing, maybe just switch to publishing negative results. It's not sexy, but it's valuable too.
- shishy 4y ago
  Yes, they'd get retracted if caught and be a black mark / destroy a career if someone did something like this and it was proven they intentionally faked it. What often happens though is that the paper will get retracted but then the authors play hot potato in blaming different people for faking it and it turns into a whole mess where you don't always know (i.e. cannot prove) exactly who did the faking. "Someone else faked it and I missed it, just like all the reviewers!!"
- logifail 4y ago
  > If publish or perish is going to continue to be a thing, maybe just switch to publishing negative results. It's not sexy, but it's valuable too. If we are to be allowed to publish papers from negative results then I should go and dig out my PhD notes and get writing! Seriously, though, I started noticing the many broken incentives early on, after a explanation about "impact factor" from an older PhD student in our lab.
  - opticfluorine 4y ago
    Same here - I went into science thinking that it would be all about trying new ideas and pushing the boundaries of what we know. And while in a certain sense that is still present, it takes a back seat to the perverse incentives you describe. This was one of (and unfortunately not the only) reason I decided to leave academia after my Ph.D., leaving my field entirely and going into software engineering rather than taking a postdoc position. Academia isn't for everyone, and neither is industry... but wow, I sure am happier where I am now.
- 2muchcoffeeman 4y ago
  >This shit has to stop. If publish or perish is going to continue to be a thing, maybe just switch to publishing negative results. It's not sexy, but it's valuable too. I had an argument recently about why fundamental research was important. The other person insisted that money should only be spent on endeavors where the outcome is known to benefit humanity. They couldn't accept, despite not having any science or math training, that sometimes we study things that start off seemingly useless and become important, or that we sometimes stumble upon things. (And this is why we should spend money on science in general). Think about all the scientifically illiterate politicians who control money and funding who probably have similar views. I can totally understand how we let things get this bad. If you don't provide a positive result, you might be facing unemployment. It's not about advancing our knowledge anymore.
  - jiggawatts 4y ago
    Why spend money on the military if we’re not at war?
Jensson 4y ago
Most of these are probably not even possible to catch, if the perpetrator has a basic understanding of the theory they could generate those correctly no problem.
- type0 4y ago
  Hence why fake it till you make it is so popular
dekhn 4y ago
Not to say this disproves anything, but you can definitely see hydrogen atoms in extremely high resolution crystal structures- typically with resolution below 1A, which is rare for proteins but should not be uncommon in inorganic crystals (right?) ANy time you have a 3D structure (model) you can run the equations in forward mode to create the structure factors of the crystal that would have created that model.
- HarryHirsch 4y ago
  Most of the time hydrogens are placed in calculated positions because you know where they are and it also reduces the number of parameters needing refinement. In good datasets hydrogens will show up in the density map (aren't all datasets good now in the age of area detectors), and you can refine (possibly with restraints) those hydrogens that you are uncertain about.
- COGlory 4y ago
  Raleigh's criterion says hydrogens should be just on the cusp, at least with home source xrays. Not sure about synchotron.
  - dekhn 4y ago
    I mean, I know this is possible- I've definitely seen maps from very high quality (<.8A) crystals that clearly had defined bumps on the heavy atoms that were where you'd expect the hydrogens to be, and later on, neutron scattering confirmed it. But the only times I've seen this it was world-class competitive groups trying to elucidate the specific enzymatic pathway for a protein of industrial interest.
- opticfluorine 4y ago
  Yes, with a powerful enough source you can resolve protons, that's true. They didn't make such a claim - they listed a run-of-the-mill instrument in their methods section that would not have been capable of this.
hidroto 4y ago
"optimize a predicted structure of their molecule using density functional theory (DFT), then draw it like a crystal structure would be displayed (a 3D plot with error ellipsoids at each atom, with probable bonds shown)" why not just publish this instead. surely an optimized model is still valuable.
- opticfluorine 4y ago
  Typically not on their own - it's very easy to calculate an optimized structure that is incorrect, so you typically want some sort of justification for your structure. Typically you would be calculating one of these geometries along the way to calculating other interesting properties to either compare with experiment or make new predictions (or both). The "gold standard" is... comparison with a crystal structure from X-ray diffraction. In my field (small inorganic molecules used as catalysts), I typically calculated the structure along the way to calculating the molecular orbitals for describing the electronic structure (important for understanding many properties of the catalyst and its reactivity) and predicting the infrared and visible spectra (useful for comparison with experiment and for explaining the electronic and magnetic properties of the molecule). It's the whole picture that's valuable, not just the structure - and the more experimental properties you include, the harder it is to fake the whole thing.
- chaxor 4y ago
  I absolutely agree. Most of the time, getting very good DFT results is actually far more difficult that getting the experimental results of cystollography itself. They're both quite hard, and it depends on which techniques are applied, but getting computational methods to match with great quantum accuracy is very, *very* difficult. I know more about condensed matter physics and GW Bethe Salpeter approaches etc, than proteins, but the difficulty in getting good computational methods to work is still out of reach for ~95% of scientists even in that field.
kergonath 4y ago
> the author attempted to fake crystallography data That is not what fake data is. It might be a misrepresentation is the author tries to pass it out as experimental results, but it is not fake, i.e. it came from a real study and actually has some meaning (though not much if it is indeed an unstable structure that can just be observed in a computer, and it is ground for rejection depending on the journal). I mean, I don’t have much patience with people over-hyping their results, but fake data of the common sort is either photoshopped graphs, re-used spectra coming from other studies or compounds, or things like that. And this definitely happens. But the “it came from a simulation, therefore it’s fake” argument is very unhelpful in general. A spectrum from a simulation is not fake, it’s just a simulation result.
- raverbashing 4y ago
  It is fake if it's being misrepresented as experimental results instead of just a simulation
  - kergonath 4y ago
    Again, that is not what this word means. It’s misrepresented data, not fake. And yes, the distinction matters. This is also not what the article is about. The problem they face is completely fabricated data, not DFT structures in a poorly worded paper. Of all places, this is probably the last one where I would expect this argument.
    - adgjlsfhk1 4y ago
      by this metric no data is fake. It's just misrepresented and was actually numbers I pulled out of my ass.
- 1vuio0pswjnm7 4y ago
  It is arguable that it is more than a misrepresentation if the author's intent is to fool the reveiwers. If so, it is "fraudulent data".
- DiggyJohnson 4y ago
muaytimbo 4y ago
I published a number of crystal structures using a powerful X ray diffractor and high quality crystals where H density was both found and modeled. Modeling H density in and of itself is in no way a sign of fake data, it is just a rare occurrence.
- verisimi 4y ago
  Hang on, isn't a model by necessity not the same as the thing it's modeling? Obviously a photo is sort of a model, but depending on what the subject is, you can visually verify it. If it's a software model, and say you enter lots of parameters, I have to wonder how valid that model is.. How can it be verified?
  - ta988 4y ago
    The way you solve x-ray diffraction structures is that you refine your model until the electron density matches.
    - verisimi 4y ago
      Thanks.. But until your electron density matches what?
      - muaytimbo 4y ago
        You change the position and identity of the atoms in your model until there is very little residual electron density left remaining.
        verisimi 4y ago
        These are words, but I don't see how they prove the model. I guess it's like some sort of check after the operation is complete? Why does a low residual electron density prove the model? What is electron density anyway? And how do we measure it?
ta988 4y ago
You can see the hydrogens if you have enough resolution (below 1A). These days you start to see such good resolutions pretty often.
- ta988 4y ago
  Here are two articles supporting my point as you don't seem to believe it. I also recommend you go look at PDB and select structures below 1A so you can see for yourself. https://bmcresnotes.biomedcentral.com/articles/10.1186/1756-0500-6-30 https://bmcresnotes.biomedcentral.com/articles/10.1186/1756-... https://www.science.org/doi/10.1126/sciadv.1600192 https://www.science.org/doi/10.1126/sciadv.1600192 """ Textbooks and teaching classes still emphasize that hydrogen atoms cannot be located with x-rays close to heavy elements; instead, neutron diffraction is needed. We show that, contrary to widespread expectation, hydrogen atoms can be located very accurately using x-ray diffraction, yielding bond lengths involving hydrogen atoms (A–H) that are in agreement with results from neutron diffraction mostly within a single standard deviation"""

echelon 4y ago

Does this not immediately destroy one's scientific career and invalidate all past papers? It seems like it should. Maybe there should be a bounty on uncovering these? This shit has to stop. If publish or perish is going to continue to be a thing, maybe just switch to publishing negative results. It's not sexy, but it's valuable too.

shishy 4y ago
Yes, they'd get retracted if caught and be a black mark / destroy a career if someone did something like this and it was proven they intentionally faked it. What often happens though is that the paper will get retracted but then the authors play hot potato in blaming different people for faking it and it turns into a whole mess where you don't always know (i.e. cannot prove) exactly who did the faking. "Someone else faked it and I missed it, just like all the reviewers!!"
logifail 4y ago
> If publish or perish is going to continue to be a thing, maybe just switch to publishing negative results. It's not sexy, but it's valuable too. If we are to be allowed to publish papers from negative results then I should go and dig out my PhD notes and get writing! Seriously, though, I started noticing the many broken incentives early on, after a explanation about "impact factor" from an older PhD student in our lab.
- opticfluorine 4y ago
  Same here - I went into science thinking that it would be all about trying new ideas and pushing the boundaries of what we know. And while in a certain sense that is still present, it takes a back seat to the perverse incentives you describe. This was one of (and unfortunately not the only) reason I decided to leave academia after my Ph.D., leaving my field entirely and going into software engineering rather than taking a postdoc position. Academia isn't for everyone, and neither is industry... but wow, I sure am happier where I am now.
2muchcoffeeman 4y ago
>This shit has to stop. If publish or perish is going to continue to be a thing, maybe just switch to publishing negative results. It's not sexy, but it's valuable too. I had an argument recently about why fundamental research was important. The other person insisted that money should only be spent on endeavors where the outcome is known to benefit humanity. They couldn't accept, despite not having any science or math training, that sometimes we study things that start off seemingly useless and become important, or that we sometimes stumble upon things. (And this is why we should spend money on science in general). Think about all the scientifically illiterate politicians who control money and funding who probably have similar views. I can totally understand how we let things get this bad. If you don't provide a positive result, you might be facing unemployment. It's not about advancing our knowledge anymore.
- jiggawatts 4y ago
  Why spend money on the military if we’re not at war?

Jensson 4y ago

Most of these are probably not even possible to catch, if the perpetrator has a basic understanding of the theory they could generate those correctly no problem.

type0 4y ago
Hence why fake it till you make it is so popular

dekhn 4y ago

Not to say this disproves anything, but you can definitely see hydrogen atoms in extremely high resolution crystal structures- typically with resolution below 1A, which is rare for proteins but should not be uncommon in inorganic crystals (right?) ANy time you have a 3D structure (model) you can run the equations in forward mode to create the structure factors of the crystal that would have created that model.

HarryHirsch 4y ago
Most of the time hydrogens are placed in calculated positions because you know where they are and it also reduces the number of parameters needing refinement. In good datasets hydrogens will show up in the density map (aren't all datasets good now in the age of area detectors), and you can refine (possibly with restraints) those hydrogens that you are uncertain about.
COGlory 4y ago
Raleigh's criterion says hydrogens should be just on the cusp, at least with home source xrays. Not sure about synchotron.
- dekhn 4y ago
  I mean, I know this is possible- I've definitely seen maps from very high quality (<.8A) crystals that clearly had defined bumps on the heavy atoms that were where you'd expect the hydrogens to be, and later on, neutron scattering confirmed it. But the only times I've seen this it was world-class competitive groups trying to elucidate the specific enzymatic pathway for a protein of industrial interest.
opticfluorine 4y ago
Yes, with a powerful enough source you can resolve protons, that's true. They didn't make such a claim - they listed a run-of-the-mill instrument in their methods section that would not have been capable of this.

hidroto 4y ago

"optimize a predicted structure of their molecule using density functional theory (DFT), then draw it like a crystal structure would be displayed (a 3D plot with error ellipsoids at each atom, with probable bonds shown)" why not just publish this instead. surely an optimized model is still valuable.

opticfluorine 4y ago
Typically not on their own - it's very easy to calculate an optimized structure that is incorrect, so you typically want some sort of justification for your structure. Typically you would be calculating one of these geometries along the way to calculating other interesting properties to either compare with experiment or make new predictions (or both). The "gold standard" is... comparison with a crystal structure from X-ray diffraction. In my field (small inorganic molecules used as catalysts), I typically calculated the structure along the way to calculating the molecular orbitals for describing the electronic structure (important for understanding many properties of the catalyst and its reactivity) and predicting the infrared and visible spectra (useful for comparison with experiment and for explaining the electronic and magnetic properties of the molecule). It's the whole picture that's valuable, not just the structure - and the more experimental properties you include, the harder it is to fake the whole thing.
chaxor 4y ago
I absolutely agree. Most of the time, getting very good DFT results is actually far more difficult that getting the experimental results of cystollography itself. They're both quite hard, and it depends on which techniques are applied, but getting computational methods to match with great quantum accuracy is very, *very* difficult. I know more about condensed matter physics and GW Bethe Salpeter approaches etc, than proteins, but the difficulty in getting good computational methods to work is still out of reach for ~95% of scientists even in that field.

kergonath 4y ago

> the author attempted to fake crystallography data That is not what fake data is. It might be a misrepresentation is the author tries to pass it out as experimental results, but it is not fake, i.e. it came from a real study and actually has some meaning (though not much if it is indeed an unstable structure that can just be observed in a computer, and it is ground for rejection depending on the journal). I mean, I don’t have much patience with people over-hyping their results, but fake data of the common sort is either photoshopped graphs, re-used spectra coming from other studies or compounds, or things like that. And this definitely happens. But the “it came from a simulation, therefore it’s fake” argument is very unhelpful in general. A spectrum from a simulation is not fake, it’s just a simulation result.

raverbashing 4y ago
It is fake if it's being misrepresented as experimental results instead of just a simulation
- kergonath 4y ago
  Again, that is not what this word means. It’s misrepresented data, not fake. And yes, the distinction matters. This is also not what the article is about. The problem they face is completely fabricated data, not DFT structures in a poorly worded paper. Of all places, this is probably the last one where I would expect this argument.
  - adgjlsfhk1 4y ago
    by this metric no data is fake. It's just misrepresented and was actually numbers I pulled out of my ass.
1vuio0pswjnm7 4y ago
It is arguable that it is more than a misrepresentation if the author's intent is to fool the reveiwers. If so, it is "fraudulent data".
DiggyJohnson 4y ago

muaytimbo 4y ago

I published a number of crystal structures using a powerful X ray diffractor and high quality crystals where H density was both found and modeled. Modeling H density in and of itself is in no way a sign of fake data, it is just a rare occurrence.

verisimi 4y ago
Hang on, isn't a model by necessity not the same as the thing it's modeling? Obviously a photo is sort of a model, but depending on what the subject is, you can visually verify it. If it's a software model, and say you enter lots of parameters, I have to wonder how valid that model is.. How can it be verified?
- ta988 4y ago
  The way you solve x-ray diffraction structures is that you refine your model until the electron density matches.
  - verisimi 4y ago
    Thanks.. But until your electron density matches what?
    - muaytimbo 4y ago
      You change the position and identity of the atoms in your model until there is very little residual electron density left remaining.
      - verisimi 4y ago
        These are words, but I don't see how they prove the model. I guess it's like some sort of check after the operation is complete? Why does a low residual electron density prove the model? What is electron density anyway? And how do we measure it?

ta988 4y ago

You can see the hydrogens if you have enough resolution (below 1A). These days you start to see such good resolutions pretty often.

ta988 4y ago
Here are two articles supporting my point as you don't seem to believe it. I also recommend you go look at PDB and select structures below 1A so you can see for yourself. https://bmcresnotes.biomedcentral.com/articles/10.1186/1756-0500-6-30 https://bmcresnotes.biomedcentral.com/articles/10.1186/1756-... https://www.science.org/doi/10.1126/sciadv.1600192 https://www.science.org/doi/10.1126/sciadv.1600192 """ Textbooks and teaching classes still emphasize that hydrogen atoms cannot be located with x-rays close to heavy elements; instead, neutron diffraction is needed. We show that, contrary to widespread expectation, hydrogen atoms can be located very accurately using x-ray diffraction, yielding bond lengths involving hydrogen atoms (A–H) that are in agreement with results from neutron diffraction mostly within a single standard deviation"""