8 ms·

Call for a Public Open Database of All Chemical Reactions

152 points by cbracketdash 4y ago

jerven 4y ago
This is an important point, a lot of this data is practically unfindable and databases like Rhea [1] for reactions of biological interest are also barely funded. Generating and maintaining these databases is not very expensive but not sexy for funders :( [1] https://www.rhea-db.org/ https://www.rhea-db.org/
- greggsy 4y ago
  If I t’s effectively a Wikipedia for chemical interactions, then most visitors would probably skip past Jimmy Wales’ appeal banner.
5d8767c68926 4y ago
Non-pdf link: https://doi.org/10.1021/acs.jcim.1c01140 https://doi.org/10.1021/acs.jcim.1c01140
entee 4y ago
Such a database would be hugely helpful across chemistry. Right now it’s extremely expensive to access databases like Reaxys or Scifinder, and they’re not usually programmatically searchable at scale. Some databases do exist based on the patent literature (https://depth-first.com/articles/2019/01/28/the-nextmove-patent-reaction-dataset/ https://depth-first.com/articles/2019/01/28/the-nextmove-pat...) but they’re not as well curated or complete. A pubchem like database for reactions would be really awesome.
deleted 4y ago
[deleted]
kurupt213 4y ago
Basically all organic synthesis pathways get published in the organic syntheses volumes (they’re online at orgsyn.org)
gilleain 4y ago
Nice paper. An interesting paragraph: > This conservative state of affairs in the chemistry community is unlikely to result from some intrinsic properties of chemistry as a science. Rather, it is likely to be the product of complex historical and sociological factors, which can be traced back at least to the Middle Ages and the secretive research projects of the alchemists in their quest for a recipe for converting vulgar metals to gold. Yes, I blame Hermes Trismegistus, Abū Mūsā Jābir ibn Ḥayyān, and Isaac Newton :)
- jona-f 4y ago
  Sounds somewhat plausible, yes, but i don't buy it. Math used to be absurdly secretive too, but changed to everyone's benefit. I blame the financial profit to be had with exclusive knowledge of a reaction and it's workup.
jona-f 4y ago
This is exciting. Chemistry is so stuck in the old ways, maybe things are finally changing. And i don't think is has to be expensive. Someone has to come up with a simple protocol to declare a chemical reaction (inputs, outputs, temp, pressure, time, yield, link to paper) and then it must become fashionable to publish this data with your paper. The resulting database shouldn't be too big for all chemical reactions. Unfortunately published data so far is highly questionable. I can totally confirm this quote from wikipedia: "A 2016 survey by Nature on 1,576 researchers [..] on reproducibility found that more than 70% of researchers have tried and failed to reproduce another scientist's experiment results (including 87% of chemists, 77% of biologists, 69% of physicists and engineers [...]" https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis
- Swizec 4y ago
  I remember talking to a friend who was doing a PhD in microbiology (implementing NAND gates in bacterial DNA) and she said reproducibility in chemistry/biology is a major pain. They couldn’t even reliably reproduce experiments within the same lab. You’d have one postdoc do the pipetting and the experiment worked. Another person tried, following the same procedures with the same equipment in the same room at the same time of day in the same environment … experiment failed. My takeaway was that a lot of this bio stuff is inherently flakey and/or poorly understood. We kinda mostly know what’s going but it’s really lots of trial and error and chasing statistical variations/effects. Best you can do is “it works N% of the time”
  - myst1 4y ago
    That's mostly true in the biological sciences (biochem included), less so in chemistry.
- xnorswap 4y ago
  It's worth noting here that % of researchers and % of attempts are very very different things. You can have a system where 100% of researchers have failed to replicate another experimental result while 99.999% of results are replicable. That's not to suggest that's what's actually happening here, because the replication crisis is well discussed, but it's not a good statistic to measure things with.
myst1 4y ago
Although this is a good idea. It is a monstrous task. A basic exercise in combinatorics would show that this is difficult. I can think of technologies capable of making this happen, but it could be expensive to complete. Sounds like a good project but could lead to some social/communal harm (it wasn't in the db so I invented it...). Would be a hell of a lot of fun to try to do though.
- gilleain 4y ago
  Ah, I do not think they are proposing a database of _predicted_ chemical reactions but of _known_ reactions. I absolutely agree that enumerating reactions would suffer from combinatorial explosion (as opposed to chemical explosions, hoho). However there have been some efforts in what I know find is called 'computational retrosynthesis' (I think): https://www.chemistryworld.com/features/computer-guided-retrosynthesis/4013627.article https://www.chemistryworld.com/features/computer-guided-retr... Of course, any 'prediction' of a reaction relies on some model to give you an idea of how feasible it is to do in the lab, so I'm less worried about . There's the classic Derek Lowe post (that I've seen on here before, but still) about 'FOOF': https://www.science.org/content/blog-post/things-i-won-t-work-dioxygen-difluoride https://www.science.org/content/blog-post/things-i-won-t-wor...
  - myst1 4y ago
    Known reactions is also a very large space. Everyday dozens of papers get published enlisting many reactions on a good deal of different starting materials. Guess it depends on what you want out of it.
surprisetalk 4y ago
I'm a database expert willing to put some time into this. Please email me at surprisetalk@gmail.com if you (or anybody you know) would be interested in collaborating.
- myst1 4y ago
  Majority of the problem here isn't a database problem ironically. It's a scraping, cleaning, and organizing problem. That said I would also be interested in helping.
jka 4y ago
Question for someone with knowledge of (computational?) chemistry: is MIT's Reaction Mechanism Generator[1] software (and associated database[2]) a step in this direction? [1] - https://reactionmechanismgenerator.github.io/RMG-Py/index.html https://reactionmechanismgenerator.github.io/RMG-Py/index.ht... [2] - https://rmg.mit.edu/database/ https://rmg.mit.edu/database/
- gilleain 4y ago
  As with the other question, above - I think this paper proposes a database of known (experimentally verified?) reactions, rather than predicted ones. Guess they should have been clearer. RMG predicts likely reactions based on chemical kinetics, which might also make a useful database itself - or link to the experimental one, of course.
  - jka 4y ago
    Got it, that makes sense - thank you!
apienx 4y ago
"Alchemists turned into chemists when they stopped keeping secrets." -- Eric S. Raymond
- contingencies 4y ago
  Haha, awesome. Added to https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup
  - carapace 4y ago
    It sounds good but it's not historically accurate.
    - contingencies 4y ago
      In war, the first casualty is truth. More seriously, acts of communication can have value outside of an objective truth: hypotheses, art, rules of thumb.
JudgePenitent 4y ago
"In short, machine learning methods are in place, but accessible data is not." "Finally, one must address the concerns of various commercial stakeholders." "A few chemical databases of reactions do exist, but these are commercial (e.g., Beilstein/Reaxys [Elsevier], SPRESI [InfoChem], and CAS [ACS]), and even when a license is purchased, the underlying data are accessible only through a narrow, one-query-at-a-time interface, completely stifling the application of powerful artificial intelligence and machine [...]" This seems like the first point of approach, not the last. Can anyone comment why these data gatekeepers have not made a business model to give programmatic access to nerds? And if they have financial reasons for not doing so, how will governments sweet talk shareholders into supposedly losing money?
- myst1 4y ago
  Ignoring machine learning entirely... Chemists benefit from this. Being able to type in a moiety and the product you want saves a bunch of trouble...
contingencies 4y ago
So the next stages here are clearly to add functional costs (time, space, equipment) to the reactions and obtain a similar database of supply chain costs and legal hassle for obtaining reactants at various volumes then you can objectively predict with high confidence interval projected market returns on various syntheses. Thus, should a given reactant become expensive or unavailable this would immediately result in a plethora of novel alternate paths. Of course, our entire global economic system is heavily weighted against doing something so useful in public, for the public good...

152 points by cbracketdash 4y ago

This is an important point, a lot of this data is practically unfindable and databases like Rhea [1] for reactions of biological interest are also barely funded. Generating and maintaining these databases is not very expensive but not sexy for funders :( [1] https://www.rhea-db.org/ https://www.rhea-db.org/

greggsy 4y ago
If I t’s effectively a Wikipedia for chemical interactions, then most visitors would probably skip past Jimmy Wales’ appeal banner.

5d8767c68926 4y ago

Non-pdf link: https://doi.org/10.1021/acs.jcim.1c01140 https://doi.org/10.1021/acs.jcim.1c01140

entee 4y ago

Such a database would be hugely helpful across chemistry. Right now it’s extremely expensive to access databases like Reaxys or Scifinder, and they’re not usually programmatically searchable at scale. Some databases do exist based on the patent literature (https://depth-first.com/articles/2019/01/28/the-nextmove-patent-reaction-dataset/ https://depth-first.com/articles/2019/01/28/the-nextmove-pat...) but they’re not as well curated or complete. A pubchem like database for reactions would be really awesome.

deleted 4y ago

[deleted]

kurupt213 4y ago

Basically all organic synthesis pathways get published in the organic syntheses volumes (they’re online at orgsyn.org)

gilleain 4y ago

Nice paper. An interesting paragraph: > This conservative state of affairs in the chemistry community is unlikely to result from some intrinsic properties of chemistry as a science. Rather, it is likely to be the product of complex historical and sociological factors, which can be traced back at least to the Middle Ages and the secretive research projects of the alchemists in their quest for a recipe for converting vulgar metals to gold. Yes, I blame Hermes Trismegistus, Abū Mūsā Jābir ibn Ḥayyān, and Isaac Newton :)

jona-f 4y ago
Sounds somewhat plausible, yes, but i don't buy it. Math used to be absurdly secretive too, but changed to everyone's benefit. I blame the financial profit to be had with exclusive knowledge of a reaction and it's workup.

jona-f 4y ago

This is exciting. Chemistry is so stuck in the old ways, maybe things are finally changing. And i don't think is has to be expensive. Someone has to come up with a simple protocol to declare a chemical reaction (inputs, outputs, temp, pressure, time, yield, link to paper) and then it must become fashionable to publish this data with your paper. The resulting database shouldn't be too big for all chemical reactions. Unfortunately published data so far is highly questionable. I can totally confirm this quote from wikipedia: "A 2016 survey by Nature on 1,576 researchers [..] on reproducibility found that more than 70% of researchers have tried and failed to reproduce another scientist's experiment results (including 87% of chemists, 77% of biologists, 69% of physicists and engineers [...]" https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis

Swizec 4y ago
I remember talking to a friend who was doing a PhD in microbiology (implementing NAND gates in bacterial DNA) and she said reproducibility in chemistry/biology is a major pain. They couldn’t even reliably reproduce experiments within the same lab. You’d have one postdoc do the pipetting and the experiment worked. Another person tried, following the same procedures with the same equipment in the same room at the same time of day in the same environment … experiment failed. My takeaway was that a lot of this bio stuff is inherently flakey and/or poorly understood. We kinda mostly know what’s going but it’s really lots of trial and error and chasing statistical variations/effects. Best you can do is “it works N% of the time”
- myst1 4y ago
  That's mostly true in the biological sciences (biochem included), less so in chemistry.
xnorswap 4y ago
It's worth noting here that % of researchers and % of attempts are very very different things. You can have a system where 100% of researchers have failed to replicate another experimental result while 99.999% of results are replicable. That's not to suggest that's what's actually happening here, because the replication crisis is well discussed, but it's not a good statistic to measure things with.

myst1 4y ago

Although this is a good idea. It is a monstrous task. A basic exercise in combinatorics would show that this is difficult. I can think of technologies capable of making this happen, but it could be expensive to complete. Sounds like a good project but could lead to some social/communal harm (it wasn't in the db so I invented it...). Would be a hell of a lot of fun to try to do though.

gilleain 4y ago
Ah, I do not think they are proposing a database of _predicted_ chemical reactions but of _known_ reactions. I absolutely agree that enumerating reactions would suffer from combinatorial explosion (as opposed to chemical explosions, hoho). However there have been some efforts in what I know find is called 'computational retrosynthesis' (I think): https://www.chemistryworld.com/features/computer-guided-retrosynthesis/4013627.article https://www.chemistryworld.com/features/computer-guided-retr... Of course, any 'prediction' of a reaction relies on some model to give you an idea of how feasible it is to do in the lab, so I'm less worried about . There's the classic Derek Lowe post (that I've seen on here before, but still) about 'FOOF': https://www.science.org/content/blog-post/things-i-won-t-work-dioxygen-difluoride https://www.science.org/content/blog-post/things-i-won-t-wor...
- myst1 4y ago
  Known reactions is also a very large space. Everyday dozens of papers get published enlisting many reactions on a good deal of different starting materials. Guess it depends on what you want out of it.

surprisetalk 4y ago

I'm a database expert willing to put some time into this. Please email me at surprisetalk@gmail.com if you (or anybody you know) would be interested in collaborating.

myst1 4y ago
Majority of the problem here isn't a database problem ironically. It's a scraping, cleaning, and organizing problem. That said I would also be interested in helping.

jka 4y ago

Question for someone with knowledge of (computational?) chemistry: is MIT's Reaction Mechanism Generator[1] software (and associated database[2]) a step in this direction? [1] - https://reactionmechanismgenerator.github.io/RMG-Py/index.html https://reactionmechanismgenerator.github.io/RMG-Py/index.ht... [2] - https://rmg.mit.edu/database/ https://rmg.mit.edu/database/

gilleain 4y ago
As with the other question, above - I think this paper proposes a database of known (experimentally verified?) reactions, rather than predicted ones. Guess they should have been clearer. RMG predicts likely reactions based on chemical kinetics, which might also make a useful database itself - or link to the experimental one, of course.
- jka 4y ago
  Got it, that makes sense - thank you!

apienx 4y ago

"Alchemists turned into chemists when they stopped keeping secrets." -- Eric S. Raymond

contingencies 4y ago
Haha, awesome. Added to https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup
- carapace 4y ago
  It sounds good but it's not historically accurate.
  - contingencies 4y ago
    In war, the first casualty is truth. More seriously, acts of communication can have value outside of an objective truth: hypotheses, art, rules of thumb.

JudgePenitent 4y ago

"In short, machine learning methods are in place, but accessible data is not." "Finally, one must address the concerns of various commercial stakeholders." "A few chemical databases of reactions do exist, but these are commercial (e.g., Beilstein/Reaxys [Elsevier], SPRESI [InfoChem], and CAS [ACS]), and even when a license is purchased, the underlying data are accessible only through a narrow, one-query-at-a-time interface, completely stifling the application of powerful artificial intelligence and machine [...]" This seems like the first point of approach, not the last. Can anyone comment why these data gatekeepers have not made a business model to give programmatic access to nerds? And if they have financial reasons for not doing so, how will governments sweet talk shareholders into supposedly losing money?

myst1 4y ago
Ignoring machine learning entirely... Chemists benefit from this. Being able to type in a moiety and the product you want saves a bunch of trouble...

contingencies 4y ago

So the next stages here are clearly to add functional costs (time, space, equipment) to the reactions and obtain a similar database of supply chain costs and legal hassle for obtaining reactants at various volumes then you can objectively predict with high confidence interval projected market returns on various syntheses. Thus, should a given reactant become expensive or unavailable this would immediately result in a plethora of novel alternate paths. Of course, our entire global economic system is heavily weighted against doing something so useful in public, for the public good...