5 ms·

>They took thousands of drugs where the precise chemical structure was known, and manually tested them on Acinetobacter baumanni... This information was fed int

by yyyk 3y ago

>They took thousands of drugs where the precise chemical structure was known, and manually tested them on Acinetobacter baumanni... This information was fed into the AI... The AI was then unleashed on a list of 6,680 compounds... took the AI an hour and a half to produce a shortlist... The researchers tested 240 in the laboratory, and found nine potential antibiotics. They started with manually testing thousands of drugs, in order to narrow another similarly sized list by one order of magnitude, which was then tested manually. Did they actually save time compared to what it would have taken to test the 6,680 list manually? I guess this needs to go up by one order of magnitude to be really worthwhile?

tehjoker 3y ago
Maybe it was easier to culture the original bug?
kurthr 3y ago
Hopefully, soon we can start treating all the cattle in the US with low doses of this drug, and dumping failed manufacturing batches in the Ganges river. Think of the poor superbugs!
- tohnjitor 3y ago
  Hopefully you are personally unable to implement such plans because antibiotics are not without their side effects. It took twelve years between Levaquin becoming FDA approved in 1996 and the addition of a black box warning label in 2008 due to risk of Achilles tendon rupture. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2483892/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2483892/
  - andrewnicolalde 3y ago
    I suspect they were being sarcastic.
    - tohnjitor 3y ago
      Hadn't even crossed my mind. I think you're right.
Thorrez 3y ago
Where does it say they manually tested thousands? It sounds to me like they manually tested 240.
- yyyk 3y ago
  The first sentence I quoted. They used thousands to train the AI, and then ran the AI on a similar sized list. Perhaps with reuse it will be worthwhile. Or the second list was harder to synthesize?
  - Thorrez 3y ago
    Oop, I need to read better...
- sp332 3y ago
  If you scroll down past the second photo, right after the heading "Artificial intelligence".
mcdonji 3y ago
I think the effort in the testing of the thousands of drugs was to help create the AI model. Then they used that model and it seems to have identified a promising antibiotic. The next time they go through the process they would not need to train the model again right? So get another big list of chemicals and run them through the model.
- geph2021 3y ago
  I think the effort in the testing of the thousands of drugs was to help create the AI model. This gets to the crux of my skepticism around the big claims around the pace of AI advancement. At a fundamental level the upper limit of AI advancement, in any area, is "the speed of information". For some areas, like pharmaceutical/drug development, the information comes from the real world, human/biological processes (e.g. clinical drug trials), which take time. At the extreme, the outcomes of interest could be long-term (i.e. years or decades). AI surely advances analytically capabilities, but ultimately models can only be developed or refined with new data/information, which unfolds at a rate that may be independent of computational speeds. AI models that are highly predictive and valuable by definition necessitates a feedback loop that is tied back to real-world outcomes/timescales. I'm no expert on AI, but I get this sense that the exponential improvements that many believe will lead to the singularity may in fact reach an inflection point where the curve flattens out becomes linear or asymptotic, as the rate of improvement is governed by the rate of new information in the real world.
  - chrisco255 3y ago
    Even for existing information, there remains an enormous amount of contextual / cultural / insider knowledge about the world that is not documented in any digestible way by an AI.
    - pixl97 3y ago
      For the moment. Things like gpt-4 are already multimodal, but not widely deployed in that fashion. Your data may just be a smart Webcam on wheels away from being ingested.
bb123 3y ago
They probably didn't save much time but you must think of the positive things mentioning the AI buzz word will have done for their funding.
- 0gravitas 3y ago
  And, of course, further research is needed…
throwuwu 3y ago
Now that they have a trained model they can continue to apply it to new chemicals. It could even be used on chemicals that have not been synthesized yet to check if they are useful and worth the effort and money to synthesize.
nestorD 3y ago
My guess is that they tested thousands of existing drugs (cheap to produce and easily available) in order to build a short list of new compound (significantly more expensive to produce).
adventured 3y ago
It's worthwhile to begin trying to use AI for these purposes. The only way we get from here to there, is by using this tech in its early forms today, exploring with it, experimenting with it. Let's see what it can do, learn, adjust, push it further. The 10x better version of it from the future doesn't just appear out of nowhere. We get there by step.
- joejerryronnie 3y ago
  I’m worried about the 10,000,000x version which may emerge organically at some point in time.
drabbiticus 3y ago
Science journalism frequently oversimplifies scientific investigation. This comment will similarly make simplifications, but hopefully overall will provide more clarity. > The AI was then unleashed on a list of 6,680 compounds whose effectiveness was unknown. The results - published in Nature Chemical Biology - showed it took the AI an hour and a half to produce a shortlist. "published in Nature Chemical Biology" is a link you have to click to see the article in fulltext, which I would encourage you to read if you really want to understand the study. I would link it directly, but there is some site-referrer magic happening that allows the BBC article link to cause Nature publishing group to show the fulltext. To better understand what was done: Training set (manually tested): off-patent drugs (2,341 molecules) and synthetic chemicals (5,343 molecules). In particular the synthetic chemicals are likely to have unacceptable side effect profiles. Result of manual screen: 480 molecules capable of inhibiting Acinetobacter growth by 20%. 480/(2341+5343) = 6-7% Result of AI processing and additional filtering criteria: model applied to "Drug Repurposing Hub" dataset consisting of 6,680 molecules which they claim have demonstrably favorable cytotoxicity profiles and drug-like properties and yielding 3 sets of 240 drugs each. The three sets: 1. 240 drugs identified by the model as having >20% probability of at least 20% growth inhibition of Acinetobacter and structurally _dissimilar_ to those with antibiotic activity in the training set. Manually testing for those capable of more stringent criteria (80% inhibition of growth) yielded 9 drugs. 2. 240 drugs with lowest prediction scores: Manual testing yields no active drugs, providing some basic validation that classifier is functional. 3. 240 drugs with highest prediction scores (without additional filtering criteria based on structural dissimilarity): Manual testing yields 40 drugs capable of 80% inhibition. 40/240 = 16-17%. Yield enrichment: 16%/6% = 260% or a 2.6x improvement compared to naive screen of training set. To be fair, this isn't a direct claim of the paper for good reason: the drugs in their training set and "validation set/drug repurposing hub" are fundamentally different and may have different baseline antibiotic activity across the set. The process of narrowing down these datasets using the model could be accomplished in hours (their claim) instead of days (my claim). Days is optimistic prediction, requiring high-throughput systems and/or staffing in place to run these screening assays mostly in parallel instead of serially. Also prevents costs associated with further biochemical investigation (cultures, chemical synthesis and assays are not free). Most direct value of this work is in accelerating drug screening process, reducing cost and developing AI-tractable representations of pharmaceutically-relevant chemical features. Additionally, proof of concept for identifying drugs with appropriate side effect profiles that happen to have antibiotic activity but would not have been identified with existing/common structural analysis approaches, since they are structurally dissimilar to the testing dataset screen. In this case they identified a "CCR2− selective chemokine receptor antagonist" that had antibiotic properties; some googling suggests that this drug class mostly has roles in fibrosis/inflammation regulation and may have roles in autoimmune disorders and those with significant fibrosis as part of the pathology (e.g. cardiovascular disease, liver disease, diabetes). You wouldn't expect most drugs in this class to have any antibiotic properties and many companies would not focus their first efforts on screening such drugs with biochemical assays.

by yyyk 3y ago

tehjoker 3y ago

Maybe it was easier to culture the original bug?

kurthr 3y ago

Hopefully, soon we can start treating all the cattle in the US with low doses of this drug, and dumping failed manufacturing batches in the Ganges river. Think of the poor superbugs!

tohnjitor 3y ago
Hopefully you are personally unable to implement such plans because antibiotics are not without their side effects. It took twelve years between Levaquin becoming FDA approved in 1996 and the addition of a black box warning label in 2008 due to risk of Achilles tendon rupture. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2483892/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2483892/
- andrewnicolalde 3y ago
  I suspect they were being sarcastic.
  - tohnjitor 3y ago
    Hadn't even crossed my mind. I think you're right.

Thorrez 3y ago

Where does it say they manually tested thousands? It sounds to me like they manually tested 240.

yyyk 3y ago
The first sentence I quoted. They used thousands to train the AI, and then ran the AI on a similar sized list. Perhaps with reuse it will be worthwhile. Or the second list was harder to synthesize?
- Thorrez 3y ago
  Oop, I need to read better...
sp332 3y ago
If you scroll down past the second photo, right after the heading "Artificial intelligence".

mcdonji 3y ago

I think the effort in the testing of the thousands of drugs was to help create the AI model. Then they used that model and it seems to have identified a promising antibiotic. The next time they go through the process they would not need to train the model again right? So get another big list of chemicals and run them through the model.

geph2021 3y ago
I think the effort in the testing of the thousands of drugs was to help create the AI model. This gets to the crux of my skepticism around the big claims around the pace of AI advancement. At a fundamental level the upper limit of AI advancement, in any area, is "the speed of information". For some areas, like pharmaceutical/drug development, the information comes from the real world, human/biological processes (e.g. clinical drug trials), which take time. At the extreme, the outcomes of interest could be long-term (i.e. years or decades). AI surely advances analytically capabilities, but ultimately models can only be developed or refined with new data/information, which unfolds at a rate that may be independent of computational speeds. AI models that are highly predictive and valuable by definition necessitates a feedback loop that is tied back to real-world outcomes/timescales. I'm no expert on AI, but I get this sense that the exponential improvements that many believe will lead to the singularity may in fact reach an inflection point where the curve flattens out becomes linear or asymptotic, as the rate of improvement is governed by the rate of new information in the real world.
- chrisco255 3y ago
  Even for existing information, there remains an enormous amount of contextual / cultural / insider knowledge about the world that is not documented in any digestible way by an AI.
  - pixl97 3y ago
    For the moment. Things like gpt-4 are already multimodal, but not widely deployed in that fashion. Your data may just be a smart Webcam on wheels away from being ingested.

bb123 3y ago

They probably didn't save much time but you must think of the positive things mentioning the AI buzz word will have done for their funding.

0gravitas 3y ago
And, of course, further research is needed…

throwuwu 3y ago

Now that they have a trained model they can continue to apply it to new chemicals. It could even be used on chemicals that have not been synthesized yet to check if they are useful and worth the effort and money to synthesize.

nestorD 3y ago

My guess is that they tested thousands of existing drugs (cheap to produce and easily available) in order to build a short list of new compound (significantly more expensive to produce).

adventured 3y ago

It's worthwhile to begin trying to use AI for these purposes. The only way we get from here to there, is by using this tech in its early forms today, exploring with it, experimenting with it. Let's see what it can do, learn, adjust, push it further. The 10x better version of it from the future doesn't just appear out of nowhere. We get there by step.

joejerryronnie 3y ago
I’m worried about the 10,000,000x version which may emerge organically at some point in time.

drabbiticus 3y ago

Science journalism frequently oversimplifies scientific investigation. This comment will similarly make simplifications, but hopefully overall will provide more clarity. > The AI was then unleashed on a list of 6,680 compounds whose effectiveness was unknown. The results - published in Nature Chemical Biology - showed it took the AI an hour and a half to produce a shortlist. "published in Nature Chemical Biology" is a link you have to click to see the article in fulltext, which I would encourage you to read if you really want to understand the study. I would link it directly, but there is some site-referrer magic happening that allows the BBC article link to cause Nature publishing group to show the fulltext. To better understand what was done: Training set (manually tested): off-patent drugs (2,341 molecules) and synthetic chemicals (5,343 molecules). In particular the synthetic chemicals are likely to have unacceptable side effect profiles. Result of manual screen: 480 molecules capable of inhibiting Acinetobacter growth by 20%. 480/(2341+5343) = 6-7% Result of AI processing and additional filtering criteria: model applied to "Drug Repurposing Hub" dataset consisting of 6,680 molecules which they claim have demonstrably favorable cytotoxicity profiles and drug-like properties and yielding 3 sets of 240 drugs each. The three sets: 1. 240 drugs identified by the model as having >20% probability of at least 20% growth inhibition of Acinetobacter and structurally _dissimilar_ to those with antibiotic activity in the training set. Manually testing for those capable of more stringent criteria (80% inhibition of growth) yielded 9 drugs. 2. 240 drugs with lowest prediction scores: Manual testing yields no active drugs, providing some basic validation that classifier is functional. 3. 240 drugs with highest prediction scores (without additional filtering criteria based on structural dissimilarity): Manual testing yields 40 drugs capable of 80% inhibition. 40/240 = 16-17%. Yield enrichment: 16%/6% = 260% or a 2.6x improvement compared to naive screen of training set. To be fair, this isn't a direct claim of the paper for good reason: the drugs in their training set and "validation set/drug repurposing hub" are fundamentally different and may have different baseline antibiotic activity across the set. The process of narrowing down these datasets using the model could be accomplished in hours (their claim) instead of days (my claim). Days is optimistic prediction, requiring high-throughput systems and/or staffing in place to run these screening assays mostly in parallel instead of serially. Also prevents costs associated with further biochemical investigation (cultures, chemical synthesis and assays are not free). Most direct value of this work is in accelerating drug screening process, reducing cost and developing AI-tractable representations of pharmaceutically-relevant chemical features. Additionally, proof of concept for identifying drugs with appropriate side effect profiles that happen to have antibiotic activity but would not have been identified with existing/common structural analysis approaches, since they are structurally dissimilar to the testing dataset screen. In this case they identified a "CCR2− selective chemokine receptor antagonist" that had antibiotic properties; some googling suggests that this drug class mostly has roles in fibrosis/inflammation regulation and may have roles in autoimmune disorders and those with significant fibrosis as part of the pathology (e.g. cardiovascular disease, liver disease, diabetes). You wouldn't expect most drugs in this class to have any antibiotic properties and many companies would not focus their first efforts on screening such drugs with biochemical assays.