12 ms·

>First, AI can quickly read through the scientific literature, allowing it to learn the fundamental rules, facts, and equations of science, and help scientists

by mateo1 4y ago

>First, AI can quickly read through the scientific literature, allowing it to learn the fundamental rules, facts, and equations of science, and help scientists manage the flood of papers and data that is drowning every field. That's the first in a series of bs claims. AI tools can, in time, bring an engineering revolution but a new scientific dark age whereby idiots with little to no domain knowledge use "ai" tools such as the super resolution ones mentioned, draw wrong conclusions and then publish them so other idiots can do flawed ai-powered (or not) meta-analysis and further damage the state of concensus and integrity in their field.

visarga 4y ago
Isn't that how we use all technology? As soon as we abstract something away we are susceptible to modelling problems. Even speech is a poor approximation of what we sometimes want to express - another imperfect model shoehorned on reality.
walnutclosefarm 4y ago
I'm missing how you own unsupported claim in any way refutes the claim in the article that you assert to be bs. I have in fact been involved in projects that did exactly what you claim is BS - read a large corpus of scientific papers (and written medical records), and discerned connections worth further scientific investigation that had not been detected by the body of scientists following the field, ultimately leading to new drug discovery science.
- AnimalMuppet 4y ago
  "Worth further scientific investigation" is one thing. But mateo1 called BS on the claim that AI could "learn the fundamental rules, facts, and equations of science" by reading the literature, which is a different claim. You claim that the AI can find patterns and correlations; mateo1 disputes that the AI can actually learn the rules of the domain from the papers. So while you are right that mateo1 also did not prove his claim, your experience doesn't disprove his claim.
  - visarga 4y ago
    Depends on what you mean by "learn the rules, facts, and equations of science". I mean, a language model could memorise it and retell it in poetry if you wanted. On the other hand it's not just a parrot, as some have claimed. AI is useful in accelerating progress in hard scientific problems. For one - a neural network is a universal function approximator. That's got to be really useful in many places where an approximation is good enough.
    - mdp2021 4y ago
      I used in this pages the expression "parroting" for some AI implementations. Let us disambiguate. It is called an Artificial Intelligence that engineering product that can replace a Natural Intelligence in finding solutions - such as, through the automated development of sophisticated function approximators. This is one thing, and akin to the content of the article. And it is one meaning of the term "intelligence" ("intelligence₂"). On the other hand, I have seen algorithms that appear to collect associations by frequency being called Intelligence, in a very different sense of the term, and to that I note: -- that Intelligence₁ is an ontology developer - in my terms; it "investigates what things are". In the very recent terms of James Fodor, which I present here because it is a different formulation by coincidence just published today, «humans learn by making structured mental concepts, in which many different properties and associations are linked together» [actually the crucial part is in /how/ the concepts are refined]; -- and that without operation of "critical reflection" crucial in the ontology development, and instead with a "tendency" to just trust the input and solve conflict without foundational considerations, an inputs-imitating system, which may be said "based on parroting", is the opposite of Intelligence₁ .
      - visarga 4y ago
        Very few people are capable of "investigating what things are" with "critical reflection", usually trained professionals in their field of expertise. Most people think by correlation and adjust based on the feedback. I think with the right feedback an AI could get by with parroting and correlations, and parroting becomes critical reflexion at some threshold of complexity.
andrewmutz 4y ago
I agree that this line is BS. I disagree that it discredits the whole article. In particular, the protein folding example is extremely important and validates that AI is indeed having a big and real impact on science.
- visarga 4y ago
  Why is it BS. AI allows scientists to zoom into information they seek. For example by training a language model on the latest papers, a scientist could use question & answering to find answers to specific questions. It's similar to using a search engine.
  - Swizec 4y ago
    Sounds to me like what we call "A Search Engine" in software engineering. I sure as heck can't keep track of all the blogposts and articles that come out every day about software development. But I can expertly wield "an AI" to answer my questions and do just-in-time research when it's needed to solve problems.
  - andrewmutz 4y ago
    I've never seen anything in the AI space that can "quickly read through the scientific literature, allowing it to learn the fundamental rules, facts, and equations of science".
    - randcraw 4y ago
      Yeah, that's exactly what IBM claimed Watson would do -- learn biology, chemistry, and medicine on its own and then outperform human physicians. Instead, Watson Medicine was a catastrophe and revealed their AI marketeers to be hopelessly clueless. That's a mistake all AI boosters should take to heart. Don't overstate your accomplishment. To wit, claims that the protein folding problem has been entirely solved by AlphaFold is such an overstatement. For a more measured view on the use of AF in drug development (especially), check out med chemist Derek Lowe's blog, especially his articles: "Fooling the Protein Folding Software" https://www.science.org/content/blog-post/fooling-protein-folding-software https://www.science.org/content/blog-post/fooling-protein-fo... , and "AlphaFold Excitement" https://www.science.org/content/blog-post/alphafold-excitement https://www.science.org/content/blog-post/alphafold-exciteme...
andreyk 4y ago
How is this BS? "How AI technology can tame the scientific literature" https://www.nature.com/articles/d41586-018-06617-5 https://www.nature.com/articles/d41586-018-06617-5 "SCIBERT: A Pretrained Language Model for Scientific Text" https://arxiv.org/pdf/1903.10676.pdf https://arxiv.org/pdf/1903.10676.pdf "Domain-specific language model pretraining for biomedical natural language processing" https://dl.acm.org/doi/pdf/10.1145/3458754 https://dl.acm.org/doi/pdf/10.1145/3458754 These are just examples I pulled up in a couple minutes of Googling. Yeah, the wording here may be a little too grandiose, but nevertheless the fact that AI can read through vast sets of papers and data quickly and help humans sort through it is fundamentally true.
- YeGoblynQueenne 4y ago
  Can I ask? Did you read those three papers carefully before linking them? I'm asking because a scientific paper's purpose is to make one or more claims (but usually one central claim), that are supported in the paper by theoretical and empirical results. It is hard to know whether any particular claim is true, even when reading the paper carefully and scrutinising, or even attempting to replicate, its results - let alone otherwise.
narrator 4y ago
Right now we can look at DALL-E and say that it did a good job or it didn't because it is producing familiar objects like teddy bears or whatever. If the science equivalent of DALL-E said this is a process to extract an enantiomer with right chirality and it's actually is for left chirality who is going to know it made a mistake except a trained chemist? Thus, double checking the work of the AI is going to be a lot of the work of scientists in the future.
- pessimizer 4y ago
  > Thus, double checking the work of the AI is going to be a lot of the work of scientists in the future. That seems similar to the idea of self-driving cars that screw up frequently and have to suddenly turn over control to the driver being safer than just paying attention to driving. Peer review sucks now. Reviewing something that makes errors that no human would make seems like it would be a lot worse.
  - hypertele-Xii 4y ago
    On the other hand, such AI will usher in a new era of art and comedy. Watching computers fail in absurd ways.
    - mdp2021 4y ago
      Different is the laugh, when slapstick occurs on a stage, versus when your doctor had bad ideas. This stresses the call for humans not to be lightminded and decadent: new instruments are around, and they require to be managed competently and properly.
version_five 4y ago
I agree with your comment, though I've found it's hard to defend statements like that, people can always pull out some headlines (that are easy to find, like ths guy telling you that all he had to do was google) and claim that makes you wrong and AI really is revolutionary. It's been overhyped so much that it's very hard to convince non specialists that basically all the framing of AI they can find is hype. The best I can say is, show me a company that's actually making money off of a product that's based around one of these claims, that actually has the ML as a core differentiator. No doubt people will still find headlines, but they'll be more scarce, and if you look into the commercial success of such companies, you'll find almost nothing. Disclaimer, I run an AI company and am an AI bull. Misplaced hype helps charlatans and scammers

by mateo1 4y ago

visarga 4y ago

Isn't that how we use all technology? As soon as we abstract something away we are susceptible to modelling problems. Even speech is a poor approximation of what we sometimes want to express - another imperfect model shoehorned on reality.

walnutclosefarm 4y ago

I'm missing how you own unsupported claim in any way refutes the claim in the article that you assert to be bs. I have in fact been involved in projects that did exactly what you claim is BS - read a large corpus of scientific papers (and written medical records), and discerned connections worth further scientific investigation that had not been detected by the body of scientists following the field, ultimately leading to new drug discovery science.

AnimalMuppet 4y ago
"Worth further scientific investigation" is one thing. But mateo1 called BS on the claim that AI could "learn the fundamental rules, facts, and equations of science" by reading the literature, which is a different claim. You claim that the AI can find patterns and correlations; mateo1 disputes that the AI can actually learn the rules of the domain from the papers. So while you are right that mateo1 also did not prove his claim, your experience doesn't disprove his claim.
- visarga 4y ago
  Depends on what you mean by "learn the rules, facts, and equations of science". I mean, a language model could memorise it and retell it in poetry if you wanted. On the other hand it's not just a parrot, as some have claimed. AI is useful in accelerating progress in hard scientific problems. For one - a neural network is a universal function approximator. That's got to be really useful in many places where an approximation is good enough.
  - mdp2021 4y ago
    I used in this pages the expression "parroting" for some AI implementations. Let us disambiguate. It is called an Artificial Intelligence that engineering product that can replace a Natural Intelligence in finding solutions - such as, through the automated development of sophisticated function approximators. This is one thing, and akin to the content of the article. And it is one meaning of the term "intelligence" ("intelligence₂"). On the other hand, I have seen algorithms that appear to collect associations by frequency being called Intelligence, in a very different sense of the term, and to that I note: -- that Intelligence₁ is an ontology developer - in my terms; it "investigates what things are". In the very recent terms of James Fodor, which I present here because it is a different formulation by coincidence just published today, «humans learn by making structured mental concepts, in which many different properties and associations are linked together» [actually the crucial part is in /how/ the concepts are refined]; -- and that without operation of "critical reflection" crucial in the ontology development, and instead with a "tendency" to just trust the input and solve conflict without foundational considerations, an inputs-imitating system, which may be said "based on parroting", is the opposite of Intelligence₁ .
    - visarga 4y ago
      Very few people are capable of "investigating what things are" with "critical reflection", usually trained professionals in their field of expertise. Most people think by correlation and adjust based on the feedback. I think with the right feedback an AI could get by with parroting and correlations, and parroting becomes critical reflexion at some threshold of complexity.

andrewmutz 4y ago

I agree that this line is BS. I disagree that it discredits the whole article. In particular, the protein folding example is extremely important and validates that AI is indeed having a big and real impact on science.

visarga 4y ago
Why is it BS. AI allows scientists to zoom into information they seek. For example by training a language model on the latest papers, a scientist could use question & answering to find answers to specific questions. It's similar to using a search engine.
- Swizec 4y ago
  Sounds to me like what we call "A Search Engine" in software engineering. I sure as heck can't keep track of all the blogposts and articles that come out every day about software development. But I can expertly wield "an AI" to answer my questions and do just-in-time research when it's needed to solve problems.
- andrewmutz 4y ago
  I've never seen anything in the AI space that can "quickly read through the scientific literature, allowing it to learn the fundamental rules, facts, and equations of science".
  - randcraw 4y ago
    Yeah, that's exactly what IBM claimed Watson would do -- learn biology, chemistry, and medicine on its own and then outperform human physicians. Instead, Watson Medicine was a catastrophe and revealed their AI marketeers to be hopelessly clueless. That's a mistake all AI boosters should take to heart. Don't overstate your accomplishment. To wit, claims that the protein folding problem has been entirely solved by AlphaFold is such an overstatement. For a more measured view on the use of AF in drug development (especially), check out med chemist Derek Lowe's blog, especially his articles: "Fooling the Protein Folding Software" https://www.science.org/content/blog-post/fooling-protein-folding-software https://www.science.org/content/blog-post/fooling-protein-fo... , and "AlphaFold Excitement" https://www.science.org/content/blog-post/alphafold-excitement https://www.science.org/content/blog-post/alphafold-exciteme...

andreyk 4y ago

How is this BS? "How AI technology can tame the scientific literature" https://www.nature.com/articles/d41586-018-06617-5 https://www.nature.com/articles/d41586-018-06617-5 "SCIBERT: A Pretrained Language Model for Scientific Text" https://arxiv.org/pdf/1903.10676.pdf https://arxiv.org/pdf/1903.10676.pdf "Domain-specific language model pretraining for biomedical natural language processing" https://dl.acm.org/doi/pdf/10.1145/3458754 https://dl.acm.org/doi/pdf/10.1145/3458754 These are just examples I pulled up in a couple minutes of Googling. Yeah, the wording here may be a little too grandiose, but nevertheless the fact that AI can read through vast sets of papers and data quickly and help humans sort through it is fundamentally true.

YeGoblynQueenne 4y ago
Can I ask? Did you read those three papers carefully before linking them? I'm asking because a scientific paper's purpose is to make one or more claims (but usually one central claim), that are supported in the paper by theoretical and empirical results. It is hard to know whether any particular claim is true, even when reading the paper carefully and scrutinising, or even attempting to replicate, its results - let alone otherwise.

narrator 4y ago

Right now we can look at DALL-E and say that it did a good job or it didn't because it is producing familiar objects like teddy bears or whatever. If the science equivalent of DALL-E said this is a process to extract an enantiomer with right chirality and it's actually is for left chirality who is going to know it made a mistake except a trained chemist? Thus, double checking the work of the AI is going to be a lot of the work of scientists in the future.

pessimizer 4y ago
> Thus, double checking the work of the AI is going to be a lot of the work of scientists in the future. That seems similar to the idea of self-driving cars that screw up frequently and have to suddenly turn over control to the driver being safer than just paying attention to driving. Peer review sucks now. Reviewing something that makes errors that no human would make seems like it would be a lot worse.
- hypertele-Xii 4y ago
  On the other hand, such AI will usher in a new era of art and comedy. Watching computers fail in absurd ways.
  - mdp2021 4y ago
    Different is the laugh, when slapstick occurs on a stage, versus when your doctor had bad ideas. This stresses the call for humans not to be lightminded and decadent: new instruments are around, and they require to be managed competently and properly.

version_five 4y ago

I agree with your comment, though I've found it's hard to defend statements like that, people can always pull out some headlines (that are easy to find, like ths guy telling you that all he had to do was google) and claim that makes you wrong and AI really is revolutionary. It's been overhyped so much that it's very hard to convince non specialists that basically all the framing of AI they can find is hype. The best I can say is, show me a company that's actually making money off of a product that's based around one of these claims, that actually has the ML as a core differentiator. No doubt people will still find headlines, but they'll be more scarce, and if you look into the commercial success of such companies, you'll find almost nothing. Disclaimer, I run an AI company and am an AI bull. Misplaced hype helps charlatans and scammers