26 ms·

AI Is Ushering in a New Scientific Revolution

94 points by spansoa 4y ago

lettergram 4y ago
Current AI methods are great for classification. That said, I’m not convinced AI is making discoveries. What I mean by that, is that AI is often non-explainable, can be non-deterministic and only as good as it’s training data. For instance, asteroid discovery. Yes AI can be used to discover asteroids. BUT every asteroid needs to be checked by a human and some will undoubtably be false positives. We also miss some unknown true positives. We often (depending on method) can’t explain why [in-fact] why something is a true positive vs false positive from the network itself. Which is why I say AI doesn’t make discoveries. It’s not identifying something knew, it’s identifying something that looks like what it’s seen before. It’s error rates in the scientific frontier is also going to be unknown; as we often lack data. Using AI could potentially limit the scope of new discoveries - depending how heavily it’s relied on by hyper optimizing on the wrong items. Anyway, sorry for the rambling, as a research in deep learning I find this trend something to monitor.
- sumy23 4y ago
  I disagree. Yes, results need to be validated, but that doesn't mean ML is not making discoveries. Even for basic classification tasks, discoveries are still being made. For instance, ML can reliably detect the race of a person from an x-ray. There is clearly some anatomical difference that we're not aware of that's detectable in an x-ray. Most of human inference, in general, is "connecting the dots" between related phenomena. ML is a great tool for doing this kind of correlative work at scale.
  - omgwtfbyobbq 4y ago
    I think we need to be cautious about what ML is picking up. Finding patterns is great, but we need to be fairly sure about what those patterns are so we don't misapply them in some different context we don't understand. As for whether finding patterns is discovery, I feel very whatever about it.
  - andi999 4y ago
    I remember the paper and the conclusion you say is doubtful. Even when the data was heavily distorted and resolution reduced to insanely big pixels it still could do this. So, since magic does not exist, there is probably a hidden cofactor (like xrays done on different machines, one in a hospital in a white neighbourhood, the other in a black one, and the machine giving a technical artifact like colorrange). This is just an example of course, I dont know the cofactor. Here is the study:https://www.thelancet.com/journals/landig/article/PIIS2589-7500(22)00063-2/fulltext https://www.thelancet.com/journals/landig/article/PIIS2589-7... When i said insanely i didnt remember correctly, just saw even if reduce to 4x4 pixels they claim a statistical difference. So mind blowingly ludicrous resolution might have been a better term.
    - walnutclosefarm 4y ago
      I agree that there is something odd going on in the race study. However, I've seen analogous "discovery" results that don't have this problem. Scientists at my former employer built a model, e.g., that diagnosed asymptomatic heart failure from ECGs. No electrophysiologist in that world-class cardiac medicine unit can do this, but the algorithm clearly perceives the signal. And this was in fact a true "discovery" in that they didn't set out to diagnose heart failure, but rather to "score" ventricular ejection fraction using ECGs. Training the algorithm revealed a signal (a so far unexplained cofactor to reduced ejection fraction) that predicts reduced ejection fraction, and thus heart failure, prior to grossly measurable reduced ejection fraction.
      - andi999 4y ago
        Nice. I think this is very interesting, is it published?
        walnutclosefarm 4y ago
        https://www.nature.com/articles/s41591-021-01335-4 https://www.nature.com/articles/s41591-021-01335-4
  - idontpost 4y ago
    > For instance, ML can reliably detect the race of a person from an x-ray. There is clearly some anatomical difference that we're not aware of that's detectable in an x-ray. I'm pretty sure forensic anthropologists have been doing that for decades.
- knicholes 4y ago
  I think we've all seen what happens when we extrapolate.
- cameronh90 4y ago
  I think like anything it's a tool that can be used to great effect. However, I suppose there's a possibility we could become over-reliant on it and blind to the sorts of new ideas that you'd generate from manually exploring the data set yourself. For example, if the state-of-the-art AI asteroid discovering program has a blind spot where it never detects a certain type of asteroid, we might stop finding that type of asteroid altogether. Then when the training datasets themselves are mostly derived from asteroids that the AI had previously found, any blind spots and biases could end up self-reinforcing. Then again, is this really all that different to the many human-driven blind spots science has had in the past?
  - visarga 4y ago
    > is this really all that different to the many human-driven blind spots No. Humans are fallible like AI. Most humans can't perfectly apply logical reasoning to common day situations.
    - mensetmanusman 4y ago
      They are both fallible because humans supply the data.
      - JoshCole 4y ago
        AI includes supervised learning, but supervised learning is a very narrow part of the field of AI. In some problems, humans don't supply the data. For example, in reinforcement learning, rewards are provided by the environment, but there is no point where humans give labeled examples of what constitutes good play. For an even more extreme example, in evolutionary learning there is no reward provided to the algorithm and there can be no training data visible to the algorithm. In effect, it manages to work as an optimization because sometimes there is a lack of data - something dies before reaching a point where it should produce another generation.
    - YeGoblynQueenne 4y ago
      Humans can fail, but humans can also succeed, in applying logical reasoning- not only to common situations, but also to scientific inquiry. That's how we have science, today. If we were only bumbling fools that stumble in the dark we'd still be stuck in dark caves without fire. But we can be -and we often are- much more than that. But not our machine learning systems. Those are stuck in making stupid mistakes and not being able to generalise away from them. For the time being. You can ask GPT-3 a question and it may come up with a smart answer or a dumb answer, and it won't know which is which. But we will. Because, yes, we can make dumb mistakes but we can also understand that they are mistakes, and that they are dumb, and why they are mistakes and dumb, and we can recover from them and stop ourselves from making the same mistakes. We have blind spots, sure, but we can see through them, eventually, with hard work, after a very long time, very haphazardly. But not our machine learning systems. For our machine learning systems the entire world is one big blind spot. That is how humans are different to machine learning systems.
- scroot 4y ago
  Neil Postman used to say that we live in an era that is awash in information overabundance but simultaneously starving for wisdom. One of the immediate and great potential use for this kind of AI (ie, Skinner Boxes for computation) is to help us focus on what matters when faced with overwhelming piles of chaff.
- schaefer 4y ago
  Counterpoint: AlphaGo discovered new moves in the Game of Go, to the extent that books have been written about those new moves and how they change modern opening theory[1]. In Go, there are hundreds of years of tradition studying the games of past master players, and analyzing how their innovations changed our cultural understanding of the game. Humanity can't have it both ways. If those contributions were valuable when made by a human, they should be equally valuable when made by an AI. [1] https://www.amazon.com/Using-AlphaGos-Enlarged-Corner-Enclosures/dp/1090423365/ https://www.amazon.com/Using-AlphaGos-Enlarged-Corner-Enclos...
  - visarga 4y ago
    Another counterpoint - AlphaFold. It really discovers structure we could not previously discover.
    - dekhn 4y ago
      Not really- all of alphafold depends on having enough sequence alignment information to existing proteins with known structures. It really does not perform all that well on out-of-dataset prediction.
      - fsloth 4y ago
        Still having good combinatorical guestimates extrapolated from existing knowledge is not a useless trick in itself.
        visarga 4y ago
        If you can simulate you can check and throw out the garbage, or get more training data for the next iteration.
  - JoshCole 4y ago
    Another counterpoint is that the same technology applied to chess rediscovered human opening theory - then extended it in some openings. In chess, you don't have to write a research paper to make a notable contribution. If people find your move beautiful, they put a little exclamation point next to it in the notation. AlphaZero gets plenty of those - enough for a book about its play [1]. This took multiple human generations for us to accomplish, with literal geniuses applied to the task and spending their whole lives on it. Rediscovery and expansion happened in a wall clock time that is better measured in minutes than generations. [1]: https://www.amazon.com/Game-Changer-AlphaZeros-Groundbreaking-Strategies/dp/9056918184 https://www.amazon.com/Game-Changer-AlphaZeros-Groundbreakin...
- albertzeyer 4y ago
  Current humans are great for classification. A human is often non-explainable, can be non-deterministic and only as good as it’s training data. Yes humans can be used to discover asteroids. BUT every asteroid needs to be checked by a human (or AI) and some will undoubtably be false positives. We often can’t explain from the brain itself. Humans don’t make discoveries. (I guess you get my point.)
  - coding123 4y ago
    That's not a point, you just replaced words but his is true and yours is not. Unless you can explain why "Humans don't make discoveries" is true??? Also this is a VERY tired trope in Hacker News. Please stop doing this. Here's the gist: A human can show that 50 people are dying of cancer because they were recently exposed to toxic sludge. An AI can't say a bunch of people were exposed to toxic sludge - it can only raise your prediction of getting cancer because you live near such and such coordinates (the coordinates where toxic sludge exist - but that's not visible to the AI)
    - BobbyJo 4y ago
      He was making a counter argument. Parent used all the same logic to posit 'ai don't make discoveries'. If those are reasons ai don't make discoveries, and they all apply to humans as well, then it must be that humans also don't make discoveries. This is obviously false, which refutes the parent's point.
      - coding123 4y ago
        Multiple places to refute the core logic: The asteroid has to be checked by a human (or AI) part is incorrect because a computer AI that's checking just images or data can't (yet) call a team of astronomers to focus telescopes somewhere and double check it. That part would almost nearly require general AI. Yes, someone could write a single-use software that tells a bunch of astronomers to check something that AI has found, but that's NOT "the AI" doing that. Humans don't have "training data" they have something else. I don't think "training data" and that "something else" is the same (yet).
- deleted 4y ago
  [deleted]
- dekhn 4y ago
  I don't know that I agree. Historically, many good scientific models had generalizable predictive ability (relativity being a good example) that was used to make predcitions that led to discoveries). I would say that if a scientist can use a model (that's all an ML model is) to anticipate likely regions of phase space to explore (say, in making engineered glass or steel), it is in fact "discovering" things. Also, when you reach the point of needing a human to double check all your predictions and the humans experts disagree about the golden label as much as the model has uncertainty in the prediction, the model has reached human performance, and "human checking the results" doesn't even make sense since they would increase the false positive or negative rate of the system. Already we are seeing some image recognition systems whose classificaftion of hard examples exceed that of the best human experts (while being hundreds of times fasteR).
- JoshCole 4y ago
  > What I mean by that, is that AI is often non-explainable, can be non-deterministic and only as good as it’s training data. By your logic, humans can't discover things, but AI can. You think you made the opposite claim - you didn't. 1. Consider that you can freeze a random seed, making an algorithm deterministic, but you can't do the same for a human. 2. Secondly, I dare you to explain to me exactly, down to the very activation of cells and synapses in your mind, how you came to understand these words as having meaning. You can't do it, but an AI that did this would do this implicitly. So it will do a much better job of explaining how this happens than you will. So you haven't ruled out AI being able to discover things. You've claimed that humans can't, but AI can. It directly contradicts your thesis. 3. We can point to specific examples of novel discoveries produced by modern ML that were made without the use of good training data provided by humans - indeed, without any training data provided by humans. Moreover, these techniques weren't limited by their training data being bad - they were built around the idea that the training data they would generate was bad.
  - karpierz 4y ago
    > 2. Secondly, I dare you to explain to me exactly, down to the very activation of cells and synapses in your mind, how you came to understand these words as having meaning. You can't do it, but an AI that did this would do this implicitly. So it will do a much better job of explaining how this happens than you will. I think you're working with a very different definition of "explainable" than is commonly used in the machine learning literature.
    - JoshCole 4y ago
      They are trying to create a distinction, not between the explainability of machine learning approaches, but between human approaches and machine learning approaches. Whatever paper you think I should be using for my definition, just go to there citations. You'll trace it back eventually to things like the Imitation Game paper by Alan Turing. You'll trace it back to the debate about whether machines can think, discover, learn, know. The papers you reference are built atop a mountain of papers that came before them which largely reject the claim that machines aren't capable of discovery. You'll also find my term usage early on in most books about machine learning. Most authors like introducing machine learning by pointing out why we need machine learning. They point out that humans not being able to explain solutions to very hard problems means they can't build programs that solve those problems. If they could explain their solution, they could code it up and they would have a program. Since they can't, they reach to learning tools to figure it out. It really shouldn't surprise you that we have to go back to the early parts of the field; his argument restated is "machines can't make discoveries, only humans can" and this confused thinking is something that gets addressed in the early writings much moreso than the latter writings. Frankly, we've known algorithms as far back as Aristotle that were capable of discovery. Peter Norvig in AIMA cites Aristotle when talking about some search problems and path discovery is still discovery. Some of these algorithms literally include frontier in the algorithmic description; clearly a framing that implies thinking of them in terms of discovery. Honestly, what is really happening in my estimation is some people do this thing where they pretend AI is categorically reduced to the subset of things which they can criticize. It is a really weird tendency, but extremely common. You can see it in the persons post actually. Look at the contortions they go through to only focus on supervised learning - to only focus on a narrow subset of supervised learning restricted to deep learning - to only focus on a narrow problem domain of supervised learning - classification - and to only focus on a specific example of that problem domain (we can give good explanations for smaller models) - the ones that humans can't easily explain.
- ethanwillis 4y ago
  It's not that it's only as good as it's training data. But depending on the underlying implementation it's also only as good as the quality of the random number it uses in its implementation. Let me give a somewhat broad example. In cases where a state space to be explored is absolutely massive and we have no currently good way to 1. prune it 2. traverse it to land at the states we want(that have certain properties), and so on. Then the best we have right now is "sample the state space." But "sampling" requires very good random number generation so that we're 1. not biasing ourselves (this is clear to everyone) but deeper than that 2. Good random number generation reduces the problem of us only grabbing states from the state space that confirm our existing ideas rather than the ones that might challenge our existing ideas. How many scientists are developing their ideas on single workstations that are running out of entropy?
- mensetmanusman 4y ago
  AI is analogous to a new type of lens. Lenses can be thought of thermodynamically as channeling the information included in rays of light into a different form more appreciable by humans. AI is that for rays of data :)
mateo1 4y ago
>First, AI can quickly read through the scientific literature, allowing it to learn the fundamental rules, facts, and equations of science, and help scientists manage the flood of papers and data that is drowning every field. That's the first in a series of bs claims. AI tools can, in time, bring an engineering revolution but a new scientific dark age whereby idiots with little to no domain knowledge use "ai" tools such as the super resolution ones mentioned, draw wrong conclusions and then publish them so other idiots can do flawed ai-powered (or not) meta-analysis and further damage the state of concensus and integrity in their field.
- visarga 4y ago
  Isn't that how we use all technology? As soon as we abstract something away we are susceptible to modelling problems. Even speech is a poor approximation of what we sometimes want to express - another imperfect model shoehorned on reality.
- walnutclosefarm 4y ago
  I'm missing how you own unsupported claim in any way refutes the claim in the article that you assert to be bs. I have in fact been involved in projects that did exactly what you claim is BS - read a large corpus of scientific papers (and written medical records), and discerned connections worth further scientific investigation that had not been detected by the body of scientists following the field, ultimately leading to new drug discovery science.
  - AnimalMuppet 4y ago
    "Worth further scientific investigation" is one thing. But mateo1 called BS on the claim that AI could "learn the fundamental rules, facts, and equations of science" by reading the literature, which is a different claim. You claim that the AI can find patterns and correlations; mateo1 disputes that the AI can actually learn the rules of the domain from the papers. So while you are right that mateo1 also did not prove his claim, your experience doesn't disprove his claim.
    - 4y ago
Ericson2314 4y ago
Artificial Intelligence is Human Ignorance. Being able to do things without insteranding why can only be temporary stop-gap in the knowledge-gaining process. It cannot revolutionize science by definition. The biggest effect it could have is ending science, convincing the powers that be that understanding is no longer a prerequisite of capability, and thus science can once again be relegated as a nerdy hobby that is of no interest to the state or corporation.
- alar44 4y ago
  Agreed. This is why I do long division by hand.
  - Ericson2314 4y ago
    What the hell. The ALU of the machine does something not to dissimilar to long division. I would be fine with saying no getting to program with division until you learn how to do the long division algorithm by hand, however. But once you have learned to do the algorithm it's fine to let the computer do it. Anti-ignorance != anti-automation
- visarga 4y ago
  When you do a Covid test do you understand why the colour changes? Some people do, but not most. We can use black box systems and we understand how they make false positives and negatives. In fact most complex systems are impossible to "understand". We still have to act with imperfect understanding.
  - mdp2021 4y ago
    > Some people do, but not most Knowledge (of something) is something one could achieve, not something everyone must achieve. The relevant purpose is that those who should become competent and knowledgeable do so - for judgement and for production of further knowledge.
  - Ericson2314 4y ago
    That is not a good comparison. People taking the test are not in general scientists. There was never any expectation they would understand how the test works. The proper analogy would be scientists have some big machine which tries different chemical experiments until it has a Covid test that matches the training data. But how would we even trust the training data in that case? Having a good understanding of the basics of genetics and microbiology allows us both to understand how the tests work, and tests in the QAing of the testing procedure. There is no substitute.
gumby 4y ago
It's exciting that these computational tools have unlocked a new phase of "butterfly collecting". Can't wait for the theoretical developments people come up with on the back of it.
raincom 4y ago
Classification, at best, helps in developing heuristics. Such a classifying activity can never become a scientific theory. There are two things. (1) what makes some phenomenon, any phenomenon, into X? This is the identity question. (2) Given a group, how to classify some phenomenon as an instance of X or an instance of Y, etc,. This is the individuation question. When we do "folk" science, we conflate the question of individuation with that of identity. It is like asking what makes something a cat, then coming up with a classificatory answer: oh, this image is a cat, that image, another cat, etc.
frakt0x90 4y ago
I agree with the skepticism in the comments but one area where I think ML does have a lot of potential in scientific discovery is the field of SciML and Symbolic Regression. The first uses physics-informed neural networks in systems of ODEs to solve complex problems faster and even helps derive DEs that fit some data. The second uses a fixed set of symbols to fit a tree of them to data. This gives a closed-form solution that best fits the data. Both are pretty hot in the Julia community and have results published
andreyk 4y ago
This article has some great examples, but there are MANY it does not mention. Here are just a few others I think are super cool (links to media articles, papers are typically linked to within article): * AI model’s insight helps astronomers propose new theory for observing far-off worlds https://techcrunch.com/2022/06/03/ai-models-insight-helps-astronomers-propose-new-theory-for-observing-far-off-worlds/ https://techcrunch.com/2022/06/03/ai-models-insight-helps-as... * Scientists Discover Method to Break Down Plastic in Days, Not Centuries https://www.vice.com/en/article/akvm5b/scientists-discover-method-to-break-down-plastic-in-one-week-not-centuries https://www.vice.com/en/article/akvm5b/scientists-discover-m... * Machine Learning Helps See into a Volcano’s Depths https://eos.org/editor-highlights/machine-learning-helps-see-into-a-volcanos-depths https://eos.org/editor-highlights/machine-learning-helps-see... * Stanford University use AI computing to cut DNA sequencing down to five hours https://www.zdnet.com/article/stanford-uni-nvidia-use-ai-computing-to-cut-dna-sequencing-down-to-five-hours/ https://www.zdnet.com/article/stanford-uni-nvidia-use-ai-com... AI is just a tool, and is increasingly becoming a big part of scientists' toolsets. I don't know if it's fair to say it will lead to a scientific revolution, but it's impacts will most certainly be vast. Self-plug warning: I know of all of these articles because I run the newsletter Last Week in AI (lastweekin.ai) and co-host its related podcast (https://www.lastweekinai.com https://www.lastweekinai.com). We've covered a dozen or more of these types of stories, and new ones keep showing up. So it's definitely a big trend IMHO.
nathias 4y ago
if scientific discovery could be automated, that would be the most significant event in human history
TaupeRanger 4y ago
Whenever you see the words "AI" and "ushering" in a headline, you can be sure it's mostly BS. All you have to do to write one of these articles is wildly extrapolate from very modest current systems to completely unfounded speculative miracles in the future. Could Alpha Fold eventually lead to some drugs that help real patients live longer or better lives? Sure...maybe...or maybe not. Maybe literally nothing will come from this technology, except some great headlines for companies claiming to have created a "wholly-AI-designed drug", whatever the hell that actually means.
- mensetmanusman 4y ago
  This is actually an appropriate use of the word ushering. We finally have great uses for all these transistors:)
- andreyk 4y ago
  AlphaFold already has 3000 citations... For instance "A structural biology community assessment of AlphaFold 2 applications" which states "In summary, we find that these advances are likely to have a transformative impact in structural biology and broader life science research." https://www.biorxiv.org/content/10.1101/2021.09.26.461876v1.abstract https://www.biorxiv.org/content/10.1101/2021.09.26.461876v1.... Is the title hyperbolic wrt this being a revolution? Maybe. Is it fair to say AI is 'ushering' a lot of new science research into existence? Absolutely.
  - TaupeRanger 4y ago
    And yet, human lives haven't improved one iota from it. What good is a technology if all it does is cause lots of citations and padding of researchers' CVs?
    - rsfern 4y ago
      There’s a few drugs currently in clinical trials that used AlphaFold in the design pipeline, IIRC. So that is something.
stevenjgarner 4y ago
Would it behoove the standardization of some kind of rubric to codify scientific papers on publication where they clearly xml-style state dependencies, conclusions, conditionalities, need for further research, keywords etc? Or is the state of natural language processing adequate to parse that information out of the publications the way they are?
concinds 4y ago
I don't understand the comments here. AI means we can literally brute-force any scientific mystery by trying out millions of algorithms until we find one that fits (like Wolfram's New Kind of Science). General relativity was a brilliant theory, but we don't have billions of Einsteins to keep coming up with new theories. Soon, we will. And obviously, brute forcing is only a last resort, AIs will be smarter than that for most problem-sets. Who cares that it's a black box? Humans still need to verify a new model or theory (the AI will help with that too) but it'll shave decades off of discoveries. Why would humans lose domain knowledge? (There's plenty of non-economists writing economics papers already, and they seem human!) AI might dramatically increase the rate of scientific discoveries, and make a lot more people want to become scientists. It'll also help reclaim science from big institutions and academia, and help small independent groups make significant discoveries again without gatekeepers. Patent clerks might be able to make scientific discoveries again! And why would AI necessarily be purely imitative (i.e. only be capable of building off past knowledge, rather than generating new knowledge or at least using abstract reasoning)? See this tweet from a DeepMind researcher, plenty of people are working on this: https://twitter.com/Inoryy/status/1522621712382234624 https://twitter.com/Inoryy/status/1522621712382234624
- mdp2021 4y ago
  > Who cares that it's a black box To solve a specific problem, you just need a "mysterious solver". To increase human knowledge, you instead need a formally expressed insight. In the article I read, «In 2020, Google’s AI team DeepMind announced that its algorithm, AlphaFold, had solved the protein-folding problem». Now, I understand that AlphaFold "solves protein-folding problem_s_", while "the protein-folding problem" would be a theoretical problem which calls for a theory to be solved. This is the difference. They are two different matters, having things done and having us more knowledgeable.
- randcraw 4y ago
  "AI means we can literally brute-force any scientific mystery by trying out millions of algorithms..." This is exactly wrong. If true, then AI could readily solve any human-level problem we could throw at it. But AI still can't master any of the basic skills inherent in human children, much less in adults working in construction crews, or health care aides, or janitors, or even just driving a car. AI's ability to build robots capable of even basic tasks like giving a haircut or climbing over broken objects littering a stair -- they're nonexistent, and even after a decade of deep learning, they're not getting appreciably better. Deep learning has shown us two things: 1) deep NNs can learn how to solve approximation problems astonishingly efficiently and capably. That's AlphaGo and AlphaFold. And 2) a lot of what humans assumed to be signs of intelligence, like playing go and chess, understanding voice and speaking, recognizing objects in images -- these are just "monkey see, monkey do" skills based in probabilistic pattern matching, which deep NNs can do very, very well. But cognitive skills that require reasoning -- deduction, induction, and the recognition and rejection of contradiction -- these must identify relevant domain knowledge (semantics) and the ability to reason about it, not just the efficient matching of a pattern in a matrix using a GPU. Alas, the effective and scaleable means of solving these problems still remain a total mystery. So if you can generate a zillion clueless hypotheses per second using the very best deep NN, it will still drive over a cop dressed like a clown who's waving his arms the path of your smartcar, nor will it ever invent general relativity.

94 points by spansoa 4y ago

Current AI methods are great for classification. That said, I’m not convinced AI is making discoveries. What I mean by that, is that AI is often non-explainable, can be non-deterministic and only as good as it’s training data. For instance, asteroid discovery. Yes AI can be used to discover asteroids. BUT every asteroid needs to be checked by a human and some will undoubtably be false positives. We also miss some unknown true positives. We often (depending on method) can’t explain why [in-fact] why something is a true positive vs false positive from the network itself. Which is why I say AI doesn’t make discoveries. It’s not identifying something knew, it’s identifying something that looks like what it’s seen before. It’s error rates in the scientific frontier is also going to be unknown; as we often lack data. Using AI could potentially limit the scope of new discoveries - depending how heavily it’s relied on by hyper optimizing on the wrong items. Anyway, sorry for the rambling, as a research in deep learning I find this trend something to monitor.

sumy23 4y ago
I disagree. Yes, results need to be validated, but that doesn't mean ML is not making discoveries. Even for basic classification tasks, discoveries are still being made. For instance, ML can reliably detect the race of a person from an x-ray. There is clearly some anatomical difference that we're not aware of that's detectable in an x-ray. Most of human inference, in general, is "connecting the dots" between related phenomena. ML is a great tool for doing this kind of correlative work at scale.
- omgwtfbyobbq 4y ago
  I think we need to be cautious about what ML is picking up. Finding patterns is great, but we need to be fairly sure about what those patterns are so we don't misapply them in some different context we don't understand. As for whether finding patterns is discovery, I feel very whatever about it.
- andi999 4y ago
  I remember the paper and the conclusion you say is doubtful. Even when the data was heavily distorted and resolution reduced to insanely big pixels it still could do this. So, since magic does not exist, there is probably a hidden cofactor (like xrays done on different machines, one in a hospital in a white neighbourhood, the other in a black one, and the machine giving a technical artifact like colorrange). This is just an example of course, I dont know the cofactor. Here is the study:https://www.thelancet.com/journals/landig/article/PIIS2589-7500(22)00063-2/fulltext https://www.thelancet.com/journals/landig/article/PIIS2589-7... When i said insanely i didnt remember correctly, just saw even if reduce to 4x4 pixels they claim a statistical difference. So mind blowingly ludicrous resolution might have been a better term.
  - walnutclosefarm 4y ago
    I agree that there is something odd going on in the race study. However, I've seen analogous "discovery" results that don't have this problem. Scientists at my former employer built a model, e.g., that diagnosed asymptomatic heart failure from ECGs. No electrophysiologist in that world-class cardiac medicine unit can do this, but the algorithm clearly perceives the signal. And this was in fact a true "discovery" in that they didn't set out to diagnose heart failure, but rather to "score" ventricular ejection fraction using ECGs. Training the algorithm revealed a signal (a so far unexplained cofactor to reduced ejection fraction) that predicts reduced ejection fraction, and thus heart failure, prior to grossly measurable reduced ejection fraction.
    - andi999 4y ago
      Nice. I think this is very interesting, is it published?
      - walnutclosefarm 4y ago
        https://www.nature.com/articles/s41591-021-01335-4 https://www.nature.com/articles/s41591-021-01335-4
- idontpost 4y ago
  > For instance, ML can reliably detect the race of a person from an x-ray. There is clearly some anatomical difference that we're not aware of that's detectable in an x-ray. I'm pretty sure forensic anthropologists have been doing that for decades.
knicholes 4y ago
I think we've all seen what happens when we extrapolate.
cameronh90 4y ago
I think like anything it's a tool that can be used to great effect. However, I suppose there's a possibility we could become over-reliant on it and blind to the sorts of new ideas that you'd generate from manually exploring the data set yourself. For example, if the state-of-the-art AI asteroid discovering program has a blind spot where it never detects a certain type of asteroid, we might stop finding that type of asteroid altogether. Then when the training datasets themselves are mostly derived from asteroids that the AI had previously found, any blind spots and biases could end up self-reinforcing. Then again, is this really all that different to the many human-driven blind spots science has had in the past?
- visarga 4y ago
  > is this really all that different to the many human-driven blind spots No. Humans are fallible like AI. Most humans can't perfectly apply logical reasoning to common day situations.
  - mensetmanusman 4y ago
    They are both fallible because humans supply the data.
    - JoshCole 4y ago
      AI includes supervised learning, but supervised learning is a very narrow part of the field of AI. In some problems, humans don't supply the data. For example, in reinforcement learning, rewards are provided by the environment, but there is no point where humans give labeled examples of what constitutes good play. For an even more extreme example, in evolutionary learning there is no reward provided to the algorithm and there can be no training data visible to the algorithm. In effect, it manages to work as an optimization because sometimes there is a lack of data - something dies before reaching a point where it should produce another generation.
  - YeGoblynQueenne 4y ago
    Humans can fail, but humans can also succeed, in applying logical reasoning- not only to common situations, but also to scientific inquiry. That's how we have science, today. If we were only bumbling fools that stumble in the dark we'd still be stuck in dark caves without fire. But we can be -and we often are- much more than that. But not our machine learning systems. Those are stuck in making stupid mistakes and not being able to generalise away from them. For the time being. You can ask GPT-3 a question and it may come up with a smart answer or a dumb answer, and it won't know which is which. But we will. Because, yes, we can make dumb mistakes but we can also understand that they are mistakes, and that they are dumb, and why they are mistakes and dumb, and we can recover from them and stop ourselves from making the same mistakes. We have blind spots, sure, but we can see through them, eventually, with hard work, after a very long time, very haphazardly. But not our machine learning systems. For our machine learning systems the entire world is one big blind spot. That is how humans are different to machine learning systems.
scroot 4y ago
Neil Postman used to say that we live in an era that is awash in information overabundance but simultaneously starving for wisdom. One of the immediate and great potential use for this kind of AI (ie, Skinner Boxes for computation) is to help us focus on what matters when faced with overwhelming piles of chaff.
schaefer 4y ago
Counterpoint: AlphaGo discovered new moves in the Game of Go, to the extent that books have been written about those new moves and how they change modern opening theory[1]. In Go, there are hundreds of years of tradition studying the games of past master players, and analyzing how their innovations changed our cultural understanding of the game. Humanity can't have it both ways. If those contributions were valuable when made by a human, they should be equally valuable when made by an AI. [1] https://www.amazon.com/Using-AlphaGos-Enlarged-Corner-Enclosures/dp/1090423365/ https://www.amazon.com/Using-AlphaGos-Enlarged-Corner-Enclos...
- visarga 4y ago
  Another counterpoint - AlphaFold. It really discovers structure we could not previously discover.
  - dekhn 4y ago
    Not really- all of alphafold depends on having enough sequence alignment information to existing proteins with known structures. It really does not perform all that well on out-of-dataset prediction.
    - fsloth 4y ago
      Still having good combinatorical guestimates extrapolated from existing knowledge is not a useless trick in itself.
      - visarga 4y ago
        If you can simulate you can check and throw out the garbage, or get more training data for the next iteration.
- JoshCole 4y ago
  Another counterpoint is that the same technology applied to chess rediscovered human opening theory - then extended it in some openings. In chess, you don't have to write a research paper to make a notable contribution. If people find your move beautiful, they put a little exclamation point next to it in the notation. AlphaZero gets plenty of those - enough for a book about its play [1]. This took multiple human generations for us to accomplish, with literal geniuses applied to the task and spending their whole lives on it. Rediscovery and expansion happened in a wall clock time that is better measured in minutes than generations. [1]: https://www.amazon.com/Game-Changer-AlphaZeros-Groundbreaking-Strategies/dp/9056918184 https://www.amazon.com/Game-Changer-AlphaZeros-Groundbreakin...
albertzeyer 4y ago
Current humans are great for classification. A human is often non-explainable, can be non-deterministic and only as good as it’s training data. Yes humans can be used to discover asteroids. BUT every asteroid needs to be checked by a human (or AI) and some will undoubtably be false positives. We often can’t explain from the brain itself. Humans don’t make discoveries. (I guess you get my point.)
- coding123 4y ago
  That's not a point, you just replaced words but his is true and yours is not. Unless you can explain why "Humans don't make discoveries" is true??? Also this is a VERY tired trope in Hacker News. Please stop doing this. Here's the gist: A human can show that 50 people are dying of cancer because they were recently exposed to toxic sludge. An AI can't say a bunch of people were exposed to toxic sludge - it can only raise your prediction of getting cancer because you live near such and such coordinates (the coordinates where toxic sludge exist - but that's not visible to the AI)
  - BobbyJo 4y ago
    He was making a counter argument. Parent used all the same logic to posit 'ai don't make discoveries'. If those are reasons ai don't make discoveries, and they all apply to humans as well, then it must be that humans also don't make discoveries. This is obviously false, which refutes the parent's point.
    - coding123 4y ago
      Multiple places to refute the core logic: The asteroid has to be checked by a human (or AI) part is incorrect because a computer AI that's checking just images or data can't (yet) call a team of astronomers to focus telescopes somewhere and double check it. That part would almost nearly require general AI. Yes, someone could write a single-use software that tells a bunch of astronomers to check something that AI has found, but that's NOT "the AI" doing that. Humans don't have "training data" they have something else. I don't think "training data" and that "something else" is the same (yet).
deleted 4y ago
[deleted]
dekhn 4y ago
I don't know that I agree. Historically, many good scientific models had generalizable predictive ability (relativity being a good example) that was used to make predcitions that led to discoveries). I would say that if a scientist can use a model (that's all an ML model is) to anticipate likely regions of phase space to explore (say, in making engineered glass or steel), it is in fact "discovering" things. Also, when you reach the point of needing a human to double check all your predictions and the humans experts disagree about the golden label as much as the model has uncertainty in the prediction, the model has reached human performance, and "human checking the results" doesn't even make sense since they would increase the false positive or negative rate of the system. Already we are seeing some image recognition systems whose classificaftion of hard examples exceed that of the best human experts (while being hundreds of times fasteR).
JoshCole 4y ago
> What I mean by that, is that AI is often non-explainable, can be non-deterministic and only as good as it’s training data. By your logic, humans can't discover things, but AI can. You think you made the opposite claim - you didn't. 1. Consider that you can freeze a random seed, making an algorithm deterministic, but you can't do the same for a human. 2. Secondly, I dare you to explain to me exactly, down to the very activation of cells and synapses in your mind, how you came to understand these words as having meaning. You can't do it, but an AI that did this would do this implicitly. So it will do a much better job of explaining how this happens than you will. So you haven't ruled out AI being able to discover things. You've claimed that humans can't, but AI can. It directly contradicts your thesis. 3. We can point to specific examples of novel discoveries produced by modern ML that were made without the use of good training data provided by humans - indeed, without any training data provided by humans. Moreover, these techniques weren't limited by their training data being bad - they were built around the idea that the training data they would generate was bad.
- karpierz 4y ago
  > 2. Secondly, I dare you to explain to me exactly, down to the very activation of cells and synapses in your mind, how you came to understand these words as having meaning. You can't do it, but an AI that did this would do this implicitly. So it will do a much better job of explaining how this happens than you will. I think you're working with a very different definition of "explainable" than is commonly used in the machine learning literature.
  - JoshCole 4y ago
    They are trying to create a distinction, not between the explainability of machine learning approaches, but between human approaches and machine learning approaches. Whatever paper you think I should be using for my definition, just go to there citations. You'll trace it back eventually to things like the Imitation Game paper by Alan Turing. You'll trace it back to the debate about whether machines can think, discover, learn, know. The papers you reference are built atop a mountain of papers that came before them which largely reject the claim that machines aren't capable of discovery. You'll also find my term usage early on in most books about machine learning. Most authors like introducing machine learning by pointing out why we need machine learning. They point out that humans not being able to explain solutions to very hard problems means they can't build programs that solve those problems. If they could explain their solution, they could code it up and they would have a program. Since they can't, they reach to learning tools to figure it out. It really shouldn't surprise you that we have to go back to the early parts of the field; his argument restated is "machines can't make discoveries, only humans can" and this confused thinking is something that gets addressed in the early writings much moreso than the latter writings. Frankly, we've known algorithms as far back as Aristotle that were capable of discovery. Peter Norvig in AIMA cites Aristotle when talking about some search problems and path discovery is still discovery. Some of these algorithms literally include frontier in the algorithmic description; clearly a framing that implies thinking of them in terms of discovery. Honestly, what is really happening in my estimation is some people do this thing where they pretend AI is categorically reduced to the subset of things which they can criticize. It is a really weird tendency, but extremely common. You can see it in the persons post actually. Look at the contortions they go through to only focus on supervised learning - to only focus on a narrow subset of supervised learning restricted to deep learning - to only focus on a narrow problem domain of supervised learning - classification - and to only focus on a specific example of that problem domain (we can give good explanations for smaller models) - the ones that humans can't easily explain.
ethanwillis 4y ago
It's not that it's only as good as it's training data. But depending on the underlying implementation it's also only as good as the quality of the random number it uses in its implementation. Let me give a somewhat broad example. In cases where a state space to be explored is absolutely massive and we have no currently good way to 1. prune it 2. traverse it to land at the states we want(that have certain properties), and so on. Then the best we have right now is "sample the state space." But "sampling" requires very good random number generation so that we're 1. not biasing ourselves (this is clear to everyone) but deeper than that 2. Good random number generation reduces the problem of us only grabbing states from the state space that confirm our existing ideas rather than the ones that might challenge our existing ideas. How many scientists are developing their ideas on single workstations that are running out of entropy?
mensetmanusman 4y ago
AI is analogous to a new type of lens. Lenses can be thought of thermodynamically as channeling the information included in rays of light into a different form more appreciable by humans. AI is that for rays of data :)

mateo1 4y ago

>First, AI can quickly read through the scientific literature, allowing it to learn the fundamental rules, facts, and equations of science, and help scientists manage the flood of papers and data that is drowning every field. That's the first in a series of bs claims. AI tools can, in time, bring an engineering revolution but a new scientific dark age whereby idiots with little to no domain knowledge use "ai" tools such as the super resolution ones mentioned, draw wrong conclusions and then publish them so other idiots can do flawed ai-powered (or not) meta-analysis and further damage the state of concensus and integrity in their field.

visarga 4y ago
Isn't that how we use all technology? As soon as we abstract something away we are susceptible to modelling problems. Even speech is a poor approximation of what we sometimes want to express - another imperfect model shoehorned on reality.
walnutclosefarm 4y ago
I'm missing how you own unsupported claim in any way refutes the claim in the article that you assert to be bs. I have in fact been involved in projects that did exactly what you claim is BS - read a large corpus of scientific papers (and written medical records), and discerned connections worth further scientific investigation that had not been detected by the body of scientists following the field, ultimately leading to new drug discovery science.
- AnimalMuppet 4y ago
  "Worth further scientific investigation" is one thing. But mateo1 called BS on the claim that AI could "learn the fundamental rules, facts, and equations of science" by reading the literature, which is a different claim. You claim that the AI can find patterns and correlations; mateo1 disputes that the AI can actually learn the rules of the domain from the papers. So while you are right that mateo1 also did not prove his claim, your experience doesn't disprove his claim.
  - 4y ago

Ericson2314 4y ago

Artificial Intelligence is Human Ignorance. Being able to do things without insteranding why can only be temporary stop-gap in the knowledge-gaining process. It cannot revolutionize science by definition. The biggest effect it could have is ending science, convincing the powers that be that understanding is no longer a prerequisite of capability, and thus science can once again be relegated as a nerdy hobby that is of no interest to the state or corporation.

alar44 4y ago
Agreed. This is why I do long division by hand.
- Ericson2314 4y ago
  What the hell. The ALU of the machine does something not to dissimilar to long division. I would be fine with saying no getting to program with division until you learn how to do the long division algorithm by hand, however. But once you have learned to do the algorithm it's fine to let the computer do it. Anti-ignorance != anti-automation
visarga 4y ago
When you do a Covid test do you understand why the colour changes? Some people do, but not most. We can use black box systems and we understand how they make false positives and negatives. In fact most complex systems are impossible to "understand". We still have to act with imperfect understanding.
- mdp2021 4y ago
  > Some people do, but not most Knowledge (of something) is something one could achieve, not something everyone must achieve. The relevant purpose is that those who should become competent and knowledgeable do so - for judgement and for production of further knowledge.
- Ericson2314 4y ago
  That is not a good comparison. People taking the test are not in general scientists. There was never any expectation they would understand how the test works. The proper analogy would be scientists have some big machine which tries different chemical experiments until it has a Covid test that matches the training data. But how would we even trust the training data in that case? Having a good understanding of the basics of genetics and microbiology allows us both to understand how the tests work, and tests in the QAing of the testing procedure. There is no substitute.

gumby 4y ago

It's exciting that these computational tools have unlocked a new phase of "butterfly collecting". Can't wait for the theoretical developments people come up with on the back of it.

raincom 4y ago

Classification, at best, helps in developing heuristics. Such a classifying activity can never become a scientific theory. There are two things. (1) what makes some phenomenon, any phenomenon, into X? This is the identity question. (2) Given a group, how to classify some phenomenon as an instance of X or an instance of Y, etc,. This is the individuation question. When we do "folk" science, we conflate the question of individuation with that of identity. It is like asking what makes something a cat, then coming up with a classificatory answer: oh, this image is a cat, that image, another cat, etc.

frakt0x90 4y ago

I agree with the skepticism in the comments but one area where I think ML does have a lot of potential in scientific discovery is the field of SciML and Symbolic Regression. The first uses physics-informed neural networks in systems of ODEs to solve complex problems faster and even helps derive DEs that fit some data. The second uses a fixed set of symbols to fit a tree of them to data. This gives a closed-form solution that best fits the data. Both are pretty hot in the Julia community and have results published

andreyk 4y ago

This article has some great examples, but there are MANY it does not mention. Here are just a few others I think are super cool (links to media articles, papers are typically linked to within article): * AI model’s insight helps astronomers propose new theory for observing far-off worlds https://techcrunch.com/2022/06/03/ai-models-insight-helps-astronomers-propose-new-theory-for-observing-far-off-worlds/ https://techcrunch.com/2022/06/03/ai-models-insight-helps-as... * Scientists Discover Method to Break Down Plastic in Days, Not Centuries https://www.vice.com/en/article/akvm5b/scientists-discover-method-to-break-down-plastic-in-one-week-not-centuries https://www.vice.com/en/article/akvm5b/scientists-discover-m... * Machine Learning Helps See into a Volcano’s Depths https://eos.org/editor-highlights/machine-learning-helps-see-into-a-volcanos-depths https://eos.org/editor-highlights/machine-learning-helps-see... * Stanford University use AI computing to cut DNA sequencing down to five hours https://www.zdnet.com/article/stanford-uni-nvidia-use-ai-computing-to-cut-dna-sequencing-down-to-five-hours/ https://www.zdnet.com/article/stanford-uni-nvidia-use-ai-com... AI is just a tool, and is increasingly becoming a big part of scientists' toolsets. I don't know if it's fair to say it will lead to a scientific revolution, but it's impacts will most certainly be vast. Self-plug warning: I know of all of these articles because I run the newsletter Last Week in AI (lastweekin.ai) and co-host its related podcast (https://www.lastweekinai.com https://www.lastweekinai.com). We've covered a dozen or more of these types of stories, and new ones keep showing up. So it's definitely a big trend IMHO.

nathias 4y ago

if scientific discovery could be automated, that would be the most significant event in human history

TaupeRanger 4y ago

Whenever you see the words "AI" and "ushering" in a headline, you can be sure it's mostly BS. All you have to do to write one of these articles is wildly extrapolate from very modest current systems to completely unfounded speculative miracles in the future. Could Alpha Fold eventually lead to some drugs that help real patients live longer or better lives? Sure...maybe...or maybe not. Maybe literally nothing will come from this technology, except some great headlines for companies claiming to have created a "wholly-AI-designed drug", whatever the hell that actually means.

mensetmanusman 4y ago
This is actually an appropriate use of the word ushering. We finally have great uses for all these transistors:)
andreyk 4y ago
AlphaFold already has 3000 citations... For instance "A structural biology community assessment of AlphaFold 2 applications" which states "In summary, we find that these advances are likely to have a transformative impact in structural biology and broader life science research." https://www.biorxiv.org/content/10.1101/2021.09.26.461876v1.abstract https://www.biorxiv.org/content/10.1101/2021.09.26.461876v1.... Is the title hyperbolic wrt this being a revolution? Maybe. Is it fair to say AI is 'ushering' a lot of new science research into existence? Absolutely.
- TaupeRanger 4y ago
  And yet, human lives haven't improved one iota from it. What good is a technology if all it does is cause lots of citations and padding of researchers' CVs?
  - rsfern 4y ago
    There’s a few drugs currently in clinical trials that used AlphaFold in the design pipeline, IIRC. So that is something.

stevenjgarner 4y ago

Would it behoove the standardization of some kind of rubric to codify scientific papers on publication where they clearly xml-style state dependencies, conclusions, conditionalities, need for further research, keywords etc? Or is the state of natural language processing adequate to parse that information out of the publications the way they are?

concinds 4y ago

I don't understand the comments here. AI means we can literally brute-force any scientific mystery by trying out millions of algorithms until we find one that fits (like Wolfram's New Kind of Science). General relativity was a brilliant theory, but we don't have billions of Einsteins to keep coming up with new theories. Soon, we will. And obviously, brute forcing is only a last resort, AIs will be smarter than that for most problem-sets. Who cares that it's a black box? Humans still need to verify a new model or theory (the AI will help with that too) but it'll shave decades off of discoveries. Why would humans lose domain knowledge? (There's plenty of non-economists writing economics papers already, and they seem human!) AI might dramatically increase the rate of scientific discoveries, and make a lot more people want to become scientists. It'll also help reclaim science from big institutions and academia, and help small independent groups make significant discoveries again without gatekeepers. Patent clerks might be able to make scientific discoveries again! And why would AI necessarily be purely imitative (i.e. only be capable of building off past knowledge, rather than generating new knowledge or at least using abstract reasoning)? See this tweet from a DeepMind researcher, plenty of people are working on this: https://twitter.com/Inoryy/status/1522621712382234624 https://twitter.com/Inoryy/status/1522621712382234624

mdp2021 4y ago
> Who cares that it's a black box To solve a specific problem, you just need a "mysterious solver". To increase human knowledge, you instead need a formally expressed insight. In the article I read, «In 2020, Google’s AI team DeepMind announced that its algorithm, AlphaFold, had solved the protein-folding problem». Now, I understand that AlphaFold "solves protein-folding problem_s_", while "the protein-folding problem" would be a theoretical problem which calls for a theory to be solved. This is the difference. They are two different matters, having things done and having us more knowledgeable.
randcraw 4y ago
"AI means we can literally brute-force any scientific mystery by trying out millions of algorithms..." This is exactly wrong. If true, then AI could readily solve any human-level problem we could throw at it. But AI still can't master any of the basic skills inherent in human children, much less in adults working in construction crews, or health care aides, or janitors, or even just driving a car. AI's ability to build robots capable of even basic tasks like giving a haircut or climbing over broken objects littering a stair -- they're nonexistent, and even after a decade of deep learning, they're not getting appreciably better. Deep learning has shown us two things: 1) deep NNs can learn how to solve approximation problems astonishingly efficiently and capably. That's AlphaGo and AlphaFold. And 2) a lot of what humans assumed to be signs of intelligence, like playing go and chess, understanding voice and speaking, recognizing objects in images -- these are just "monkey see, monkey do" skills based in probabilistic pattern matching, which deep NNs can do very, very well. But cognitive skills that require reasoning -- deduction, induction, and the recognition and rejection of contradiction -- these must identify relevant domain knowledge (semantics) and the ability to reason about it, not just the efficient matching of a pattern in a matrix using a GPU. Alas, the effective and scaleable means of solving these problems still remain a total mystery. So if you can generate a zillion clueless hypotheses per second using the very best deep NN, it will still drive over a cop dressed like a clown who's waving his arms the path of your smartcar, nor will it ever invent general relativity.