7 ms·

I am not an ML engineer (except when I program in ML, of course ;)). But this sounds a lot like the following: 1. We had a model that worked in principle, but

I am not an ML engineer (except when I program in ML, of course ;)). But this sounds a lot like the following: 1. We had a model that worked in principle, but the search space was practically infeasible. 2. We made an observation that a different model might exist that makes the search space irrelevant. 3. We threw ML at it. 4. Now we might have a model that fulfills (2) but we cannot be sure because we used a black-box approach. 5. Somehow the results are exciting. Better results would be really exciting. 6. We hope that more data yields these better results. Is that correct? Am I the only one to lament these black-box approaches? Should there not be a bunch of people now studying the learned models to figure out if much better results can actually exist?

mlthoughts2018 7y ago
What do you mean by “black box” model? How do you know when a model “explains” something or not? How do you know if you are missing a key feature or degree of freedom in a model? Suppose you have two models, A and B, where A gives much poorer predictions than B. Is it possible for A to “explain” things better when it is not capable of producing adequate predictions? Since nature itself is not boiled down to logical elements like math proofs, what about the inherent ambiguity of language or mental models when assessing explainability? Did we really explain anything, or just decompose a problem into units of thought that artificially feel compelling for human minds at this moment of history? What would distinguish a “real explanation” from a convenient fiction, if not purely predictive capability?
- brmgb 7y ago
  Black box model is a perfectly well understood term of art. It means a model which has a somewhat opaque internal working. A lot of modern ML approaches treat the model as a black box in that it is not particularly clear how the features actually interact to reach the prediction.
  - mlthoughts2018 7y ago
    I work professionally in machine learning for over ten years now and I deeply dispute what you say, and many other practitioners do as well. > “not particularly clear how the features actually interact to reach the prediction.“ This is quite true of many models, such as linear regression in the wild. You may also have a clear but wrong picture of how features interact, eg looking at coefficients of interaction terms in a misspecified linear model. See for example, “The Mythos of Model Interpretability” - https://arxiv.org/abs/1606.03490 https://arxiv.org/abs/1606.03490 Without a coherent definition of what “not black box” or “explainable” means scientifically, the buzzword of “black box” is also meaningless and is more of a political game to be a gatekeeper over what models are allowed to be used than any kind of honest intellectual inquiry. I’ve worked professionally on systems where misspecified linear models and text models using simple ngram boosting were vastly more inscrutable than comparable neural networks or dimensionality reduction models for the same applications. Nobody has any scientifically cogent idea of what makes a model “explainable” other than arguing semantics.
    - SQueeeeeL 7y ago
      >Nobody has any scientifically cogent idea of what makes a model “explainable” other than arguing semantics. Bullshit, any client you show a regression to can easily fully understand how the inputs change the output.
      - ovi256 7y ago
        That's only true in some limited cases. Section 4.1 of the paper he linked discusses extensively the limitations of linear model interpretability. Here's an example: "With respect to algorithmic transparency, [the claim that linear models are more interpretable] seems uncontroversial, but given high dimensional or heavily engineered features, linear models lose simulatability or decomposability, respectively." Given how many interesting problems are high-dimensional or solved through heavy feature engineering, you must be lucky to live in the happy space where problems are either low-dimensional enough to not need heavy feature engineering or where the client is already familiar with the high-dimensional features you're going to limit the model to using.
      - 7y ago
stopads 7y ago
This is 90% of science these days, just keep tweaking numbers in the model until it fits with what you want to see. The why and the how is not really something we have the capability or tooling to tackle. I got bored with physics really quickly when I realized this is what everyone does.
- coldtea 7y ago
  >This is 90% of science these days, just keep tweaking numbers in the model until it fits with what you want to see. If the result is immediately applicable to real world problems, like in this case, that's a perfectly valid approach. In this case we just need the model to produce answers we can use -- how we made the model, what's in it, etc, we don't need to care.
  - not_qe 7y ago
    Did you read the end of the paper.
ramraj07 7y ago
Unlike in other cases, with protein structure, we really only care about the structure, most of the time no one cares how the chain folded to get there. ML based methods seem perfect for that.
ArtWomb 7y ago
>>> Am I the only one to lament these black-box approaches? Far from it. Prediction looks like a tool in the arsenal for better understanding. One still has to correlate the structure with the complex interactions in vivo. Even using AI in classification mode, where we can segment a large atlas of tumor cells and identify a dozen or so classes of cell anomalies may lead to faster breakthroughs in immunotherapy. What I am trying to wrap my head around is the synthesis problem. Say AlphaFold generates a promising candidate. One that does not exist naturally. You still need the DNA or mRNA transcription sequence to synthesize the protein, right? Won't some candidates simply be too complex and unstable to reliably produce using existing mammalian or baculovirus platforms?
- RocketSyntax 7y ago
  You can use permuted feature importance to tell you what inputs are most correlated with the condition. WAYYYY better than the binary correlation methods most statistics use for each feature.
- ovi256 7y ago
  >Won't some candidates simply be too complex and unstable to reliably produce using existing mammalian or baculovirus platforms? You can add that to the objecting function that your model training function is optimizing, ensuring model output is not "too complex or unstable to reliably produce".
nestorD 7y ago
Not an expert but I have read a lot on the subject recently. Methods from the domain where already not truly model-based (compared to things like physical equations). It is mostly observations on existing proteins coupled with a form of gradient descent so, for this particular application, things have not degraded. (I am aware that there is more to it, this is just a fast summary) But to be honest, it is far from a solved problem and I expect more breakthroughs from deep understanding and modeling than ML.
- hobofan 7y ago
  > I expect more breakthroughs from deep understanding and modeling than ML Huh, I would've said exactly the opposite. The feature space of the variations of amino acid sequences is so big that I wouldn't bet on breakthroughs derived from understanding. Most recent advances seem to focus on how specific sequence motifs interact with each other, which are generally only applicable to certain kinds of proteins, but not protein folding as a whole. My prediction would be that next breakthroughs that push the field over the usability limit are derived from more general machine learning advances.
  - nestorD 7y ago
    I think deep learning can only get us so far with the existing, limited, data. I doubt it will be enough for precise structure reconstruction (but I would love to be wrong). What it can be is a force multiplier for every progress in our understanding of why those sequences are transcribed into those structures. But I have to admit that, with years of head-on, the research has not gotten there so I might be wrong.
Damogran6 7y ago
This is far from the only time this has happened, though...Many moons ago I was required to take a class in fluid dynamics in College. It was all observational best-fit statistical estimation that kinda modeled observed behavior. ML is that with significantly more horsepower.
- semi-extrinsic 7y ago
  Either it was a poorly taught class, it was aimed at civil engineering and similar where you want simple models with broad applicability and don't care about the wide error bars, or maybe you remember best the observational stuff. But fluid mechanics has a very deep theoretical underpinning, and generally has interpretable models. I'd suggest skimming through a copy of either Lamb or Batchelor's book, to see how far you can get just with pen and pencil and no statistical input at all.
  - Damogran6 7y ago
    It WAS for Civil Engineering and it was 30 years ago. FEA was JUST starting to be a thing and it sure didn't reach down into anything I encountered. looking back through the google, I may be dimly remembering the calculations dealing with turbulent flow. It was a whole different experience to a College Junior that was used to the relatively simple equations from Physics, Statics and Dynamics.
    - semi-extrinsic 7y ago
      Sounds like the standard way to teach fluids for civ.eng. All the flows you'll ever encounter will be turbulent, no point in learning about all the beautiful results that are mainly for laminar flows. The books I mentioned are classics in the field and were first published in 1895 and 1967, respectively. Both are still in print. No computers, just advanced math (vector calculus etc).
marcosdumay 7y ago
> Now we might have a model that fulfills (2) but we cannot be sure because we used a black-box approach. It's not like the alternative is any different. Being sure you have the right answer is not viable, the other methods of getting there are just more transparent heuristics.
neltnerb 7y ago
3a. We took the best results from ML and did full quantum mechanical simulations on the much smaller set of remaining candidates.
gumby 7y ago
Think of it as an engineering problem rather than a science problem. In much of drug discovery/development (and disease research), being able to predict protein structure would be very valuable. Being able to quickly find candidate structures (that can then be searched for in the lab) speeds things up immensely. Reducing false positives (or just coming up with possibilities at all) is a huge win. But you’re right that this probably doesn’t help the protein theoretician much if at all. We already “know” how it works (it’s just thermodynamics and quantum mechanics) and of course have no idea how it works (“well they wiggle around until they find a low energy state” doesn’t really tell you anything). But that doesn’t keep this from being exciting.
- irq11 7y ago
  Protein structure prediction, at the current levels of precision (and I include AlphaFold here), is not useful for drug discovery. It’s the sort of thing researchers say to get grants, but as a distant goal, not a practical reality. For structure-based drug discovery (which isn’t even the majority of drug discovery), the details are what matter (e.g. “does this water molecule mediate a binding interaction, or do the sidechains shuffle a bit, and kick the water out?”), and these methods don’t even come close to predicting detailed interactions. Metrics in this space are focused on “general correctness” of protein backbone conformation. Success is to achieve a kind of blurry view of the overall shape of the molecule, and drug design is trying to predict specific atomic interactions. They’re two wildly different problems. About the best you can say is that if we had a generalizable model of physics that could predict protein structure, it might also be able to do a good job of evaluating how a small molecule binds to a protein target. But even that is a huge leap, and when you start using black-box methods like AlphaFold to specifically solve the problem of structure prediction, it’s not really clear that generalization is even possible. There are potential practical uses in drug discovery for a method that can design a protein which takes a particular shape, but even that is pretty different from what AlphaFold actually does.
  - lucidrains 7y ago
    Not everyone is focused on solving folding. It's just one step of many. https://www.nature.com/articles/s41592-019-0666-6 https://www.nature.com/articles/s41592-019-0666-6 https://github.com/yangkky/Machine-learning-for-proteins https://github.com/yangkky/Machine-learning-for-proteins
not_qe 7y ago
I have seen the word “watershed” and immediately guessed some it’s some biased unverifiable claptrap from google.
- AlexCoventry 7y ago
  They provide code and data for full verification. https://github.com/deepmind/deepmind-research/blob/master/alphafold_casp13/README.md https://github.com/deepmind/deepmind-research/blob/master/al...
  - not_qe 7y ago
    Will they provide thousands of tensor cpus as well
- mda 7y ago
  Except that they won a competition for protein folding by a huge margin as explained. Such claptrap indeed.
  - deleted 7y ago
    [deleted]
  - not_qe 7y ago
    Review the casps13 results pages and then tell about “huge margin”
    - mda 7y ago
      http://predictioncenter.org/casp13/zscores_final.cgi?formula=assessors http://predictioncenter.org/casp13/zscores_final.cgi?formula...
      - not_qe 7y ago
        First place z score sum 54 vs second place 42 and in a single category. The only thing watershed here is the headline dishonesty.

mlthoughts2018 7y ago

What do you mean by “black box” model? How do you know when a model “explains” something or not? How do you know if you are missing a key feature or degree of freedom in a model? Suppose you have two models, A and B, where A gives much poorer predictions than B. Is it possible for A to “explain” things better when it is not capable of producing adequate predictions? Since nature itself is not boiled down to logical elements like math proofs, what about the inherent ambiguity of language or mental models when assessing explainability? Did we really explain anything, or just decompose a problem into units of thought that artificially feel compelling for human minds at this moment of history? What would distinguish a “real explanation” from a convenient fiction, if not purely predictive capability?

brmgb 7y ago
Black box model is a perfectly well understood term of art. It means a model which has a somewhat opaque internal working. A lot of modern ML approaches treat the model as a black box in that it is not particularly clear how the features actually interact to reach the prediction.
- mlthoughts2018 7y ago
  I work professionally in machine learning for over ten years now and I deeply dispute what you say, and many other practitioners do as well. > “not particularly clear how the features actually interact to reach the prediction.“ This is quite true of many models, such as linear regression in the wild. You may also have a clear but wrong picture of how features interact, eg looking at coefficients of interaction terms in a misspecified linear model. See for example, “The Mythos of Model Interpretability” - https://arxiv.org/abs/1606.03490 https://arxiv.org/abs/1606.03490 Without a coherent definition of what “not black box” or “explainable” means scientifically, the buzzword of “black box” is also meaningless and is more of a political game to be a gatekeeper over what models are allowed to be used than any kind of honest intellectual inquiry. I’ve worked professionally on systems where misspecified linear models and text models using simple ngram boosting were vastly more inscrutable than comparable neural networks or dimensionality reduction models for the same applications. Nobody has any scientifically cogent idea of what makes a model “explainable” other than arguing semantics.
  - SQueeeeeL 7y ago
    >Nobody has any scientifically cogent idea of what makes a model “explainable” other than arguing semantics. Bullshit, any client you show a regression to can easily fully understand how the inputs change the output.
    - ovi256 7y ago
      That's only true in some limited cases. Section 4.1 of the paper he linked discusses extensively the limitations of linear model interpretability. Here's an example: "With respect to algorithmic transparency, [the claim that linear models are more interpretable] seems uncontroversial, but given high dimensional or heavily engineered features, linear models lose simulatability or decomposability, respectively." Given how many interesting problems are high-dimensional or solved through heavy feature engineering, you must be lucky to live in the happy space where problems are either low-dimensional enough to not need heavy feature engineering or where the client is already familiar with the high-dimensional features you're going to limit the model to using.
    - 7y ago

stopads 7y ago

This is 90% of science these days, just keep tweaking numbers in the model until it fits with what you want to see. The why and the how is not really something we have the capability or tooling to tackle. I got bored with physics really quickly when I realized this is what everyone does.

coldtea 7y ago
>This is 90% of science these days, just keep tweaking numbers in the model until it fits with what you want to see. If the result is immediately applicable to real world problems, like in this case, that's a perfectly valid approach. In this case we just need the model to produce answers we can use -- how we made the model, what's in it, etc, we don't need to care.
- not_qe 7y ago
  Did you read the end of the paper.

ramraj07 7y ago

Unlike in other cases, with protein structure, we really only care about the structure, most of the time no one cares how the chain folded to get there. ML based methods seem perfect for that.

ArtWomb 7y ago

>>> Am I the only one to lament these black-box approaches? Far from it. Prediction looks like a tool in the arsenal for better understanding. One still has to correlate the structure with the complex interactions in vivo. Even using AI in classification mode, where we can segment a large atlas of tumor cells and identify a dozen or so classes of cell anomalies may lead to faster breakthroughs in immunotherapy. What I am trying to wrap my head around is the synthesis problem. Say AlphaFold generates a promising candidate. One that does not exist naturally. You still need the DNA or mRNA transcription sequence to synthesize the protein, right? Won't some candidates simply be too complex and unstable to reliably produce using existing mammalian or baculovirus platforms?

RocketSyntax 7y ago
You can use permuted feature importance to tell you what inputs are most correlated with the condition. WAYYYY better than the binary correlation methods most statistics use for each feature.
ovi256 7y ago
>Won't some candidates simply be too complex and unstable to reliably produce using existing mammalian or baculovirus platforms? You can add that to the objecting function that your model training function is optimizing, ensuring model output is not "too complex or unstable to reliably produce".

nestorD 7y ago

Not an expert but I have read a lot on the subject recently. Methods from the domain where already not truly model-based (compared to things like physical equations). It is mostly observations on existing proteins coupled with a form of gradient descent so, for this particular application, things have not degraded. (I am aware that there is more to it, this is just a fast summary) But to be honest, it is far from a solved problem and I expect more breakthroughs from deep understanding and modeling than ML.

hobofan 7y ago
> I expect more breakthroughs from deep understanding and modeling than ML Huh, I would've said exactly the opposite. The feature space of the variations of amino acid sequences is so big that I wouldn't bet on breakthroughs derived from understanding. Most recent advances seem to focus on how specific sequence motifs interact with each other, which are generally only applicable to certain kinds of proteins, but not protein folding as a whole. My prediction would be that next breakthroughs that push the field over the usability limit are derived from more general machine learning advances.
- nestorD 7y ago
  I think deep learning can only get us so far with the existing, limited, data. I doubt it will be enough for precise structure reconstruction (but I would love to be wrong). What it can be is a force multiplier for every progress in our understanding of why those sequences are transcribed into those structures. But I have to admit that, with years of head-on, the research has not gotten there so I might be wrong.

Damogran6 7y ago

This is far from the only time this has happened, though...Many moons ago I was required to take a class in fluid dynamics in College. It was all observational best-fit statistical estimation that kinda modeled observed behavior. ML is that with significantly more horsepower.

semi-extrinsic 7y ago
Either it was a poorly taught class, it was aimed at civil engineering and similar where you want simple models with broad applicability and don't care about the wide error bars, or maybe you remember best the observational stuff. But fluid mechanics has a very deep theoretical underpinning, and generally has interpretable models. I'd suggest skimming through a copy of either Lamb or Batchelor's book, to see how far you can get just with pen and pencil and no statistical input at all.
- Damogran6 7y ago
  It WAS for Civil Engineering and it was 30 years ago. FEA was JUST starting to be a thing and it sure didn't reach down into anything I encountered. looking back through the google, I may be dimly remembering the calculations dealing with turbulent flow. It was a whole different experience to a College Junior that was used to the relatively simple equations from Physics, Statics and Dynamics.
  - semi-extrinsic 7y ago
    Sounds like the standard way to teach fluids for civ.eng. All the flows you'll ever encounter will be turbulent, no point in learning about all the beautiful results that are mainly for laminar flows. The books I mentioned are classics in the field and were first published in 1895 and 1967, respectively. Both are still in print. No computers, just advanced math (vector calculus etc).

marcosdumay 7y ago

> Now we might have a model that fulfills (2) but we cannot be sure because we used a black-box approach. It's not like the alternative is any different. Being sure you have the right answer is not viable, the other methods of getting there are just more transparent heuristics.

neltnerb 7y ago

3a. We took the best results from ML and did full quantum mechanical simulations on the much smaller set of remaining candidates.

gumby 7y ago

Think of it as an engineering problem rather than a science problem. In much of drug discovery/development (and disease research), being able to predict protein structure would be very valuable. Being able to quickly find candidate structures (that can then be searched for in the lab) speeds things up immensely. Reducing false positives (or just coming up with possibilities at all) is a huge win. But you’re right that this probably doesn’t help the protein theoretician much if at all. We already “know” how it works (it’s just thermodynamics and quantum mechanics) and of course have no idea how it works (“well they wiggle around until they find a low energy state” doesn’t really tell you anything). But that doesn’t keep this from being exciting.

irq11 7y ago
Protein structure prediction, at the current levels of precision (and I include AlphaFold here), is not useful for drug discovery. It’s the sort of thing researchers say to get grants, but as a distant goal, not a practical reality. For structure-based drug discovery (which isn’t even the majority of drug discovery), the details are what matter (e.g. “does this water molecule mediate a binding interaction, or do the sidechains shuffle a bit, and kick the water out?”), and these methods don’t even come close to predicting detailed interactions. Metrics in this space are focused on “general correctness” of protein backbone conformation. Success is to achieve a kind of blurry view of the overall shape of the molecule, and drug design is trying to predict specific atomic interactions. They’re two wildly different problems. About the best you can say is that if we had a generalizable model of physics that could predict protein structure, it might also be able to do a good job of evaluating how a small molecule binds to a protein target. But even that is a huge leap, and when you start using black-box methods like AlphaFold to specifically solve the problem of structure prediction, it’s not really clear that generalization is even possible. There are potential practical uses in drug discovery for a method that can design a protein which takes a particular shape, but even that is pretty different from what AlphaFold actually does.
- lucidrains 7y ago
  Not everyone is focused on solving folding. It's just one step of many. https://www.nature.com/articles/s41592-019-0666-6 https://www.nature.com/articles/s41592-019-0666-6 https://github.com/yangkky/Machine-learning-for-proteins https://github.com/yangkky/Machine-learning-for-proteins

not_qe 7y ago

I have seen the word “watershed” and immediately guessed some it’s some biased unverifiable claptrap from google.

AlexCoventry 7y ago
They provide code and data for full verification. https://github.com/deepmind/deepmind-research/blob/master/alphafold_casp13/README.md https://github.com/deepmind/deepmind-research/blob/master/al...
- not_qe 7y ago
  Will they provide thousands of tensor cpus as well
mda 7y ago
Except that they won a competition for protein folding by a huge margin as explained. Such claptrap indeed.
- deleted 7y ago
  [deleted]
- not_qe 7y ago
  Review the casps13 results pages and then tell about “huge margin”
  - mda 7y ago
    http://predictioncenter.org/casp13/zscores_final.cgi?formula=assessors http://predictioncenter.org/casp13/zscores_final.cgi?formula...
    - not_qe 7y ago
      First place z score sum 54 vs second place 42 and in a single category. The only thing watershed here is the headline dishonesty.