5 ms·

I don't agree. The specific techniques we used (a variant of the transformer architecture behind most large language models) means, among other things, that y

I don't agree. The specific techniques we used (a variant of the transformer architecture behind most large language models) means, among other things, that you can ask the model questions, and get answers about what the model "knows" in return. So, if you train a model on thousands of scientific papers that contain information about drug effects on biomarkers, side effects, and the like, the model will learn, e.g., that drug A is typically used to treat cancer of type beta, and that it is frequently associated with lowering of biomarker X. It can then answer questions such as "what drugs are useful for lowering biomarker X in patients?" It can further answer a question "what mechanisms could be associated with this effect?" Of course, it can only answer with mechanisms that are identified in the literature it analyzed - it's not a PhD biochemist / M.D. able to infer or speculate on mechanism from first biochemical and medical principles. But there is no way to interpret what it can do, in my mind, other than to say that the model has learned drug biochemistry at some significant level, and can represent that learning to a user. In the case of what Deep Mind has done with protein folding, I don't know how you interpret its capabilities other than to say their model has learned much of the fundamental body of rules and facts that govern how amino acid sequences role up into proteins. There is a very real sense in which it embodies a theory of protein folding, much as say, Maxwell's equations and a knowledge of calculus embodies a theory of electromagnetism. That it cannot as yet articulate the rules of that theory is clearly a profound limitation, but it's a limitation of conveyance, not of utility.

AnimalMuppet 4y ago
Did Deep Mind get that from reading a bunch of papers, though? Or did it get it from having those rules and facts programmed into it? (The original claim was about the ability of AIs to extract the rules from the literature.) But your last sentence... if it had the rules programmed in, shouldn't it have also programmed in the ability to name the rules? If it learned them from reading papers, shouldn't it know the terms from the papers? Or is the problem that it doesn't have at all the ability to trace out the chain it used to reach the conclusion? Is the problem not so much expressing the chain as much as it is knowing the chain?
- walnutclosefarm 4y ago
  You raise good questions. I'm mixing two different types of models to make a point from two different directions. Each has it's own and different limitations. Deep Mind Alpha Fold did not get the rules from reading papers. It uses a combination of learned (from being trained on known protein structures) prediction of structure, and iterative application of learned refinement, constrained by physical and chemical rules. So, some of the knowledge required is "programmed in" as you suggest, but most has been learned by the algorithm through training on known structures. My point was that the result is nevertheless a workable effective procedure for predicting protein structures. The other system that I referred to did learn from reading literature. It is a fundamentally different AI architecture (a variant of transformer architecture for large language models). It has no physical knowledge programmed in.
deleted 4y ago
[deleted]
YeGoblynQueenne 4y ago
>> The specific techniques we used (a variant of the transformer architecture behind most large language models) means, among other things, that you can ask the model questions, and get answers about what the model "knows" in return. You know, for me it is clear that the ability to retrieve information is not the same as the ability to understand the retrieved information. For example, you could store a corpus of scientific texts in a database and then retrieve pieces of it with a query, but that doesn't say the database understands the corpus. And this is relevant to your comment because with transformers we still have essentially databases of information that can only be retrieved by the right query, where the information is stored as weights in the trained model and the query is a suitable prompt. In fact, prompts to transformers are so finicky and must be tailored so carefully to retrieve the desired piece of information that they are not very different than a SQL query, albeit one in natural language. I'm offering this as a counterpoint to your assertion that "the model has learned drug biochemistry at some significant level". What I understand from your comment is that your model has learned to represent a body of text, nothing more, nothing less.
- walnutclosefarm 4y ago
  I'm not claiming that any of these models have achieved a generalized level of scientific understanding. They are narrow in their abilities - savants of a sort. But it seems to me to be missing an important point to reduce them to mere databases. Learned knowledge and intelligent behavior are multi-dimensional, and I think we're seeing a different sampling with the current generation of AI models than we see in our own sampling. They are highly specific, and refined for one "intelligent" task, but not in any way broadly general. We are comparing them to human intelligence, which leans heavily in the other direction.
  - YeGoblynQueenne 4y ago
    I don't see it that way. What I see is systems that have behaviours that look like human behaviours - for example, generating and recognising text, or recognising images, etc; but what these systems really do is nothing like whatever humans do, and the behaviours they produce are not the same behaviours as those of humans, after all. For example, we can see that clearly in chess and other board games. Chess engines play by executing some variant of the minimax algorithm and they can out-play humans. But human players don't typcially try to run minimax in their heads when they play and in any case can play chess without trying to emulate a chess engine. The same goes for image classifiers (it is extremely unlikely that we match sets of pixels to numerical labels in our heads, to the point of absurdity) and for language models (again it's absurdly unlikely that we calculate the probabilities of sequences of tokens). I think what you say comes from an assumption that because humans exhibit certain behaviours, and humans have human intelligence, any system that exhibits similar behaviours must also have human intelligence. Which, if you look carefully, does not follow. Anyway historically people have always been quick to identify the capabilities of computers as artificial intelligence, just because we tend to think of anything to do with computation as requiring intelligence. But we had machines that could perform calculations automatically, through entirely mechanical procedures, for more than 200 years now, and most of those we don't think are artificially intelligent. For example, I don't think anyone would consider the Pascaline to be an AI, and yet the Pascaline can do better arithmetic from any non-human animal, and from many human animals, also. So, does arithmetic require intelligence, or not? How much of what we can do thanks to our intelligence can also be done by a dumb, mechanical automatism? This I think is the important question in AI research today and one that, if we could answer it (which we can't) would finally put the lid on wishful thinking about LLMs exhibiting "a different sampling" of intelligence than ours.
mdp2021 4y ago
> But there is no way to interpret what it can do, in my mind Would not you say instead that «model has learned /given statements about/ drug biochemistry at some significant level»? You say it yourself that it cannot «infer or speculate on mechanism from first biochemical and medical principles». Students should not pass exams because they have memorized the book - they would be said to "have memorized the notions" and not to "have learned the matter". (We may note that, linguistically, 'learn' has a strong connotation of activity: it is "track chasing", so "investigating".) > it embodies a theory of protein folding This reminds me of the surprise researchers got from some AI (possibly even classical - simulated annealing?) produced models for microprocessors, where the "monstrous" (unordered) layout was discovered to have used important non-processing chunks of immediately unexplicable circuitry just for thermal properties and timing. The AI can come up to an effective solution, which can be theoretically very much not clean.

AnimalMuppet 4y ago

Did Deep Mind get that from reading a bunch of papers, though? Or did it get it from having those rules and facts programmed into it? (The original claim was about the ability of AIs to extract the rules from the literature.) But your last sentence... if it had the rules programmed in, shouldn't it have also programmed in the ability to name the rules? If it learned them from reading papers, shouldn't it know the terms from the papers? Or is the problem that it doesn't have at all the ability to trace out the chain it used to reach the conclusion? Is the problem not so much expressing the chain as much as it is knowing the chain?

walnutclosefarm 4y ago
You raise good questions. I'm mixing two different types of models to make a point from two different directions. Each has it's own and different limitations. Deep Mind Alpha Fold did not get the rules from reading papers. It uses a combination of learned (from being trained on known protein structures) prediction of structure, and iterative application of learned refinement, constrained by physical and chemical rules. So, some of the knowledge required is "programmed in" as you suggest, but most has been learned by the algorithm through training on known structures. My point was that the result is nevertheless a workable effective procedure for predicting protein structures. The other system that I referred to did learn from reading literature. It is a fundamentally different AI architecture (a variant of transformer architecture for large language models). It has no physical knowledge programmed in.

deleted 4y ago

[deleted]

YeGoblynQueenne 4y ago

>> The specific techniques we used (a variant of the transformer architecture behind most large language models) means, among other things, that you can ask the model questions, and get answers about what the model "knows" in return. You know, for me it is clear that the ability to retrieve information is not the same as the ability to understand the retrieved information. For example, you could store a corpus of scientific texts in a database and then retrieve pieces of it with a query, but that doesn't say the database understands the corpus. And this is relevant to your comment because with transformers we still have essentially databases of information that can only be retrieved by the right query, where the information is stored as weights in the trained model and the query is a suitable prompt. In fact, prompts to transformers are so finicky and must be tailored so carefully to retrieve the desired piece of information that they are not very different than a SQL query, albeit one in natural language. I'm offering this as a counterpoint to your assertion that "the model has learned drug biochemistry at some significant level". What I understand from your comment is that your model has learned to represent a body of text, nothing more, nothing less.

walnutclosefarm 4y ago
I'm not claiming that any of these models have achieved a generalized level of scientific understanding. They are narrow in their abilities - savants of a sort. But it seems to me to be missing an important point to reduce them to mere databases. Learned knowledge and intelligent behavior are multi-dimensional, and I think we're seeing a different sampling with the current generation of AI models than we see in our own sampling. They are highly specific, and refined for one "intelligent" task, but not in any way broadly general. We are comparing them to human intelligence, which leans heavily in the other direction.
- YeGoblynQueenne 4y ago
  I don't see it that way. What I see is systems that have behaviours that look like human behaviours - for example, generating and recognising text, or recognising images, etc; but what these systems really do is nothing like whatever humans do, and the behaviours they produce are not the same behaviours as those of humans, after all. For example, we can see that clearly in chess and other board games. Chess engines play by executing some variant of the minimax algorithm and they can out-play humans. But human players don't typcially try to run minimax in their heads when they play and in any case can play chess without trying to emulate a chess engine. The same goes for image classifiers (it is extremely unlikely that we match sets of pixels to numerical labels in our heads, to the point of absurdity) and for language models (again it's absurdly unlikely that we calculate the probabilities of sequences of tokens). I think what you say comes from an assumption that because humans exhibit certain behaviours, and humans have human intelligence, any system that exhibits similar behaviours must also have human intelligence. Which, if you look carefully, does not follow. Anyway historically people have always been quick to identify the capabilities of computers as artificial intelligence, just because we tend to think of anything to do with computation as requiring intelligence. But we had machines that could perform calculations automatically, through entirely mechanical procedures, for more than 200 years now, and most of those we don't think are artificially intelligent. For example, I don't think anyone would consider the Pascaline to be an AI, and yet the Pascaline can do better arithmetic from any non-human animal, and from many human animals, also. So, does arithmetic require intelligence, or not? How much of what we can do thanks to our intelligence can also be done by a dumb, mechanical automatism? This I think is the important question in AI research today and one that, if we could answer it (which we can't) would finally put the lid on wishful thinking about LLMs exhibiting "a different sampling" of intelligence than ours.

mdp2021 4y ago

> But there is no way to interpret what it can do, in my mind Would not you say instead that «model has learned /given statements about/ drug biochemistry at some significant level»? You say it yourself that it cannot «infer or speculate on mechanism from first biochemical and medical principles». Students should not pass exams because they have memorized the book - they would be said to "have memorized the notions" and not to "have learned the matter". (We may note that, linguistically, 'learn' has a strong connotation of activity: it is "track chasing", so "investigating".) > it embodies a theory of protein folding This reminds me of the surprise researchers got from some AI (possibly even classical - simulated annealing?) produced models for microprocessors, where the "monstrous" (unordered) layout was discovered to have used important non-processing chunks of immediately unexplicable circuitry just for thermal properties and timing. The AI can come up to an effective solution, which can be theoretically very much not clean.