4 ms·

It's likely that many of the patterns it learned are encoded understanding of the form you mention, but not at a formal level of explication. The architecture

It's likely that many of the patterns it learned are encoded understanding of the form you mention, but not at a formal level of explication. The architecture of the system and design of the training methodology are laid out to specifically prevent direct database-esque "pattern in pattern out" failure mode. Similar to Google deep dream, there will be contextual features and feature clusters encoded into neurons that can be explored and extracted, and those could provide insights that can be sourcing translated into "hard" science, with explicit formulae and theory allowing a fully transparent model to be created. Like other transformer models, you can elicit the training data intact, but such scenarios are a statistically insignificant selection of the range of outputs the models are capable of producing. That doesn't mean anything with regards to accuracy of the novel output, though. With alphafold 2 going open source, it's possible that tools and methodologies to extract hard science from transformers will be formalized quickly and in the public eye. We have an amazingly powerful new tool, and the coming decades will be fascinating.

joe_the_user 5y ago
It's likely that many of the patterns it learned are encoded understanding of the form you mention, but not at a formal level of explication. - The thing I'd be curious about is whether or not "not formalized" would imply "not consistently generalizing", whether it would have to be train all over if given a problem similar to but identical with, the problem it solves.
- robbedpeter 5y ago
  No, there wouldn't be an implication toward performance by "not formalized." I mean the function is obfuscated by its encoding in the model. To formalize, you'd need to pick apart those encodings and derive explicit formula and conjectures. The models in situ are essentially black boxes, but that can be fixed. The tools to do that with transformers in particular and neural networks in general are still pretty new and specific to particular models. https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html https://ai.googleblog.com/2015/06/inceptionism-going-deeper-... Someone with extensive knowledge of protein folding and chemistry and any relevant domain would need to manually categorize different neurons, their contexts, and effects, then experiment with the model in a static/procedural inference mode, then parse out the overall functions. By reproducing well known simple instances and tracing the propagation of information through the network, you can validate what a model knows by mapping to current knowledge. Using the model in this way can also lead to interesting real world experiment choices, by processing targets suggested by real life phenomena and then coordinating from known functions of the model and real life experiment. Different proteins in coronavirus, for example, would likely have a different cascade of activations in the model than the flu virus. Any shared features between the two might be captured in the activation patterns, possibly leading to new information about the diseases that would opaque to science without these models.

joe_the_user 5y ago

It's likely that many of the patterns it learned are encoded understanding of the form you mention, but not at a formal level of explication. - The thing I'd be curious about is whether or not "not formalized" would imply "not consistently generalizing", whether it would have to be train all over if given a problem similar to but identical with, the problem it solves.

robbedpeter 5y ago
No, there wouldn't be an implication toward performance by "not formalized." I mean the function is obfuscated by its encoding in the model. To formalize, you'd need to pick apart those encodings and derive explicit formula and conjectures. The models in situ are essentially black boxes, but that can be fixed. The tools to do that with transformers in particular and neural networks in general are still pretty new and specific to particular models. https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html https://ai.googleblog.com/2015/06/inceptionism-going-deeper-... Someone with extensive knowledge of protein folding and chemistry and any relevant domain would need to manually categorize different neurons, their contexts, and effects, then experiment with the model in a static/procedural inference mode, then parse out the overall functions. By reproducing well known simple instances and tracing the propagation of information through the network, you can validate what a model knows by mapping to current knowledge. Using the model in this way can also lead to interesting real world experiment choices, by processing targets suggested by real life phenomena and then coordinating from known functions of the model and real life experiment. Different proteins in coronavirus, for example, would likely have a different cascade of activations in the model than the flu virus. Any shared features between the two might be captured in the activation patterns, possibly leading to new information about the diseases that would opaque to science without these models.