No, there wouldn't be an implication toward performance by "not formalized." I mean the function is obfuscated by its encoding in the model. To formalize, you'd

No, there wouldn't be an implication toward performance by "not formalized." I mean the function is obfuscated by its encoding in the model. To formalize, you'd need to pick apart those encodings and derive explicit formula and conjectures. The models in situ are essentially black boxes, but that can be fixed. The tools to do that with transformers in particular and neural networks in general are still pretty new and specific to particular models. https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html https://ai.googleblog.com/2015/06/inceptionism-going-deeper-... Someone with extensive knowledge of protein folding and chemistry and any relevant domain would need to manually categorize different neurons, their contexts, and effects, then experiment with the model in a static/procedural inference mode, then parse out the overall functions. By reproducing well known simple instances and tracing the propagation of information through the network, you can validate what a model knows by mapping to current knowledge. Using the model in this way can also lead to interesting real world experiment choices, by processing targets suggested by real life phenomena and then coordinating from known functions of the model and real life experiment. Different proteins in coronavirus, for example, would likely have a different cascade of activations in the model than the flu virus. Any shared features between the two might be captured in the activation patterns, possibly leading to new information about the diseases that would opaque to science without these models.