There's no technological use for neuroscience analysis tools to model a microchip, nor for NLP modeling source code, but that isn't the point. It's a potentiall

There's no technological use for neuroscience analysis tools to model a microchip, nor for NLP modeling source code, but that isn't the point. It's a potentially useful target to hone the methodology itself on. I actually don't think something like BERT would do so well at learning to essentially execute code. The counterexample is for a transformer with context width of N tokens, to try to model "def f(x) { ... }; <at least N more tokens here>; def g(x) { y = f(x) ...};". In this case, in trying to evaluate "g(4)", f appears as the literal token "f" and nothing more. To be able to learn to evaluate g, a model would need to be able to do (at least) distant coreference resolution.