Even if the outputs are indistinguishable there could be different internal algorithms and different computational efficiencies. In a way that is what these aut

Even if the outputs are indistinguishable there could be different internal algorithms and different computational efficiencies. In a way that is what these authors and Chomsky and probably other skeptics are concerned about: with a black box it lets the other faction of scientists off the hook, they can just claim ChatGPT is a bona fide model.. but it's a black box so we don't know how it learned English. We don't even know how ChatGPT learned the grammars for C++ and other programming languages and whether its internal learned algorithm is like or unlike a context-free grammar formalism which is what we learn to write a compiler, i.e. a grammar that is mathematically clearly defined and yet a neural network can learn it. So it's an interesting and problematic debate. I think it would be an interesting computer science experiment, if ChatGPT scientists showed that machine could simply learn programming grammar by brute force. They could then formally prove that the information in the trained network eventually contains the actual grammar formalism that defines the programming language. Thus by restricting the domain, that could shed some light on how much the thing is actually learning it completely, v.s. by "super-autocompletion". With a programming language there's no excuse that it didn't learn the formalism, with English language it is not practically definable, maybe not even definable in principle.