> One central factor behind this failure is the learning method: LLMs are trained using backpropagation, which pushes the network in very un-scientist-like dire

> One central factor behind this failure is the learning method: LLMs are trained using backpropagation, which pushes the network in very un-scientist-like directions and prevents it from generalizing and reasoning about inputs in anything even remotely similar to how humans generalize. It's hard to take this backpropogation dis as fact given the performance and progression of AlphaGo, AlphaGoZero, AlphaZero, MuZero. > When instead of backpropagation we trained networks using Minimum Description Length (MDL) — a learning criterion that does correspond to rational scientific reasoning — the networks were able to find perfect solutions to patterns that remained outside the reach of networks trained with backpropagation. Could there eventually be future LLMs (MDL-based or otherwise) that would strike us as scientist-like? Perhaps. But if such models do arrive they will be very different from current LLMs. In the meantime, any suggestion that LLMs are automated scientists should be treated with suspicion. Seems like the critique is specifically about how current LLMs operate and not about them in principle.