He's saying that pre-training an LLM alone can't do it, but if you run an LLM in a loop with tools (like any coding agent) then it can. Also, the technique his

He's saying that pre-training an LLM alone can't do it, but if you run an LLM in a loop with tools (like any coding agent) then it can. Also, the technique his group came up with should be used more: > This is the weakness of deep learning that is alleviated with a new algorithm that my group presented in Nature a couple of years ago. Our “continual backpropagation” made one small change: every so often a less-used neuron would be re-initialized to small random weights. This allows the variation to continue and plasticity to be retained. Here's the paper: https://www.nature.com/articles/s41586-024-07711-7 https://www.nature.com/articles/s41586-024-07711-7 It has a fair number of citations, but I haven't looked into how much it's used.