The information needs to propagate through the network either forward (when the model has the codebase in context) or backward (when it updates its weights). Y

The information needs to propagate through the network either forward (when the model has the codebase in context) or backward (when it updates its weights). You can have the models pseudo “learn” by putting things in something like a system prompt but this is limited by context, and they will never permanently learn. But we don’t train at inference time with today’s LLMs. We can explicitly reject this possibility by looking at the information that goes into the model at train and test time.