I think I get your point. I can't say that I've observed it writing boilerplate just for the sake of it, but it is disproportionately/abundantly present in the

I think I get your point. I can't say that I've observed it writing boilerplate just for the sake of it, but it is disproportionately/abundantly present in the training set. One of the first rules they teach in ML is to balance datasets, even if it's with fake/interpolated samples. The linear L->R generation is definitely a thing, it's much more costly for an LLM to iterate edits, where a skilled vim coder will be jumping all over the place, trying to make all the LEGO pieces fit. The skill therefore relies on just being able to one-shot entire chunks of code correctly, and it's amazingly good at this... But even the SOTA models still have a lot of unused imports and unused variable declarations. They just have to "guess" what they'll need and hope for the best. If they include a mass of numpy/scipi/react/icon imports that they might need, it opens the landscape for them later on when predicting relevant tokens, reaching a more ideal solution. It doesn't hurt to add imports that might be helpful, rather than penalize the solution because you haven't got them. Although the last few SOTA models are more "harness/tool aware", they're starting to have the instinct to write the code anyway, and to be allowed to go and fix the imports later via tool calls. For anyone who's seen the film Arrival (2016), their entire language is formed of complete concepts, not sequences of words and time. I keep thinking back to this.