On the contrary: they're bad at it because our examples are too good. There is information in documentation (requirements, rationales, performance characteristi

On the contrary: they're bad at it because our examples are too good. There is information in documentation (requirements, rationales, performance characteristics, safety concerns) that cannot be deduced from the code. The language model anticipates that such information would be present, so it writes it anyway – despite having no idea what it should say. If you're lucky, you get something bland and verbose, but not incorrect. If you're unlucky, you get something dangerously wrong that a later reader assumes is true / important, since "otherwise, why would they have put it in the documentation?". This is part of why I refuse to work on codebases which LLMs have mucked about with. Even the most basic things cannot be relied upon. (Note: refusal can be overridden by suitable application of financial compensation.)