5 ms·
It is primarily a principal agent problem, with a hint of marshmallow test. If you are a developer who is not writing the documents for consumption by AI, you
by 7thpower 1y ago
It is primarily a principal agent problem, with a hint of marshmallow test.
If you are a developer who is not writing the documents for consumption by AI, you are primarily writing documents for someone who is not you; you do not know what this person will need or if they will ever even look at them.
They may, of course, help you, but you may not understand that, have the time, or discipline.
If you are writing them because the AI using them will help you, you have a very strong and immediate incentive to document the necessary information. You also have the benefit of a short feedback loop.
Side note, thanks to the LLMs penchant of wiping out comments, I have a lot more docs these days and far fewer comments.
- Wowfunhappy 1y agoYour LLMs get rid of comments? Mine add them incessantly.
- ls612 1y agoI have to yell at Gemini to not add so many comments it almost writes more comments than code by default.
- hu3 1y agocheck your AGENTS.md or equivalent most of the times when LLM is misbehaving, it's my fault for leaving outdated instructions
- taormina 1y agoNah, it's definitely not that. I have it explicitly several times over in that file and it insists on commenting so much it's absurd.
- lukeschlather 1y agoI think it's very valuable. If there are a bunch of redundant comments I know I haven't actually validated this code does anything useful. I go through line by line and delete all the comments while validating that they do in fact reflect what the code does.
- evanmoran 1y agoI know this is highly controversial, but I now leave the comments in. My theory is that the “probability space” the LLM is writing code in can’t help but write them, so if i leave them next LLM that reads the code will start in the same space. Maybe it’s too much, but currently I just want the code to be right and I’ve let go of the exact wording of comments/variables/types to move faster.
- jychang 1y agoI think the code comments straight up just help understanding, whether human or AI. There's a piece of common knowledge that NBA basketball players can all hit over 90% on free throws, if they shot underhand (granny style). But for pride reasons, they don't throw underhand. Shaq just shot 52%, even though it'd be free points if he could easily shoot better. I suspect there's similar things in software engineering. I've seen plenty of comments on HN about "adding code comments like a junior software engineer" or similar sentiment. Sure, there's legitimate gripes about comments (like how they can be misleading if you update the code without changing the comment, etc), but I strongly suspect they increase comprehension of code overall.
- thenanyu 1y agoI hate this analogy. NBA players can all hit 90% of their free throws shooting overhand too. Just some of them are much worse at handling the pressure and pace change of the situation in a game context.
- oblio 1y agoThe underhanded throw is mechanically just better for free throws. Much easier to put backspin, for example. It's just a shot that doesn't help anywhere else.
- thenanyu 1y agoThat’s beside the point. It wouldn’t help shaq or any of those big men hit free throws. They have no problems with mechanics.
- emn13 1y agoI think it's not at all a marshmellow test; quite the opposite - docs used to be written way, way in advance of their consumption. The problem that implies is twofold. Firstly, and less significantly, it's just not a great return on investment to spend tons of effort now to maybe help slightly in the far future. But the real problem with docs is that for MOST usecases, the audience and context of the readers matter HUGELY. Most docs are bad because we can't predict those. People waste ridiculous amounts of time writing docs that nobody reads or nobody needs based on hypotheses about the future that turn out to be false. And _that_ is completely different when you're writing context-window documents. These aren't really documents describing any codebase or context within which the codebase exists in some timeless fashion, they're better understood as part of a _current_ plan for action on a acute, real concern. They're battle-tested the way docs only rarely are. And as a bonus, sure, they're retainable and might help for the next problem too, but that's not why they work; they work because they're useful in an almost testable way right away. The exceptions to this pattern kind of prove the rule - people for years have done better at documenting isolatable dependencies, i.e. libraries - precisely because those happen to sit at boundaries where it's both easier to make decent predictions about future usage, and often also because those docs might have far larger readership, so it's more worth it to take the risk of having an incorrect hypothesis about the future wasting effort - the cost/benefit is skewed towards the benefit by sheer numbers and the kind of code it is. Having said that, the dust hasn't settled on the best way to distill context like this. It's be a mistake to overanalyze the current situation and conclude that documentation is certain to be the long-term answer - it's definitely helpful now, but it's certainly conceivable that more automated and structured representations might emerge, or in forms better suited for machine consumption that look a little more alien to us than conventional docs.
- neves 1y agoI always write documentation thinking of myself in the future.