3 ms·

My old boss and I used to defend ourselves to younger colleagues with the argument that "This is how you did it back in the day". Mostly it was a joke, to "cove

My old boss and I used to defend ourselves to younger colleagues with the argument that "This is how you did it back in the day". Mostly it was a joke, to "cover up" our screw-ups and "back in the day" could be two weeks ago. Still, for some things we weren't wrong, our weird hacks where do to crazy edge cases or integrations into systems designed in a different era. But we where around to help assess if the code could be yanked or at least attempt to be yanked. LLM assisted coding could technically be better for technical debt, assuming that you store the prompts along side the code. Letting someone what prompt generated a piece of code could be really helpful. Imagine having "ensure to handle the edge case where the client is running AIX 6". That answers a lot of questions and while you still don't know who was running AIX, you can now start investigating if this is still needed.

mplanchard 2y ago
> ensure to handle the edge case where the client is running AIX 6 Regardless if the source was AI or not, this should just be a comment in the code, shouldn’t it? This is exactly the sort of thing I would ask for in code review, so that future authors understand why some weird code or optimization exists.
- mrweasel 2y ago
  It should, but I think most of us read enough old code to know that this doesn't happen as often as we'd like. With an LLM you already wrote the prompt, so if you had an easy way to attach the prompt to the code it could make it more likely that some form of documentation exists. Some times you also fail to write the comment because at the time everyone knew why you did it like that, because that's what everyone did. Now it's 10 years later and everyone doesn't know that. The LLM prompt could still require you to type out the edge case that everyone in your line of business knows about, but might not a generalised across the entirety of the software industry.
  - mplanchard 2y ago
    I'm always a fan of more historical context on code, so maybe it'd be an okay fallback for a proper comment and git blame, but you'd have to find a way to separate the meaningful part of the prompt from the preamble that is implicitly inserted into most prompts ("you are a helpful assistant, only output code, no formatting markers, etc etc"). That said, I suspect it wouldn't actually be that useful for the "why" very often, which is usually what's lacking in these cases. I would guess that far more people prompt the LLMs with the "how," e.g. "write a function that takes these things as input and produces this output," which of course we can just read in the code.
- aledalgrande 2y ago
  Or a good commit/PR description?
int_19h 2y ago
If you have that kind of thing in the prompt, LLMs will almost certainly put it in a comment in the relevant section of the code. Both Gemini and Sonnet are very comment-happy, in my experience, to the point where it's actually detrimental - but it covers your particular scenario for sure.

mplanchard 2y ago

> ensure to handle the edge case where the client is running AIX 6 Regardless if the source was AI or not, this should just be a comment in the code, shouldn’t it? This is exactly the sort of thing I would ask for in code review, so that future authors understand why some weird code or optimization exists.

mrweasel 2y ago
It should, but I think most of us read enough old code to know that this doesn't happen as often as we'd like. With an LLM you already wrote the prompt, so if you had an easy way to attach the prompt to the code it could make it more likely that some form of documentation exists. Some times you also fail to write the comment because at the time everyone knew why you did it like that, because that's what everyone did. Now it's 10 years later and everyone doesn't know that. The LLM prompt could still require you to type out the edge case that everyone in your line of business knows about, but might not a generalised across the entirety of the software industry.
- mplanchard 2y ago
  I'm always a fan of more historical context on code, so maybe it'd be an okay fallback for a proper comment and git blame, but you'd have to find a way to separate the meaningful part of the prompt from the preamble that is implicitly inserted into most prompts ("you are a helpful assistant, only output code, no formatting markers, etc etc"). That said, I suspect it wouldn't actually be that useful for the "why" very often, which is usually what's lacking in these cases. I would guess that far more people prompt the LLMs with the "how," e.g. "write a function that takes these things as input and produces this output," which of course we can just read in the code.
aledalgrande 2y ago
Or a good commit/PR description?

int_19h 2y ago

If you have that kind of thing in the prompt, LLMs will almost certainly put it in a comment in the relevant section of the code. Both Gemini and Sonnet are very comment-happy, in my experience, to the point where it's actually detrimental - but it covers your particular scenario for sure.