4 ms·
This feels like a job ripe for startup disruption. LLM documentation generators tend to benefit from context, and nothing provides better context than a mostly
by cheschire 2y ago
This feels like a job ripe for startup disruption.
LLM documentation generators tend to benefit from context, and nothing provides better context than a mostly functional code base. The best part is the code doesn’t even need to compile for the LLM to build the context needed.
- remoquete 2y agoAuthor here. No, I don't think it does. https://passo.uno/ai-anxiety-tech-writer-howto/ https://passo.uno/ai-anxiety-tech-writer-howto/
- chaffroomba 2y agoProblem I'm having as a developer with LLM documentation is their reliability, or rather lack of it. Every time there is an assertation I end up having to double-confirm it because they tend to be wrong as often as they're right. Reading imaginary hallucinated documentation is just about as useful as zero documentation. While I could keep doing this for the rest of my life, my employer doesen't really appreciate the extra expense. A technical writer is much, much cheaper than the dozens of developers trying to confirm the docs.
- exe34 2y ago> Reading imaginary hallucinated documentation is just about as useful as zero documentation. no, it's worse. it's closer to reading outdated documentation that outright lies and gives examples that don't work, and will cause you to waste hours/days learning things that aren't relevant to the api anymore.
- joshuanapoli 2y agoI think we will go the other way: A person will write the documentation and the AI will build the system that delivers the described product.
- chaffroomba 2y agoIsn't that what sw development today is? A description of a system and what we want it to do. With the advance of compilers, libraries, frameworks, linters, autocomplete systems and so on, we're already very close to describing the minimum amount of information the system needs in order to produce the correct result. To my knowledge actually physically writing the software has not been a bottleneck in a very, very long time.
- joshuanapoli 2y agoRight now, it takes skill and labor to move descriptions between representations for business goals, engineering (where we have frameworks and linters, etc.), and external/customer facing documentation. The customer is faced with an output from the design process. I think that we can turn that around now. Let customers edit part of the documentation, and let the AI adapt the system to their need.
- philipwhiuk 2y agohttps://www.commitstrip.com/en/2016/08/25/a-very-comprehensive-and-precise-spec/ https://www.commitstrip.com/en/2016/08/25/a-very-comprehensi...
- euroderf 2y agoI would think that Requirements Engineering would be the ideal prep for AI code generation. A natural language counterpart to (e.g.) TLA+. Is this too damned obvious? Or am I missing something?
- smokel 2y agoThe main problem is that documentation should be written on why code is written the way it is, and why it exists in the first place. This context is typically not available in the code itself. In the best case, this is encoded in requirements and design documentation, but more often than not, the information remains only in the heads of customers, architects, and developers. The code itself merely describes how something is solved. Summarizing that in documentation can be useful at times, but it is not the full story. Especially for code that lives on for some time, the original design philosophy is often lost, and it is forced in horrible directions.
- spencerchubb 2y agoThis article mostly seems to be about documentation for users of the code, not documentation within the code.
- janice1999 2y agoCode doesn't capture architectural decision making, one of the most important being why other solutions were not used. LLMs would need to be nearly omniscient (understanding underlying hardware, customer needs, budgets even) to derive the reasoning behind those decisions afterwards from code alone.
- from-nibly 2y agoA source code repository is actually a terrible place to get that context. All of the things like decisions and how this relates to the customer are completely missing.
- Etheryte 2y agoThis completely misses what valuable code documentation is for. Anyone can read the code and figure out what it does, what as a documentation can be convenient, but it's not really all that useful, especially if it rots over time. Even if it's painful at times, the what is all there in front of you in the code. Valuable documentation explains the why, why was one approach chosen over another, why do we need to do this to begin with, why are the edge cases the way they are. This information is not present in the code and no tool can ever extract it since it isn't there.
- keybored 2y ago> LLM documentation generators I can’t wait for the LLM digression-generators to quit.