3 ms·
My observations suggest that frontier models consistently produce correct, high-quality code for small to medium-sized tasks provided the goals are well-specifi
by bitwize 1mo ago
My observations suggest that frontier models consistently produce correct, high-quality code for small to medium-sized tasks provided the goals are well-specified and testable. Even when they appear to get distracted, they converge on a solution.
So what that means is that you the software engineer need to be doing what you should have been doing all along: start with systems analysis, stepwise refine that into a program specification, then break that into tasks to give to your programmers to implement. Per PRIDE, the sadly forgotten but comprehensive common-sense information systems methodology, the analysis and specification phases should take up the bulk of the time on a project, with actual programming taking up 15% or less. LLMs can drive that time asymptotically towards zero for human developers. The specification is the deliverable. As a former boss of mine put it, you should be able to delete the entire source code repository and get functionally the exact same software back by running the spec through the LLM.
- shakna 1mo agoITIL is at version 5 now - the idea of a process, is not exactly forgotten in the industry. But you're working with a stochastic system. Even GPU matrix calculations are nondeterministic. It isn't possible, to the delete the entire source code and get exactly the same software back. And any spec, that is specific and non-ambiguous enough to carefully construct the exact needs of each is... Well, that's code. Not a PBI, not an Epic. It's code.
- bitwize 1mo agoI said "functionally", not (as Claude says) "byte-identically".
- shakna 1mo agoI said Epic and PBI, not bytes. User Stories are ambiguous. Acceptance Criteria can be argued. And which path in our languages ambiguities that the model will follow, cannot be deterministic by the nature of our hardware. If you produce a spec that is entirely unambiguous in every way, again, that's code. That's the purpose of a programming language - distilling from the natural ambiguities of natural language, to specificity.