4 ms·
> The people saying LLM can code are hard for me to understand. Just today, I spent an hour documenting a function that performs a set of complex scientific si
by crazygringo 1y ago
> The people saying LLM can code are hard for me to understand.
Just today, I spent an hour documenting a function that performs a set of complex scientific simulations. Defined the function input structure, the outputs, and put a bunch of references in the body to function calls it would use.
I then spent 15 minutes explaining to the free version of ChatGPT what the function needs to do both in scientific terms and in computer architecture terms (e.g. what needed to be separated out for unit tests). Then it asked me to answer ~15 questions it had (most were yes/no, it took about 5 min), then it output around 700 lines of code.
It took me about 5 minutes to get it working, since it had a few typos. It ran.
Then I spent another 15 minutes laying out all the categories of unit tests and sanity tests I wanted it to write. It produced ~1500 lines of tests. It took me half an hour to read through them all, adjusting some edge cases that didn't make sense to me and adjusting the code accordingly. And a couple cases where it was testing the right part of the code, but had made valiant but wrong guesses as to what the scientifically correct answer would be. All the tests then passed.
All in all, a little over two hours. And it ran perfectly. In contrast, writing the code and tests myself entirely by hand would have taken at least a couple of entire days.
So when you say they're good for those simple things you list and "that's about it", I couldn't disagree more. In fact, I find myself relying on them more and more for the hardest scientific and algorithmic programming, when I provide the design and the code is relatively self-contained and tests can ensure correctness. I do the thinking, it does the coding.
- DougWebb 1y ago> Just today, I spent an hour documenting a function that performs a set of complex scientific simulations. Defined the function input structure, the outputs, and put a bunch of references in the body to function calls it would use. So that's... math. A very well defined problem, defined very well. Any decent programmer should be able to produce working software from that, and it's great that ChatGPT was able to help you get it done much faster than you could have done it yourself. That's also the kind of project that's very well suited for unit testing, because again: math. Functions with well defined inputs, outputs, and no side-effects. Only a tiny subset of software development projects are like that though.
- simonw 1y ago> Only a tiny subset of software development projects are like that though. Right: the majority of software development is things like "build a REST API for these three database tables" or "build a contact form with these four fields" or "write unit tests for this new function" or "update my YAML CI configuration to run this extra command".
- skydhash 1y agoYou do know that system programming is a thing? Or that desktop applications are software too?
- simonw 1y agoI said "the majority of software development". Those are both relatively niche disciplines in 2025.
- somebehemoth 1y agoCan you please explain? Are you saying all software development outside of the web is "niche"?
- theshrike79 1y agoNiche as in for every one systems programmer there are dozens of people writing API Glue. By hours of work spent and lines of code produced the latter is in a whole different scale than systems programmers (which is a very badly designed term anyway).
- somebehemoth 1y agoNon web programming is not niche by any definition of the word niche.
- simonw 1y ago
- prmph 1y ago> documenting a function that performs a set of complex scientific simulations. The example you gave sounds like the problem is deterministic, even if composed of many moving parts. That's one way of looking at complexity. When I talk about complex problems I'm not just talking about intricate problems. I'm talking about problems where the "problem" is design, not just implementing a design, and that is where LLMs struggle a lot. Example, I want to design a strongly typed fluent API interface to some functionality. Even knowing how to shape the fluent interface so that is powerful, intuitive, well/strongly typed, and maintainable is a deep art. The intuitive design constraints that I'm designing under would be hard to even explain to an LLM.
- simonw 1y agoFor the problems like that I consider my role to be the expert designer. I figure out a the design, then get the LLM to write the code and the tests for me. It is a lot faster at typing than I am.