4 ms·
That’s the thing, isn’t it? The craft of programming in the small is one of being intimate with the details, thinking things through conscientiously. LLMs don’t
by enneff 2y ago
That’s the thing, isn’t it? The craft of programming in the small is one of being intimate with the details, thinking things through conscientiously. LLMs don’t do that.
- Nevermark 2y agoPerhaps it should be prompted to then? Ask it to review its own code for any problems? Also identify typical and corner cases and generate tests? Question marks here because I have not used the tool. The size & depth of each accepted code step is still up to the developer slash prompter
- nrclark 2y agoI use Chatgpt for coding / API questions pretty frequently. It's bad at writing code with any kind of non-trivial design complexity. There have been a bunch of times where I've asked it to write me a snippet of code, and it cheerfully gave me back something that doesn't work for one reason or another. Hallucinated methods are common. Then I ask it to check its code, and it'll find the error and give me back code with a different error. I'll repeat the process a few times before it eventually gets back to code that resembles its first attempt. Then I'll give up and write it myself. As an example of a task that it failed to do: I asked it to write me an example Python function that runs a subprocess, prints its stdout transparently (so that I can use it for running interactive applications), but also records the process's stdout so that I can use it later. I wanted something that used non-blocking I/O methods, so that I didn't have to explicitly poll every N milliseconds or something.
- bongodongobob 2y agoHonestly I find that when GPT starts to lose the plot it's a good time to refactor and then keep on moving. "Break this into separate headers or modules and give me some YAML like markup with function names, return type, etc for each file." Or just use stubs instead of dumping every line of code in.
- tomrod 2y agoHow long are you willing to iterate to get things right?
- bongodongobob 2y agoIf it takes almost no cognitive energy, quite a while. Even if it's a little slower than what I can do, I don't care because I didn't have to focus deeply on it and have plenty of energy left to keep on pushing.
- Nevermark 2y agoAs my mother used to say, "I love work. I could watch it all day!" I can see where you are coming from. Maintaining a better creative + technical balance, instead of see-sawing. More continuous conscious planning, less drilling. Plus the unwavering tireless help of these AI's seems psychologically conducive to maintaining one's own motivation. Even if I end up designing an elaborate garden estate or a simpler better six-axis camera stabilizer/tracker, or refactoring how I think of primes before attempting a theorem, ... when that was not my agenda for the day. Or any day.
- Bjartr 2y agoI'm constantly having to go back and tell the AI about every mistake it makes and remind it not to reintroduce mistakes that were previously fixed. "no cognitive energy" is definitely not how I would describe that experience.
- bongodongobob 2y agoSounds like the context window is getting pruned. Start a new chat fresh after you make significant changes.
- EVa5I7bHFq9mnYK 2y agoThat's presumably what o1-preview does? Iterates and checks the result. It takes much longer, but does indeed write slightly better code.
- __MatrixMan__ 2y agoI find that it depends very heavily on what you're up to. When I ask it to write nix code it'll just flat out forget how the syntax works half way though. But if I want it to troubleshoot an emacs config or wield matplotlib it's downright wizardly, often including the kind of thing that does indicate an intimacy with the details. I get distracted because I'm then asking it: > I un-did your change which made no sense to me and now everything is broken, why is what you did necessary? I think we just have to ask ourselves what we want it to be good at, and then be diligent about generating decades worth of high quality training material in that domain. At some point, it'll start getting the details right.
- esafak 2y agoThat doesn't work in the tech industry, because almost nothing is decades old, for obvious reasons.
- __MatrixMan__ 2y agoWhat languages/toolkits are you working with that are less than 10 years old? Anyhow, it seems to me like it is working. It's just working better for the really old stuff because: - there has been more time for training data to accumulate - some of it predates the trend of monetizing data, so there was less hoarding and more sharing It may be that the hard slow way is the only way to get good results. If the modern trends re: products don't have the longevity/community to benefit from it, maybe we should fix that.