9 ms·
Their default solution is to keep digging. It has a compounding effect of generating more and more code. If they implement something with a not-so-great approa
by pornel 8mo ago
Their default solution is to keep digging. It has a compounding effect of generating more and more code.
If they implement something with a not-so-great approach, they'll keep adding workarounds or redundant code every time they run into limitations later.
If you tell them the code is slow, they'll try to add optimized fast paths (more code), specialized routines (more code), custom data structures (even more code). And then add fractally more code to patch up all the problems that code has created.
If you complain it's buggy, you can have 10 bespoke tests for every bug. Plus a new mocking framework created every time the last one turns out to be unfit for purpose.
If you ask to unify the duplication, it'll say "No problem, here's a brand new metamock abstract adapter framework that has a superset of all feature sets, plus two new metamock drivers for the older and the newer code! Let me know if you want me to write tests for the new adapters."
- stingraycharles 8mo ago> If you ask to unify the duplication, it'll say "No problem, here's a brand new metamock abstract adapter framework that has a superset of all feature sets, plus two new metamock drivers for the older and the newer code! Let me know if you want me to write tests for the new adapters." Nevermind the fact that it only migrated 3 out of 5 duplicated sections, and hasn’t deleted any now-dead code.
- Mavvie 8mo agoSounds like my coworkers.
- Foobar8568 8mo agoThat's the reality nobody really wants to say.
- Jweb_Guru 8mo agoIt's not reality. I'm really not a fan of the way that people excuse the really terrible code LLMs write by claiming that people write code just as bad. Even if that were true, it is not true that when you ask those people to do otherwise they simply pretend to have done it and forget you asked later.
- evolve-maz 8mo ago[dead]
- imiric 8mo agoIt's an easy copout. Tool works as expected? It's superintelligence. Programming is dead. Tool makes dumb mistake? So do humans.
- brabel 8mo agoYes and both are right. It’s a matter of which is working as expected and making fewer mistakes more often. And as someone using Claude Code heavily now, I would say we’re already at a point where AI wins.
- ttoinou 8mo agoNo but they will despise you for bringing the problem up
- Jweb_Guru 8mo agoIn the long run, good code makes everyone much happier than code that is bad because people are being "nice" and letting things slide in code review to avoid confrontation.
- darkwater 8mo ago> it is not true that when you ask those people to do otherwise they simply pretend to have done it and forget you asked later. I had a coworker that more or less exactly did that. You left a comment in a ticket about something extra to be done, he answered "yes sure" and after a few days proceeded to close the ticket without doing the thing you asked. Depending on the quantity of work you had at the moment, you might not notice that until after a few months, when the missing thing would bite you back in bitter revenge.
- Jweb_Guru 8mo agoYou may have had one. It clearly made a pretty negative impression on you because you are still complaining about them years later. I find it pretty misanthropic when people ascribe this kind of antisocial behavior to all of their coworkers.
- duskdozer 8mo agoMaybe, but it lets them pump out much, much more code than they otherwise would have been able to. That's the "100x" in their AI productivity multipliers.
- queenkjuul 8mo agoSounds like you just work at a shitty company
- lelanthran 8mo agoMaybe, but I'd bet a large sum of money that each of your coworkers aren't turning out this drivel at a rate of 3kLoC per hour. Can you imagine working with someone who produces 100k lines of unmaintainable code in a single sprint? This is your future.
- GeoAtreides 8mo agopeople also piss in rivers, yet dumping raw sewage by million m^3 in the same rivers is generally (less so in uk) frowned upon...
- 0x008 8mo agoThe problem is that you are looking at the code. /s
- vannevar 8mo agoI'd highly recommend working top down, getting it to outline a sane architecture before it starts coding. Then if one of the modules starts getting fouled up, start with a clean sheet context (for that module) incorporating any cautions or lessons learned from the bad experience. LLMs are not yet good at working and reworking the same code, for the reasons you outline. But they are pretty good at a "Groundhog Day" approach of going through the implementation process over and over until they get it right.
- coolius 8mo ago+1 if you are vibe coding projects from scratch. if the architecture you specify doesn't make sense, the llm will start struggling, the only way out of their misery is mocking tests. the good thing is that a complete rewrite with proper architecture and lessons learned is now totally affordable.
- disgruntledphd2 8mo agoI think the best thing about LLMs is how incredibly easy they make it to build one to throw away. I've definitely built the same thing a few times, getting incrementally better designs each time.
- bryanrasmussen 8mo agomaybe there should be an LLM trained on a corpus of a deletions and cleanup of code.
- krackers 8mo agoI'm guessing there's a very strong prior to "just keep generating more tokens" as opposed to deleting code that needs to be overcome. Maybe this is done already but since every git project comes with its own history, you could take a notable open-source project (like LLVM) and then do RL training against against each individual patch committed.
- movedx01 8mo agoPerhaps the problem is that you RL on one patch a time, failing to capture the overarching long term theme, an architecture change being introduced gradually over many months, that exists in the maintainer’s mental model but not really explicitly in diffs.
- bryanrasmussen 8mo agoright, it would have to a specialized tool that you used to do analysis of codebase every now and then, or parts that you thought should be cleaned up. Obviously there is a just keep generating more tokens bias in software management, since so many developer metrics over the years do various lines of code style analysis on things. But just as experience and managerial programs have over time developed to say this is a bad bias for ranking devs, it should be clear it is a bad bias for LLMs to have.
- ashdksnndck 8mo agoI think this is in the training data since they use commit data from repos, but I imagine code deletions are rarer than they should be in the real data as well.
- bryanrasmussen 8mo agodeleting and code cleanup is perhaps more an expression of seniority, and personal preferences. Maybe there should be the same kind style transfer with code that you see with graphical generative AI, "rewrite this code path in the style of Donald Knuth"
- unlikelytomato 8mo agoThis is why I'm confused when people say it isn't ready to replace most of the programmer workforce.
- lwansbrough 8mo agoFor me, I'll do the engineering work of designing a system, then give it the specific designs and constraints. I'll let it plan out the implementation, then I give it notes if it varies in ways I didn't expect. Once we agree on a solution, that's when I set it free. The frontier models usually do a pretty good job with this work flow at this point.
- m3kw9 8mo agoThat’s vibe coding and you won’t read more than 20% of the code written that way. You really can’t build complex software that way
- allajfjwbwkwja 8mo agoAt least he gets to enjoy doing the remaining 80% of the work in a lovely codebase with foundations written by an LLM.
- YesBox 8mo agoHeh, people like to have someone else to blame.
- danparsonson 8mo agoYeah that describes most legacy codebases I've worked on XD
- Foobar8568 8mo agoLLM code is higher quality than any codes I have seen in my 20 years in F500. So yeah you need to "guide" it, and ensure that it will not bypass all the security guidance for ex...But at least you are in control, although the cognitive load is much higher as well than just "blind trust of what is delivered". But I can see the carnage with offshoring+LLM, or "most employees", including so call software engineer + LLM.
- esafak 8mo agoI have run into this too. Some of it is because models lack the big picture; so called agentic search (aka grep) is myopic.
- marginalia_nu 8mo agoMy sense is that the code generation is fast, but then you always need to spend several hours making sure the implementation is appropriate, correct, well tested, based on correct assumptions, and doesn't introduce technical debt. You need to do this when coding manually as well, but the speed at which AI tools can output bad code means it's so much more important.
- LPisGood 8mo agoAnd it’s slower to review because you didn’t do the hard part of understanding the code as it was being written.
- Implicated 8mo agoYou're holding it wrong. Set the boundaries and guidelines before it starts working. Don't leave it space to do things you don't understand. ie: enforce conventions, set specific and measurable/verifiable goals, define skeletons of the resulting solutions if you want/can. To give an example. I do a lot of image similarity stuff and I wanted to test the Redis VectorSet stuff when it was still in beta and the PHP extension for redis (the fastest one, which is written in C and is a proper language extension not a runtime lib) didn't support the new commands. I cloned the repo, fired up claude code and pointed it to a local copy of the Redis VectorSet documentation I put in the directory root telling it I wanted it to update the extension to provide support for the new commands I would want/need to handle VectorSets. This was, idk, maybe a year ago. So not even Opus. It nailed it. But I chickened out about pushing that into a production environment, so I then told it to just write me a PHP run time client that mirrors the functionality of Predis (pure-php implementation of redis client) but does so via shell commands executed by php (lmao, I know). Define the boundaries, give it guard rails, use design patterns and examples (where possible) that can be used as reference.
- slopinthebag 8mo agoThey aren't holding it wrong, it's a fundamental limitation of not writing the code yourself. You can make it easier to understand later when you review it, but you still need to put in that effort.
- MattGaiser 8mo ago> If they implement something with a not-so-great approach, they'll keep adding workarounds or redundant code every time they run into limitations later. Are you using plan mode? I used to experience the do a poor approach and dig issue, but with planning that seems to have gone away?
- Implicated 8mo agoNot trying to be snarky, with all due respect... this is a skill issue. It's a tool. It's a wildly effective and capable tool. I don't know how or why I have such a wildly different experience than so many that describe their experiences in a similar manner... but... nearly every time I come to the same conclusion that the input determines the output. > If they implement something with a not-so-great approach, they'll keep adding workarounds or redundant code every time they run into limitations later. Yes, when the prompt/instructions are overly broad and there's no set of guardrails or guidelines that indicate how things should be done... this will happen. If you're not using planning mode, skill issue. You have to get all this stuff wrapped up and sorted before the implementation begins. If the implementation ends up being done in a "not-so-great" approach - that's on you. > If you tell them the code is slow Whew. Ok. You don't tell it the code is slow. Do you tell your coworker "Hey, your code is slow" and expect great results? You ask it to benchmark the code and then you ask it how it might be optimized. Then you discuss those options with it (this is where you do the part from the previous paragraph, where you direct the approach so it doesn't do "no-so-great approach") until you get to a point where you like the approach and the model has shown it understands what's going on. Then you accept the plan and let the model start work. At this point you should have essentially directed the approach and ensured that it's not doing anything stupid. It will then just execute, it'll stay within the parameters/bounds of the plan you established (unless you take it off the rails with a bunch of open ended feedback like telling it that it's buggy instead of being specific about bugs and how you expect them to be resolved). > you can have 10 bespoke tests for every bug. Plus a new mocking framework created every time the last one turns out to be unfit for purpose. This is an area I will agree that the models are wildly inept. Someone needs to study what it is about tests and testing environments and mocking things that just makes these things go off the rails. The solution to this is the same as the solution to the issue of it keeping digging or chasing it's tail in circles... Early in the prompt/conversation/message that sets the approach/intent/task you state your expectations for the final result. Define the output early, then describe/provide context/etc. The earlier in the prompt/conversation the "requirements" are set the more sticky they'll be. And this is exactly the same for the tests. Either write your own tests and have the models build the feature from the test or have the model build the tests first as part of the planned output and then fill in the functionality from the pre-defined test. Be very specific about how your testing system/environment is setup and any time you run into an issue testing related have the model make a note about that and the solution in a TESTING.md document. In your AGENTS.md or CLAUDE.md or whatever indicate that if the model is working with tests it should refer to the TESTING.md document for notes about the testing setup. Personally, I focus on the functionality, get things integrated and working to the point I'm ready to push it to a staging or production (yolo) environment and _then_ have the model analyze that working system/solution/feature/whatever and write tests. Generally my notes on the testing environment to the model are something along the lines of a paragraph describing the basic testing flow/process/framework in use and how I'd like things to work. The more you stick to convention the better off you'll be. And use planning mode.
- codebolt 8mo agoI use the restore checkpoint/fork conversation feature in GitHub Copilot heavily because of this. Most of the time it's better to just rewind than to salvage something that's gone off track.
- disgruntledphd2 8mo agoYeah I'm a big fan of branching for basically every change, as it provides a known good checkpoint.
- joquarky 8mo agoDon't let it deteriorate so far that it can't recover in one session. Perform regular sessions dedicated to cleaning up tech debt (including docs).
- leke 8mo agoi wonder if the solution is to just ask it to refactor its code once it's working.
- MadnessASAP 8mo agoYou can, and it might make things a bit better. The only real way I've found so far is to start going through file by file, picking it apart. I wouldn't be surprised if over half my prompts start with "Why ...?", usually followed by "Nope, ... instead” Maybe the occasional "Fuck that you idiot, throw the whole thing out"
- mirsadm 8mo agoI do this all the time but then you end up with really over engineered code that has way more issues than before. Then you're back to prompting to fix a bunch of issues. If you didn't write the initial code sometimes it's difficult to know the best way to refactor it. The answer people will say is to prompt it to give you ideas. Well then you're back to it generating more and more code and every time it does a refactor it introduces more issues. These issues aren't obvious though. They're really hard to spot.
- fmbb 8mo agoIt’s in the name, isn’t it? Generative AI.
- carlosjobim 8mo agoYes, this is exactly the experience I have had with LLMs as a non-programmer trying to make code. When it gets too deep into the weeds I have to ask it to get back a few steps.
- ThrowawayTestr 8mo agoI feel like there's two types of LLM users. Those that understand it's limitations, and those that ask it to solve a millennium problem on the first try.
- deleted 8mo ago[deleted]
- m3kw9 8mo agoYes that’s my observation too. I have to be double careful the longer they run a task. They like to hack and patch stuff even when I tell it I don’t prefer it.
- enraged_camel 8mo agoI have no idea what I'm doing differently because I haven't experienced this since Opus 4.5. Even with Sonnet 4.5, providing explicit instructions along the lines of "reuse code where sensible, then run static analysis tools at the end and delete unused code it flags" worked really well. I always watch Opus work, and it is pretty good with "add code, re-read the module, realize some pre-existing code (either it wrote, or was already there) is no longer needed and delete it", even without my explicit prompts.
- cyanydeez 8mo agoThe reason theyre not intelligent is becaise they want to predict the next token, so verbosity is baked in.
- bgitarts 8mo agohave you tired adding to your agents file: "Prefer solutions that reduce lines of code over adding lines of code"?
- karussell 8mo agoThe solution is to know when to use an existing solution like sqlite and when to create your own. So the biggest problem with LLMs is that they don't repel or remind you about possible consequences (too often). But if they would, I would find it even more awkward... and this is one of the reasons I prefer Claude Code over Codex.
- bluepoint 8mo agoActually I get improvements when I ask two llms to simplify each other’s work repeatedly.