9 ms·
As a fellow "staff engineer" LLMs are terrible at writing or teaching how to write idiomatic code, and they are actually causing me to spend more time reviewing
by toprerules 2y ago
As a fellow "staff engineer" LLMs are terrible at writing or teaching how to write idiomatic code, and they are actually causing me to spend more time reviewing than I was previously due to the influx of junior to senior engineers trying to sneak in LLM garbage.
In my opinion, using LLMs to write code comes as a faustian deal where you learn terrible practices and rely on code quantity, boilerplate, and indeterministic outputs - all hallmarks of poor software craftsmanship. Until ML can actually go end to end on requirements to product and they fire all of us, you can't cut corners on building intuition as a human by forgoing reading and writing code yourself.
I do think that there is a place for LLMs in generating ideas or exploring an untrusted knowledge base of information, but using code generated from an LLM is pure madness unless what you are building is truly going to be thrown away and rewritten from scratch, as is relying on it as a linting, debugging, or source of truth tool.
- doug_durham 2y agoI've had exactly the opposite experience with generating idiomatic code. I find that the models have a lot of information on the standard idioms of a particular language. If I'm having to write in a language I'm new in, I find it very useful to have the LLM do an idiomatic rewrite. I learn a lot and it helps me to get up to speed more quickly.
- deleted 2y ago[deleted]
- qqtt 2y agoI wonder if there is a big disconnect partially due to the fact that people are talking about different models. The top tier coding models (sonnet, o1, deepseek) are all pretty good, but it requires paid subscriptions to make use of them or 400GB of local memory to run deepseek. All the other distilled models and qwen coder and similar are a large step below the above models in terms of most benchmarks. If someone is running a small 20GB model locally, they will not have the same experience as those who run the top of the line models.
- Aeolun 2y agoThe top of the line models are really cheap though. Getting an anthropic key and $5 of credit costs you exactly that, and gives you hundreds of prompts.
- baq 2y agoI’ve iterated on 1k lines of react slop in 4h the other day, changed table components twice, handled errors, loading widgets, modals, you name it. It’d take me a couple days easily to get maybe 80% of that done. The result works ok, nobody cares if the code is good or bad. If it’s bad and there are bugs, doesn’t matter, no humans will look at it anymore - Claude will remix the slop until it works or a new model will rewrite the whole thing from scratch. Realized during writing this that I should’ve added the extract of requirements in the comment of the index.ts of the package, or maybe a README.CURSOR.md.
- hollowturtle 2y agoI'd pay to review one of your PRs. Maybe a consistent one with ai usage proof.
- baq 2y agoWould be great comedic relief for sure since I’m mostly working in the backend mines, where the LLM-friendly boilerplate is harder to come by admittedly. My defense is that Karpathy does the same thing, admitted himself in a tweet https://x.com/karpathy/status/1886192184808149383 https://x.com/karpathy/status/1886192184808149383 - I know exactly what he means by this.
- mrtesthah 2y agoMy experience having Claude 3.5 Sonnet or Google Gemini 2.0 Exp-12-06 rewrite a complex function is that it slowly introduces slippage of the original intention behind the code, and the more rewrites or refactoring, the more likely it is to do something other than what was originally intended. At the absolute minimum this should require including a highly detailed function specification in the prompt context and sending the output to a full unit test suite.
- n4r9 2y ago> Sometimes the LLMs can't fix a bug so I just work around it or ask for random changes until it goes away. Lordy. Is this where software development is going over the next few years?
- tokioyoyo 2y agoI will get probably heavily crucified for this, but to people who are ideologically opposing AI generated code — executives, directors and managerial staff think the opposite. Being very anti-LLM code instead of trying to understand how it can improve the speed might be detrimental for your career. Personally, I’m on the fence. But having conversations with others, and some requests from execs to implement different AI utils into our processes… making me to be on the safer side of job security, rather than dismiss it and be adamant against it.
- rectang 2y agoThis has been true for every heavily marketed development aid (beneficial or not) for as long as the industry has existed. Managing the politics and the expectations of non-technical management is part of career development.
- tokioyoyo 2y agoYeah, I totally agree, and you're 100% right. But the amount of integrations I've personally done and have instructed my team to do implies this one will be around for a while. At some point spending too much time on code that could be easily generated will be a negative point on your performance. I've heard exactly the same stories from my friends in larger tech companies as well. Every all hands there's a push for more AI integration, getting staff to use AI tools and etc., with the big expectation that development will get faster.
- rectang 2y agoI don't think AI is an exception. In organizations where there were top-down mandates for Agile, or OOP, or Test-Driven Development, or you-name-it, those who didn't take up the mandate with zeal were likely to find themselves out of favor.
- tokioyoyo 2y agoIt's not necessarily top down. I genuinely don't know a single person in my organization who doesn't use LLMs one way or another. Obviously with different degrees of applications, but literally everyone does. And we haven't had a real "EVERYONE MUST USE AI!", just people suggesting and asking for specific model usages, access to apps like Cursor and so on. (I know it because I'm in charge of maintaining all processes around LLM keys, their usages, Cursor stuff and etc.)
- jondwillis 2y agoIt isn't like you can't write tests or reason about the code, iterate on it manually, just because it is generated. You can also give examples of idioms or patterns you would like to follow. It isn't perfect, and I agree that writing code is the best way to build a mental model, but writing code doesn't guarantee intuition either. I have written spaghetti that I could not hope to explain many times, especially when exploring or working in a domain that I am unfamiliar with.
- ajmurmann 2y agoI described how I liked doing ping-pong pairing TDD with Cursor elsewhere. One of the benefits of that approach is that I write at least half the implementation and tests and review every single line. That means that there is always code that follows the patterns I want and it's right there for the LLM to see and base its work on. Edit: fix typo in last sentence
- scudsworth 2y agoi love when the llm can be its work of
- ajmurmann 2y agoUgh, sorry for the typo. That was supposed to be "can base its work on"
- axlee 2y agoWhat's your stack ? I have the complete opposite experience. LLMs are amazing at writing idiomatic code, less so at dealing with esoteric use cases. And very often, if the LLM produces a poopoo, asking it to fix it again works just well enough.
- Bjartr 2y ago> asking it to fix it again works just well enough. I've yet to encounter any LLM from chatGPT to cursor, that doesn't choke and start to repeat itself and say it changed code when it didn't, or get stuck changing something back and forth repeatedly inside of 10-20 minutes. Like just a handful of exchanges and it's worthless. Are people who make this workflow effective summarizing and creating a fresh prompt every 5 minutes or something?
- simonw 2y agoOne of the most important skills to develop when using LLMs is learning how to manage your context. If an LLM starts misbehaving or making repeated mistakes, start a fresh conversation and paste in just the working pieces that are needed to continue. I estimate a sizable portion of my successful LLM coding sessions included at least a few resets of this nature.
- dingnuts 2y agoall that fiddling and copy pasting takes me longer than just writing the code most of the time
- codr7 2y agoExactly, while not learning anything along the way.
- liamwire 2y agoOnly if you assume one is blindly copy/pasting without reading anything, or is already a domain expert. Otherwise you’ve absolutely got the ability to learn from the process, but it’s an active process you’ve got to engage with. Hell, ask questions along the way that interest you, as you would any other teacher. Just verify the important bits of course.
- deleted 2y ago[deleted]
- brandall10 2y agoIt's helpful to view working solutions and quality code as separate things to the LLM. * If you ask it to solve a problem and nothing more, chances are the code isn't the best as it will default to the most common solutions in the training data. * If you ask it to refactor some code idiomatically, it will apply most common idiomatic concepts found in the training data. * If you ask it to do both at the same time you're more likely to get higher quality but incorrect code. It's better to get a working solution first, then ask it to improve that solution, rinse/repeat in smallish chunks of 50-100 loc at a time. This is kinda why reasoning models are of some benefit, as they allow a certain amount of reflection to tie together disparate portions of the training data into more cohesive, higher quality responses.
- the_mitsuhiko 2y ago> but using code generated from an LLM is pure madness unless what you are building is truly going to be thrown away and rewritten from scratch, as is relying on it as a linting, debugging, or source of truth tool. That does not match my experience at all. You obviously have to use your brain to review it, but for a lot of problems LLMs produce close to perfect code in record time. It depends a lot on your prompting skills though.
- codr7 2y agoI would say prompting skills relative coding skills; and the more you rely on them, the less you learn.
- the_mitsuhiko 2y agoThat is not my experience. I wrote recently [1] about how I use it and it’s more like an intern, pair programmer or rubber duck. None of which make you worse. [1]: https://lucumr.pocoo.org/2025/1/30/how-i-ai/ https://lucumr.pocoo.org/2025/1/30/how-i-ai/
- lmm 2y ago> it’s more like an intern, pair programmer or rubber duck. None of which make you worse. Are you sure? I've definitely had cases where an inexperienced pair programmer made my code worse.
- the_mitsuhiko 2y agoThat’s a different question. But you don’t learn less.
- codr7 2y agoOf course you do, that's why school isn't just the teacher giving you the answers, you have to work for it.
- 2y ago
- icnexbe7 2y agoi’ve had some luck with asking conceptual questions about how something works if i am using library X with protocol Y. i usually get an answer that is either actually useful or at least gets me on the right path of what the answer should be. for code though, it will tell me to use non existent apis from that library to implement things
- arijo 2y agoLLMs can work if you program above the code. You still need to state your assertions with precision and keep a model of the code in your head. Its possible to be be precise at an higher level of abstraction as long as your prompts are consistent with a coherent model of the code.
- elliotto 2y ago> Its possible to be be precise at an higher level of abstraction as long as your prompts are consistent with a coherent model of the code. This is a fantastic quote and I will use this. I describe the future of coding as natural language coding (or maybe syntax agnostic coding). This does not mean that the llm is a magic machine that understands all my business logic. It means what you've described - I can describe my function flow in abstracted english rather than requiring adherence to a syntax
- beepbooptheory 2y agoSimply, it made my last job so nightmarish that for the first time in this career I absolutely dreaded even thinking about the codebase or having to work the next day. We can argue about the principle of it all day, or you can say things like "you are just doing it wrong," but ultimately there is just the boots-on-the-ground experience of it that is going to leave the biggest impression on me, at least. Like it's just so bad to have to work alongside, either the model itself or your coworker with the best of intentions but no domain knowledge. Its like having to forever be the most miserable detective in the world; no mystery, only clues. A method that never existed, three different types that express the same thing, the cheeky smile of your coworker who says he can turn the whole backend into using an ORM in a day because he has Cursor, the manager who signs off on this, the deranged PR the next day. This continual sense that less and less people even know whats going on anymore... "Can you make sure we support both Mongo and postgres?" "Can you put this React component inside this Angular app?" "Can you setup the kubernetes with docker compose?"
- esafak 2y agoHiring standards are important, as are managers who get it. Your organization seems to be lacking in both.
- the_real_cher 2y agoCode written from an LLM is really really good if done right, i.e. reviewing every line of code as it comes out and prompt guiding it in the right direction. If youre getting junior devs just pooping out code and sending to review thats really bad and should be a pip-able offense in my opinion.
- satellite2 2y agoI just don't fully understand this position at this level. Personally I know exactly what the next 5 lines need to be, and whether I write them or auto complete or some AI write them doesn't matter. I'll only accept what I had in mind exactly. And with Copilot for boilerplate and relatively trivial tasks that happens pretty often. I feel I'm just saving time / old age joint pain.
- purerandomness 2y agoIf the next 5 lines of code are so predictable, do they really need to be written down? If you're truly saving time by having an LLM write boiler plate code, is there maybe an opportunity to abstract things away so that higher-level concepts, or more expressive code could be used instead?
- jaredklewis 2y agoSure, but abstractions have a cost. 5 lines of code written with just the core language and standard library are often much easier to read and digest than a new abstraction or call to some library. And it’s just an unfortunate fact of life that many of the common programming languages are not terribly ergonomic; it’s not uncommon for even basic operations to require a few lines of boilerplate. That isn’t always bad as languages are balancing many different goals (expressiveness, performance, simplicity and so on).
- weitendorf 2y agoI have lately been writing a decent amount of Svelte. Svelte and frontend in general is relatively new to me, but since I’ve been programming for a while now I can usually articulate what I want to do in English. LLMs are totally a game changer for me in this scenario - they basically take me from someone who has to look everything up all the time to someone who only does so a couple times a day. In a way LLMs are ushering in a kind of boilerplate renaissance IMO. When you can have an LLM refactor a massive amount of boilerplate in one fell swoop it starts to not matter much if you repeat yourself - actually, really logically dense code would probably be harder for LLMs to understand and modify (not dissimilar from us…) so it’s even more of a liability now than in the past. I would almost always rather have simple, easy-to-understand code than something elegant and compact and “expressive” - and our tools increasingly favor this too. Also I really don’t give a shit about how to best center a div nor do I want to memorize a million different markup tags and their 25 years of baggage. I don’t find that kind of knowledge gratifying because it’s more trivia than anything insightful. I’m glad that with LLMs I can minimize the time I spend thinking about those things.
- whatever1 2y agoThe counterargument that I hear is that since writing code is now so easy and cheap, there is no need to write pretty code that generalizes well. Just have the llm write a crappy version and the necessary tests, and once your requirements change you just toss everything and start fresh.