3 ms·
Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. [...] LLMs will always suck at writing code that has not be written milli
by simonw 1y ago
Learning how to use LLMs in a coding workflow is trivial. There is no learning curve. [...]
LLMs will always suck at writing code that has not be written millions of times before. As soon as you venture slightly offroad, they falter.
That right there is your learning curve! Getting LLMs to write code that's not heavily represented in their training data takes experience and skill and isn't obvious to learn.
- TheSamFischer 1y ago[dead]
- skydhash 1y agoIf you have a big rock (a software project), there's quite a difference between pushing it uphill (LLM usage) and hauling it up with a winch (traditional tooling and methods). People are claiming that it takes time to build the muscles and train the correct footing to push, while I'm here learning mechanical theory and drawing up levers. If one managed to push the rock for one meter, he comes clamoring, ignoring the many who was injured by doing so, saying that one day he will be able to pick the rock up and throw it at the moon.
- noidesto 1y agoThen there are those who are augmenting their winch with LLM usage.
- simonw 1y agoI'd describe LLM usage as the winch and LLM avoidance as insisting on pushing it up hill without one.
- bootsmann 1y agoSimon, I have mad respect for your work but I think on this your view might be skewed because your day to day work involves a codebase where a single developer can still hold the whole context in their head. I would argue that the inadequacies of LLMs become more evident the more you have to make changes to systems that evolve at the speed of 15+ concurrent developers.
- simonw 1y agoOne of the things I'm using LLMs for a lot right now is quickly generating answers about larger codebases I'm completely unfamiliar with. Anything up to 250,000 tokens I pipe into GPT-5 (prior to that o3), and beyond that I'll send them to Gemini 2.5 Pro. For even larger code than that I'll fire up Codex CLI or Claude Code and let them grep their way to an answer. This stuff has gotten good enough now that I no longer get stuck when new tools lack decent documentation - I'll pipe in just the source code (filtered for .go or .rs or .c files or whatever) and generate comprehensive documentation for myself from scratch.
- bootsmann 1y agoDon't you see how this opens up a blindspot in your view of the code? You don't have the luxury of having someone who is deeply familiar with the code sanity check your perceived understanding of the code, i.e. you don't see where the LLM is horribly off-track because you don't have sufficient understanding of that code to see the error. In enterprise contexts this is very common tho so its quite likely that a lot of the haters here have seen PRs submitted by vibecoders to their own work which have been inadequate enough that they started to blame the tool. For example I have seen someone reinvent the wheel of the session handling by a client library because they were unaware that the existing session came batteries included and the LLM didn't hesitate to write the code again for them. The code worked, everything checked out but because the developer didn't know what they didn't know about they submitted a janky mess.
- simonw 1y agoThe LLMs go off track all the time. I spot that when I try putting what I've learned from them into action.
- deleted 1y ago[deleted]
- FeepingCreature 1y agoThis just sounds 1:1 equivalent to "there are things LLMs are good for and things LLMs are bad for."
- donperignon 1y agoI’m still waiting that someone claiming how prompting is such an skill to learn, explain just once a single technique that is not obvious, like: storing checkpoint to go back to working version (already a good practice without using Llm see:git) or launch 10 tabs with slightly different prompts and choose the best, or ask the Llm to improve my prompt, or adding more context … is that an skill? I remember when I was a child that my mom thought that programming a vcr to record the night show to be such a feat…
- simonw 1y agoSee my comment here about designing environments for coding agents to operate in: https://news.ycombinator.com/item?id=44854680 https://news.ycombinator.com/item?id=44854680 Effective LLM usage these days is about a lot more than just the prompts.
- keeda 1y agoIn my experience, it's not just prompting that needs to be figured out, it's a whole new workstyle that works for you, your technologies and even your current project. As an example, I write almost all my code functional-programming style, which I rarely did before. This lets me keep my prompts and context very focused and it essentially elminates hallucinations. Also I started in the pre-agents era and so I ended up with a pair-programming paradigm. Now everytime I conceptualize a new task in my head -- whether it is a few lines of data wrangling within a function, or generating an entire feature complete with integration tests -- I instinctively do a quick prompt-vs-manual coding evaluation and seamlessly jump to AI code generation if the prompt "feels" more promising in terms of total time and probability of correctness. I think one of the skills is learning this kind of continuous evaluation and the judgement that goes with it.
- AndyNemmity 1y agoYou may not consider it a skill, but I train multiple programming agents on different production and quality code bases, and have all of them pr review a change, with a report given at the end. it helps dramatically on finding bugs and issues. perhaps that's trivial to you, but it feels novel as we've only had effective agents in the last couple weeks.