5 ms·
Multiple things can be true at the same time: 1. LLMs do not increase general developer productivity by 10x across the board for general purpose tasks selected
by com2kid 1y ago
Multiple things can be true at the same time:
1. LLMs do not increase general developer productivity by 10x across the board for general purpose tasks selected at random.
2. LLMs dramatically increases productivity for a limited subset of tasks
3. LLMs can be automated to do busy work and although they may take longer in terms of clock time than a human, the work is effectively done in the background.
LLMs can get me up to speed on new APIs and libraries far faster than I can myself, a gigantic speedup. If I need to write a small bit of glue code in a language I do not know, LLMs not only save me time, but they make it so I don't have to learn something that I'll likely never use again.
Fixing up existing large code bases? Productivity is at best a wash.
Setting up a scaffolding for a new website? LLMs are amazing at it.
Writing mocks for classes? LLMs know the details of using mock libraries really well and can get it done far faster than I can, especially since writing complex mocks is something I do a couple times a year and completely forget how to do in-between the rare times I am doing it.
Navigating a new code base? LLMs are ~70% great at this. If you've ever opened up an over-engineered WTF project, just finding where HTTP routes are defined at can be a problem. "Yo, Claude, where are the route endpoints in this project defined at? Where do the dependency injected functions for auth live?"
Right tool, right job. Stop using a hammer on nails.
- deleted 1y ago[deleted]
- iLoveOncall 1y ago> Setting up a scaffolding for a new website? LLMs are amazing at it. So amazing that every single stat showed by the author in the article has been flat at best, despite all being based on new development rather than work on existing code-bases.
- daxfohl 1y agoMaybe the world has run out of interesting websites to create. That they are created faster doesn't necessarily imply they'll be created more frequently.
- daxfohl 1y agoOf course if that's the case (and it well may be), then THAT is the reason for tech layoffs. Not AI. If anything, it means AI came too late.
- coffeebeqn 1y agoAI still fails to extrapolate. It can interpolate between things it’s trained on but that’s not exactly a new interesting product. If it truly could extrapolate at human-ish levels we would actually maybe have 10x more games and websites and whatnot
- jfengel 1y agoIf it can figure out where dependencies come from I'm going to have to look more into this. I really hate the way injection makes other people's code bases impenetrable. "The framework scans billions of lines of code to find the implementation, and so can you!"
- com2kid 1y agoI'm not looking forward to the cancer of @ invading JavaScript code. Ugh. I am a big fan of wysiwyg. Plz don't Decorate my code....
- heavyset_go 1y ago> LLMs can get me up to speed on new APIs and libraries far faster than I can myself, a gigantic speedup. If I need to write a small bit of glue code in a language I do not know, LLMs not only save me time, but they make it so I don't have to learn something that I'll likely never use again. I wax and wane on this one. I've had the same feelings, but too often I've peaked behind the curtain, read the docs and got familiar with external dependencies and then realize whatever the LLM responds with paradoxically either wasn't following convention or tried to shoehorn your problem to fit code examples found online, used features inappropriately, took a long roundabout path to do something that can be done simply, etc. It can feel like magic until you look too closely at it, and I worry that it'll make me complacent with the feeling of understanding without actually taking away an understanding.
- SchemaLoad 1y agoYeah LLMs get me _an_ answer far faster than I could find it myself, but it's often not correct. And then I have to verify it myself which was exactly the work I was trying to skip by using the LLM to start with. If I have to manually verify every answer, I may as well read the docs myself.
- Onawa 1y agoIt doesn't completely solve this problem but definitely helps to have something like context7 MCP server running that `Copilot et al.` can reach dramatically reduces hallucinations for most tools. And additionally I've used Continue.dev VSCode along with manually specified docs and guides that you can selectively inject into your context. Both of those tactics make a huge difference in answer quality.
- emodendroket 1y agoIs it really that different from scrolling through Stack Overflow answers and rejecting the ones that aren't suitable? A lot of times you can tell it what specifically you didn't like about the solution and get another crack anyway (e.g., "let's iterate over the characters to do this rather than using a regex")
- ksenzee 1y ago> Stop using a hammer on nails. sorry, what am I supposed to use on nails?
- falcor84 1y agoNail polish remover
- dsign 1y agoCursor, is that you?
- svieira 1y agoThis glass bottle
- DonHopkins 1y agoPHP of course! https://blog.codinghorror.com/the-php-singularity/ https://blog.codinghorror.com/the-php-singularity/
- sethammons 1y agoI think it was a typo and should have been "Stop using a hammer on screws," suggesting tool / application mismatch.
- mvdtnz 1y ago> LLMs can be automated to do busy work and although they may take longer in terms of clock time than a human, the work is effectively done in the background. What is this supposed busy work that can be done in the background unsupervised? I think it's about time for the AI pushers to be absolutely clear about the actual specific tasks they are having success with. We're all getting a bit tired of the vagueness and hand waving.
- Kiro 1y agoNo, you've got it backwards. If anything, people are getting tired of comments like yours.
- dbalatero 1y agoNope!
- Kiro 1y agoThey are. There's no vagueness. It's rare to see people that still don't believe LLMs can do anything at all nowadays. Most other naysayers have moved on to the much more relevant question on whether the perceived productivity gains are real or not.
- dbalatero 1y ago> It's rare to see people that still don't believe LLMs can do anything at all nowadays. I don't think the original comment you responded to made this specific point.
- deleted 1y ago[deleted]
- iainctduncan 1y agoHN votes say otherwise, lol
- socalgal2 1y agoWould it more correct to change this > LLMs can get me up to speed on new APIs and libraries far faster than I can myself To this? > LLMs can get me up to speed on old APIs and old libraries that are new to me far faster than I can myself My experience is if the library/API/tool is new then the LLM can't help. But maybe I'm using it wrong.
- retreatguru 1y agoAn MCP server called Context7 excels at providing up to date api/library documentation for LLMs.
- alexchamberlain 1y ago> Setting up a scaffolding for a new website? LLMs are amazing at it. Weren't the code generators before this even better though? They generated consistent results and were dead quick at doing it.
- camdenreslink 1y agoAnd they were frequently in public repos that were updated with people filing issues if necessary.
- rhubarbtree 1y agoRecently I tried to scaffold a website with a well known coding agent. It didn’t work. I asked a colleague. He had the same problem. Turned out it was using out of date setup instructions for a major tool that has changed post training. After spending time fixing the problem, I realised (1) it would have been faster to do it myself and (2) I can no longer trust that tool to set anything up - what if it’s doing something else wrong?
- com2kid 1y agoYou'd run into the same issues if you followed a slightly outdated medium tutorial on the topic! React had this issue when they changed away from create react app. Basically every tutorial for years was instantly made obsolete, despite occupying all the top search results slots. Forums where filled with confusion for quite awhile. Now days most libraries seem to rely on "content producers" to document how to actually do things and don't bother with great docs filled with examples. In decades gone past, companies would hire actual authors to write real physical books on how to use new APIs. LLMs just have a higher latency on updating their reference material.
- rhubarbtree 1y ago> You'd run into the same issues if you followed a slightly outdated medium tutorial on the topic! You’re right, and in the past I have. So I know to always check the authorities sources for setup instructions.
- paool 1y agoSorry to say, but skill issue. Use MCP servers, specifically context 7. This gets up to date docs as long as you include the library name on your prompt and ask to use context 7. You did the equivalent of raw dogging gpt4(an old model) for recent news versus using an agent with web search tooling.
- margalabargala 1y ago"My shitty company's shitty app has an API for that! It's so good sales has decided to present it to the industry as though it should be incorrect not to use it"
- player1234 1y agoDoes not sound like a trillion dollar industry
- tjr 1y agoMore like a zillion dollar industry!
- 3uler 1y agoWorking with LLMs has fundamentally changed how I approach documentation and development. Traditional documentation has always been a challenge for me - figuring out where to start, what syntax conventions are being used, how pieces connect together. Good docs are notoriously hard to write, and even harder to navigate. But now, being able to query an LLM about specific tasks and get direct references to the relevant documentation sections has been a game-changer. This realization led me to flip my approach entirely. I’ve started heavily documenting my own development process in markdown files - not for humans, but specifically for LLMs to consume. The key insight is thinking of LLMs as amnesiac junior engineers: they’re capable, but they need to be taught what to do every single time. Success comes from getting the right context into them. Learning how to craft that context is becoming the critical skill. It’s not about prompting tricks - it’s about building systematic ways to feed LLMs the information they need. I’ve built up a library of commands and agents for my Claude Code installation inspired by AgentOS (https://github.com/buildermethods/agent-os https://github.com/buildermethods/agent-os) to help engineer the required context. The tool is a stochastic parrot, you need to feed it the right context to get the right answer. It is very good at what it does but you need to use it to its strengths in order to get value from it. I find people complaining about LLMs often expect vibe coding to be this magic tool that will build the app for you without thinking, which it unfortunately has been sold as, but the reality is more of a fancy prompt based IDE.
- thegrim33 1y ago"LLMs can get me up to speed on new APIs and libraries far faster than I can myself, a gigantic speedup" Just a random personal anecdote I wanted to throw out. I recently had to build some custom UI with Qt. I hadn't worked with Qt in a decade and barely remembered it. Seems like a perfect use case for AI to get me "up to speed" on the library, right? It's an incredibly well documented library with lots written on it, perfect fodder for an AI to process. So, I gave it a good description of the widget I was trying to make, what I needed it to look like and how it should be behave, and behold, it spit out the specific widget subclass I should use and how I should be overriding certain methods to customize behavior. Wow, it worked exactly like promised. So I implemented it like it suggested and was seemingly happy with the results. Went on with working on other parts of the project, dealing with Qt more and more here and there, gaining more and more experience with Qt over time. A month or two later, after gaining more experience, I looked back at what AI had told me was the right approach on that widget and realized it was completely messed up. It had me subclassing the completely wrong type of widget. I didn't need to override methods and write code to force it to behave the way I wanted. I could instead just make use of a completely different widget that literally supported everything I needed already. I could just call a couple methods on it to customize it. My new version removes 80% of the code that AI had me write, and is simpler, more idiomatic, and actually makes more sense now. So yeah, now any time I see people write about how "well, it's good for learning new libraries or new languages", I'll have that in the back of my mind. If you don't already know the library/language, you have zero idea whether what the AI teaching you is horrible or not. Whether there's a "right/better" way or not. You think it's helping you out when really you're likely just writing horrible code.
- rurp 1y agoJust recently I was having trouble getting something to work with a large well documented framework library. Turned out the solution I needed was to switch to a similar but different API. But that's not what Claude told me. Instead it wanted me to override and rewrite a bunch of core library code. Fortunately I was able to recognize that the suggested solution was almost certainly bad and did some more digging to find the right answer, but I could easily see nightmarish code that solves immediate problems in terrible ways piling up fast in a vibe coded project. I do find LLMs useful at times when working in unfamiliar areas, but there are a lot of pitfalls and newly created risks that come with it. I mostly work on large existing code bases and LLMs have very much been a mildly useful tool, still nice to have, but hardly the 100x productivity booster a lot of people are claiming.
- bluefirebrand 1y ago> Setting up a scaffolding for a new website? LLMs are amazing at it This is trivial work that you should have automated after doing it once or twice anyways :/