6 ms·
Let's talk about LLMs
- gizajob 6mo agoActually can we not thanks.
- stackghost 6mo agoLet's actually not talk about LLMs. I honestly couldn't force myself to finish yet another blog post about how "we're not yet sure what impact LLMs will have on society" or whatever beleaguered point the author was attempting to make. "Some random person's take on LLMs" was maybe interesting in 2024. Today it is not even remotely interesting. There are a gazillion more interesting things happening today that ought to be of interest to the median HN reader. Can we talk about those instead?
- mettamage 6mo agoI am an AI engineer and I honestly agree. Talking about LLMs feels like the new crypto, with some nuances (i.e. many innovative things being possible and done with LLMs whereas crypto innovations were… few and far between).
- dijksterhuis 6mo agoit’s felt like the new crypto to me for about 2-3 years now. i was doing an ML Sec phd a year or two before all this hype took off. i took one of the OG transformer papers along to present at our official little phd reading group when the paper was only a few months old (the details of this might be a bit sketchy here, was years ago now). now i want nothing to do with the field in any way shape or form. i’m just done. edit -- i got incredibly angry after writing this comment. pure hatred and spite for all the charlatans and accompanying bullshit.
- eiekeww 6mo agoSadly investing is all about making money… you should be more pissed at the naive people who have contributed to the effort and in particular those who don’t care about truth, but about cash flow potential.
- dijksterhuis 6mo agoeveryone involved is responsible, just to different degrees.
- keybored 6mo agoTedious LLM discourse isn’t aimed at AI engineers. It’s doomscrolling fodder for regular programmers.
- jubilanti 6mo agoI'm confused. If you don't want to talk about LLMs then why didn't you just flag the post and move on? Submit something interesting, upvote and comment on interesting posts, instead of feeding the engagement on this thread. It sounds like you actually do want to talk about how much you don't want other people to talk about LLMs.
- stackghost 6mo agoOh, I definitely flagged the post also.
- famouswaffles 6mo agoYou're not supposed to flag a post for something like that. Ideally you downvote and move on if you feel that strongly about it. Flagging is meant to be reserved for stuff that breaks the rules or guidelines.
- WolfeReader 6mo agoStories can't be downvoted.
- mfro 6mo agoI think you're misunderstanding the paradigm shift completely -- AI does not just generate code N(x) more quickly. It thinks N(x) faster, it researches N(x) faster, it tests N(x) faster. There are hundreds of tasks that you'll find engineers are offloading to AI every day. The major hurdle right now is actually pivoting LLMs from just generating code: integrating those tasks into workflows. This is why tool-use and agentic workflows have taken engineering by storm.
- brcmthrowaway 6mo agoTrue. Knowledge workers are cooked.
- pingou 6mo agoNot sure why you are downvoted but I agree. Additionally, perhaps LLMs are just like another higher programming language as the author said, and they still need someone to steer them. I'm sure it was very difficult to program in machine code, but if now (or soon) anyone can just write software using a LLM without any sort of learning it changes everything. LLMs can plan and create something usable from simple instructions or ideas, and they will only get better. I think LLMs will be (and already are) useful for many more things than programming anyway.
- fragmede 6mo ago> they will only get better. I don't buy that's true. The "only" part, anyway. Look at how UX with software has evolved. This is gonna be an old man yells at clouds take, but before smartphones, there were hotkeys. And man, you could fly with those things. The computers running things weren't as fast as they are today, but you could mash in a a whole sequence thru muscle memory, and just wait for it to complete. Now, you have to poke at your phone, wait for it to respond, poke at it some more. It's really not great for getting fast at it. AI advancement is going to be like that. Directionally generally it will be better, but there's going to be some niche where, y'know what, ChatGPT-4o really had it in a way that 5.5 does not. (Rose colored glasses not included.)
- 6mo ago
- smartmic 6mo agoIf you're interested in Fred Brooks's "No Silver Bullet," I also explored it in the context of LLMs: https://smartmic.bearblog.dev/no-ai-silver-bullet/ https://smartmic.bearblog.dev/no-ai-silver-bullet/
- js8 6mo agoIn fact, AI might be the opposite of managerial "silver bullet". The more we automate what is repetitive, the less predictability remains overall. Things can get more productive on average but the managing it becomes harder, as productivity amplifies risks.
- keybored 6mo agoI have no stake in Fred Brooks. But No Silver Bullet seemed to be taken as gospel on this board. Sufficiently productivity-enhancing technology? Gimme a break man. Maybe you’ll get a 30% boost. Not a 10X boost. Until recently. dramatic pause And then AI happened.
- taormina 6mo agoGreat! So all of this 10x boosting is visible in which economic indicator?
- slopinthebag 6mo agoDebt.
- AIorNot 6mo agothe problem with this article is that he is right of course, but only right now. There is no reason to believe that future AI platforms won't be able to review code themselves and manage some aspects of themselves with minimal human oversight, yes we likely will always need a few experts I'm reminded of this scene from the Matrix: https://www.youtube.com/watch?v=cD4nhYR-VRA https://www.youtube.com/watch?v=cD4nhYR-VRA where the older wise man discusses societies reliance on AI "Nobody cares how it works, as long as it works" We're done. I for one welcome our new AI Overlords, or more accurately still welcome the tech bro billionares who are pulling the strings
- frizlab 6mo ago> There is no reason to believe that future AI platforms won't be able to review code themselves and manage some aspects of themselves with minimal human oversight There are, IMHO, fewer reasons to believe they will be able to do that rather than not, though.
- CamperBob2 6mo agoLLMs became much better at both reviewing and writing code over the last 12-18 months. Did you? The current state of the art is irrelevant. Only the first couple of time derivatives matter.
- paulhebert 6mo ago> Did you? I would say I got better at both of those over the last 12-18 months. Are your skills static?
- eiekeww 6mo agoMy brain got better at thinking deeper when I stopped using llms. Lmao why does it seem outlandish to other people? Perhaps they never thought too deeply in the first place to recognise it.
- CamperBob2 6mo ago
- dabedee 6mo agoIt was a welcome change to have a deliberate, well thought, and well-written article that tries to bring readers through a rational journey. Thank you
- cadamsdotcom 6mo ago> If its two empirical premises—that the accidental/essential distinction is real and that the accidental difficulty remaining today does not represent 90%+ of total—are true, then the conclusion which rules out an order-of-magnitude gain from reducing accidental difficulty follows automatically. The article goes on to assume there’s no 10x gain to be had but misses one big truth. Needing to type the code is an enormous source of accidental difficulty (typing speed, typos, whether you can be arsed to put your hands on the keyboard today…) and it is gone thanks to coding agents.
- atleastoptimal 6mo ago"LLM's Aren't Going to Fundamentally Change Software Development" Says Increasingly Nervous Man For Seventh Time This Year
- slopinthebag 6mo agoI didn't get the sense that the author is nervous. What I tend to see are people who are nervous that going all-in on LLM workflows might not have the payoff they are expecting, and are becoming increasingly fanatical as a result. Just one more harness bro. Just one more agentic swarm. Please bro, just one more Claude Max subscription. Please bro.
- aspenmartin 6mo agoYou say this as though performance has not followed a very clear and extremely rapid improvement in a startlingly short amount of time. You’re definitely right that people adopt agentic workflows and are disappointed or worse, but the point is the disappointment has already reduced substantially and will continue to do so. We know this because we know the scaling laws, and also because learning theory has been around for many decades.
- slopinthebag 6mo agoYes but we don't know the shape of the curve and where we are on it.
- aspenmartin 6mo agoSee chinchilla scaling laws, we have the functional form of the curve and know the constants (though they change and are domain and model specific): L(N,D) ~= 1.69 + 406 / N^0.339 + 411 / D^0.285 L is loss (pre training test loss) D is the scale of the data N is the number of model parameters
- slopinthebag 6mo ago
- ilia-a 6mo agoEven without writing code LLMs are a huge help, analyzing code, doing code reviews, documenting code, etc... Even without writing a line of "code" LLM hugely speed up development and take away the annoying/boring work.
- nijave 6mo agoBeen using Claude Code for cost ops and reporting at work and it's saved an insane amount of time. I can generate a report in 10-15 minutes that would have taken 2-3 days of scripting/SQL and CC can even spit out a script to repro later. It's not terribly hard to check either. You can do some spot checks with cost dashboards in AWS, Datadog, etc and see if the numbers line up Can also tell Claude "go right size the environment, pull p95 usage metrics for the last 3 months" and a couple hours later, a bunch of money is saved. Much easier than manually pulling trend data and also easier than installing/configuring/managing tools that do it for you.
- riknos314 6mo agoIn pretty much every case where I've previously thought "I wish we had a tool for this but I can't get the time funded to make it" I now just get ai to work on the tool in the background and check in on it whenever I have a few minutes of deadtime before / after meetings. The benefits of the time savings of having progressily better tooling over time add up quickly.
- zapataband1 6mo agoI can read and write code why do I need an LLM to produce something I still have to read and test
- trwhite 6mo agoA well researched and written piece
- kelnos 6mo ago>> Within just this group the ratios between best and worst performances averaged about 10:1 on productivity measurements and an amazing 5:1 on program speed and space measurements! > (although I’m personally skeptical of the “10x programmer” concept, the software industry overall does seem to accept it as true) To be fair, this statement from Brooks doesn't entirely match with the "10x programmer" we talk about. My take on it is when someone says "10x programmer" today, they mean 10x more productive than the average, not 10x more productive than the worst. Brooks' statement is about the latter. If he'd looked at the difference between average and best, I would assume you'd get something more like a 2x or 4x programmer.
- leptons 6mo agoThere's no such thing as a "10x" programmer, and anyone who uses it doesn't know what they're talking about. 10x relative to what exactly? It's not a statement grounded in any kind of reality.
- hackable_sand 6mo agoIt's a transparent exercise in ego-stroking to justify one's commitment to capital incentives.
- jay_kyburz 6mo agoI've never understood it to be literal, but from my experience there is a big difference between the folks that show up to work on time, jump right into their work, they pay attention in meetings, know code base, and have the ability "lock in" as my kids say. On the other hand you have folks that show up late, spend all day chatting at the water cooler, get distracted with home stuff, comment on hacker news all day, and only manage to squeeze in a few hours of actual work a day.
- simonw 6mo ago10x makes sense only in terms of specific technology platforms. I'm a 10x programmer at building Django apps compared to a developer who has never worked with Django before. Someone who developers against WordPress on a daily basis will easily 10x my own attempts at building things on that platform.
- deleted 6mo ago[deleted]
- mwaddoups 6mo agoThis was a great read - thanks so much for taking the time to write this. Well researched and thought provoking. Long live the em dash.
- exographicskip 6mo agoI've been a fan of the em dash since college. Only recently learned what the shortcut on macos is (opt-shift-dash) but setup text expansion a decade ago. Think we need better AI tells than that
- slopinthebag 6mo agoI really enjoyed this article, it's well written and does a good job of dismantling the flawed arguments by language model maxis' while presenting a more realistic outlook on where we are now and where we are going. I think the biggest benefit language models have provided me is in the auxiliary aspects to programming: search, debugging, rubber ducking, planning, refactoring. The actual code generation has been mixed. I had an LLM try and implement a fairly involved feature the other day, providing it with API spec details, examples from other open source libraries, and plenty of specifications. It's also something readily available in training data as well, but still fairly involved. On first glance it looked great, and had I not spent the time to investigate deeper I would have missed some glaring deficiencies and omissions that render its implementation worthless. I am now going back and writing it by hand, but with language models providing assistance along the way, and it's going much better. I think people are being unrealistic by thinking that the usage of language models in their side projects represent something broader. It's almost the perfect situation for language models: small, greenfield code bases, no review, no responsibility, and no users. It goes up on GitHub with a pretty readme, and then off to social media where they post about how developers are "cooked". It's just not a very realistic test. In the end we will probably see large productivity increases by integrating language models, but they won't be replacing developers but rather augmenting them.
- deleted 6mo ago[deleted]
- senko 6mo agoThe accidental vs essential difficulty argument ignores the fact that you can abstract away (some) essential difficulty if you're willing to take a performance hit. Design patterns in an older (programming) language become core language features in a newer one. As we internalize and abstract away the best patterns for something, it becomes accidental but it's only obvious in retrospect. The article quotes Brooks (quoting Parnas) about just that (later, in context of LLMs): > automatic programming always has been a euphemism for programming with a higher-level language than was presently available to the programmer. [...] Once those accidents have been removed, the remaining ones are smaller, and the payoff from their removal will surely be less. Considering this was written when C was the hot new stuff, let's compare the ability to code a CRUD web app in Python/Django vs C. What Brooks and Parnas are saying that Python/Django cannot bring big improvements in building a CRUD web app when compared to C because they can only make it easier to program, reducing accidental complexity. But we've since redefined "accidental" and I would argue that you can write a CRUD web app in Python/Django at least 100x faster than in C (and probably at least 100x more secure), although it may take 1000x as more CPU and RAM while running. So "we removed most of the accidental difficulties and the most that remains is essential" is a kind of "end of history" argument. > I’d be surprised if there’s even a doubling of productivity still available from a complete elimination of remaining accidental difficulty. It's good that this statement has a conditional subjective guard, because that's just punditry. > LLM coding does not represent a silver bullet Here I agree with the author completely, but probably not for the same reasons. The definition of "silver bullet" the article uses (quoting Brooks): > There is no single development, in either technology or management technique, which by itself promises even a single order-of-magnitude improvement within a decade in productivity, in reliability, in simplicity. AI-assisted development is not a single technique, the same way "devops" or "testing" or "agile" is not a single technique. But more importantly, I agree it will take time to find best practices, for the technology change to slow down, and for the best approaches to diffuse across the industry. The article's conclusion: > You should be adopting and perfecting solid foundational software development practices like version control, comprehensive test suites, continuous integration, meaningful documentation, fast feedback cycles, iterative development, focus on users, small batches of work… things that have been known and proven for decades, but are still far too rare in actual real-world software shops. These are great and I'm gonna let him/her finish, but it's curious actual coding isn't mentioned anywhere. The author doesn't suggest "polish your understanding of C pointer semantics" or "Rust ownership model" or "Django ORM" or to really, deeply, understand B-trees. Looks like pedestrian detailes like those are left as an excercise for the reader ... or the reader's LLM.
- jwpapi 6mo agoIts the biggest swindle... You could fetch some unfinished github repos or download free templates. It’s actually faster than LLMs, still no body would do it. I don’t start my project with the ecommerce nextjs starter repo. I build it from scratch, because it’s faster...
- empthought 6mo ago> But ultimately, the only situation in which LLMs could meaningfully democratize access to software development is one where they achieve a true silver bullet, by significantly reducing or removing essential difficulty from the software development process. The author didn't seem to read the Brooks essay for comprehension. There is an entire section about expert systems that foreshadows agents. While there is no singular silver bullet, Brooks explores the most promising techniques to reduce essential complexity that were anticipated in 1986. > The most powerful contribution of expert systems will surely be to put at the service of the inexperienced programmer the experience and accumulated wisdom of the best programmers. This is no small contribution. Furthermore, his objection to automatic programming was simply an argument from incredulity, which is an understandable opinion at the time, yet quite vacuous in hindsight.
- rglover 6mo agoWe are obsessed with fortune telling. Use the damn thing or don't. It's that simple.
- zapataband1 6mo agoit would be that simple if a mental-health altering token-predictor wasn't being consistently shoved down our collective throats.
- rglover 6mo agoFair. But it's worth considering that anything "they" need to shove is a tell that they need you more than you need them. The sky-hath-fallen narrative is just what top-dollar marketing gets you these days. Clever, but mostly bark.
- marcus_holmes 6mo agoI was waiting for the "so I tried coding something with an LLM myself, and I found..." paragraph. But apparently the author never did try it, or at least if they did, they didn't write about it. This is a very academic approach to the subject - read what other people have written about it without ever doing it yourself. Study what someone said about LLM coding 50 years ago, before they were even invented, to see what you think about it. I would strongly suggest to the author that you just give it a go, and see what you think, without the preconception of other people's opinions. My experience has been remarkable, and, like others, I'm finding real joy in being able to move past the code to actually design and play with whole systems and architectures. It gets to the essence of code; which is not about the code, but about the system that code implements. Being able to write code in 3 minutes not 30 minutes does not bog us down in review (the LLM is perfectly capable of reviewing code too). It frees us to explore systems and architectures without worrying about the sunk cost of the existing code, or the effort of changing it.
- zouhair 6mo agoThe problem I have with it is the price (I am not talking about the money). I don't know if the price is worth it. For example we are literally witnessing the death of the personal computers, it will soon become a rich people's hobby. I don't know how the whole Free Software/Open source will survive that. At best we will end up not owning nothing, not even the programming skills as everyone will be at the mercy of AI companies for their coding. We are still in the honey moon phase of AI coding, I have a very pessimistic view of the future.
- marcus_holmes 6mo agoI'm not sure what LLMs have to do with the death of personal computers? Can you explain, please?
- zouhair 6mo agoPrices of RAM, GPUs, SSDs and even HDDs are now way out of reach for many people [0]. An SSD I bought 2 years ago at $300 CAD now cost $1K CAD for example and it's not gonna go down any time soon. [0]: https://www.tomshardware.com/pc-components/storage/perfect-storm-of-demand-and-supply-driving-up-storage-costs https://www.tomshardware.com/pc-components/storage/perfect-s...
- andai 6mo agoOn my device the article displayed 5 words per line, so I switched to "Desktop site" in the hamburger menu which made it much more readable. (Sorry for bikeshedding, but you can't discuss an article if you can't read it.)
- rambojohnson 6mo agothat's all we been doing now.
- furyofantares 6mo agoConsidering some project I'm able to accomplish today by hand, it is much larger than it was 25 years ago, but back then it would have mostly been inherent difficulty and now the work is mostly accidental difficulty. The libraries available have solved almost all of the inherent difficulty for many small (solo-dev) projects, and so for many small projects I already know every I need to know to do it (even though I don't know everything I would have needed to do it 25 years ago.) All that's left is doing it.
- deleted 6mo ago[deleted]
- cpharsh410 6mo ago[flagged]
- lacymorrow 6mo ago[dead]
- moritzwarhier 6mo agoNo, not again. Unless you had a point, which would probably apparent from the headline.