8 ms·

In other words, you have built exactly zero commercial-grade applications that us the working programmers work on building every day. LLMs are good for playing

In other words, you have built exactly zero commercial-grade applications that us the working programmers work on building every day. LLMs are good for playing with stuff, yes, and that has been implied by your parent commenter as well I think. But when you have to scale the work, then the code has to be easy to read, easy to extend, easy to test, have extensive test coverage, have proper dependency injection / be easy to mock the 3rd party dependencies, be easy to configure so it can be deployed in every cloud provider (i.e. by using env vars and config files for modifying its behavior)... and even more important traits. LLMs don't write code like that. Many people have tried with many prompts. It's mostly good for just generating stuff once and maybe do little modifications while convincingly arguing the code has no bugs (and it has). You seem to confuse one-off projects that have zero to little need for maintenance for actual commercial programming, perhaps? Your analogy with the automakers seems puzzlingly irrelevant to the discussion at hand, and very far from transferable to it. Also I am 100% convinced nobody is going to protect the programmers; business people trying to replace one guy like myself with 3 Indians has been a reality for 10 years at this point, and amusingly they keep failing and never learning their lesson. /off-topic Like your parent commenter, if LLMs get on my level, I'd be _happy_ to retire. I don't have a super vested interest in commercial programming, in fact I became as good at it in the last several years because I started hating it and wanted to get all my tasks done with minimal time expended; so I am quite confident in my assessment that LLMs are barely at the level of a diligent but not very good junior dev at the moment, and have been for the last at least 6 months. Your take is rather romantic.

Aeolun 2y ago
Very defensive. I love it anyhow. Sure, it generates shit code, but if you ask it it’ll gladly tell you all the ways it can be improved. And then actually do so. It’s not perfect. I spent a few hours yesterday pulling it’s massive blobby component apart by hand. But on the plus side, I didn’t have to write the whole thing. Just do a bunch of copy paste operations. I kinda like having a junior dev to do all the typing for me, and to give me feedback when I can’t think of a good way to proceed.
- olivermuty 2y ago
  If an LLM can give you feedback on a way to proceed it sounds more like you might be the junior? :P LLMs seems to be ok'ish at solving trivial boilerplate stuff. 20 attempts deep I have not yet seen it able to even remotely solve anything I have been stuck enough on to have to sit down and think hard.
  - Aeolun 2y ago
    > If an LLM can give you feedback on a way to proceed it sounds more like you might be the junior? :P More like it has no grey matter to get in the way of thinking of alternatives. It doesn’t get fixated, or at least, not on the same things as humans. It also doesn’t get tired, which is great when doing morning or late night work and you need a reality check. Deciding which option is the best one is still a human thing, though I find that those often align too.
  - whoistraitor 2y ago
    Curious: what kind of problem domain to you work on? I use LLMs every day on pretty hard problems and they are always net positive. But I imagine they’re not well trained on material related to your work if you don’t find them useful.
    - relaxing 2y ago
      What kind of problem domain do you work on?
- palata 2y ago
  > I spent a few hours yesterday pulling it’s massive blobby component apart by hand. But on the plus side, I didn’t have to write the whole thing. The question, really, is: are you confident that this was better than actually writing the whole thing yourself? Not only in terms of how long it took this one time, but also in terms of how much you learned while doing it. You accumulate experience when you write code, which is an investment. It makes you better for later. Now if the LLM makes you slightly faster in the short term, but prevents you from acquiring experience, then I would say that it's not a good deal, is it?
baq 2y ago
> LLMs are barely at the level of a diligent but not very good junior dev at the moment, and have been for the last at least 6 months. > Your take is rather romantic. I’m not sure you’re aware what you’ve written here. The contrast physically hurts.
- pdimitar 2y ago
  Apparently I'm not. Elaborate?
  - baq 2y ago
    Current generation LLMs have been in development for approximately as long as it takes for a college to ingest a high schooler and pump out a non-horrible junior software developer. The pace of progress slowed down, but if we get further 50% improvement in 200% time it’s still you who is being the romantic, not the op.
    - pdimitar 2y ago
      I honestly don't follow your angle at all. Mind telling me your complete takeaway? You are kind of throwing just bits and pieces at me and I followed too many sub-threads to keep full context. I don't see where I was romantic either.
      - felideon 2y ago
        Not the OP, but I imagine it has to do with the "LLMs will improve over time" trope. You said "and have been for the last at least 6 months" and it's confusing what you meant or what you expected should have happened in the past 6 months.
        pdimitar 2y ago
        I am simply qualifying my claims with the fact that they might not be super up to date is all. And my comments here mostly stem from annoyance that people claim that we already have this super game-changing AI that will remove programmers. And I still say: no, we don't have it. It works for some things. _Some_. Maybe 10% of the whole thing, if we are being generous.
GeoAtreides 2y ago
> But when you have to scale the work, then the code has to be easy to read, easy to extend, easy to test, have extensive test coverage, have proper dependency injection / be easy to mock the 3rd party dependencies, be easy to configure so it can be deployed in every cloud provider the code doesn't have to be anything like that, it only has to do one thing and one thing only: ship
- NorthTheRock 2y ago
  But in a world of subscription based services, it has to ship more than once. And as soon as that's a requirement, all of the above applies, and the LLM model breaks down.
- pdimitar 2y ago
  If and only if it's a one-off. I already addressed this in my comment that you are replying to, and you and several others happily ignored it. Really no idea why.
  - GeoAtreides 2y ago
    i'm sorry, both you and /u/NorthTheRock have a dev first mindset. Or a google-work-of-art codebase mindset. Or a bespoke little dev boutique mindset. Something like that. For the vast, vast majority of software devs it doesn't work that way. The way it works is: a PM says: we need this out ASAP, then we need this feature, this bug, hurry up, close your tickets, no, a clean codebase and unit tests are not needed, just get this out, the client is complaining. And so it goes. I'm happy you guys work in places where you can take your time to design beautiful work of arts, I really am. Again, that's not the experience for everyone else, who are toiling in the fields out there, chased by large packs of rabid tickets.
    - pdimitar 2y ago
      I am aware that the way I work is not the only one, of course, but I am also not so sure about your "vast, vast majority" thing. You can call it dev-first mindset, I call it a sustainable work mindset. I want the people I work for to be happy with my output. Not everything is velocity. I worked in high-velocity companies and ultimately got tired and left. It's not for me. And it's not about code being beautiful or other artsy snobby stuff like that. It's about it being maintainable really. And no I am not given much of a time, I simply refused to work in places where I am not given any time is all. I am a foot solider in the trenches just like you, I just don't participate in the suicidal charges. :)
nadam 2y ago
"But when you have to scale the work, then the code has to be easy to read, easy to extend, easy to test, have extensive test coverage, have proper dependency injection / be easy to mock the 3rd party dependencies, be easy to configure so it can be deployed in every cloud provider (i.e. by using env vars and config files for modifying its behavior).." Interestingly I do not find that the stuff you mentioned are the things that LLLMs are bad at. It can generate easy to read code. It can document its code extensively. It can write tests. It can use dependency injection especially if you ask it to. What I noticed where I am currently am much better than an LLM is that I can still have a very nuanced very complicated problem space in my head and create a solution based on that. The LLM currently cannot solve a problem which is so nuanced and complicated that even I cannot fully describe and work partially from insincts. It is the 'engineering taste' or 'engineering instincts' and our ability to absorb complicated nuanced design problems in our head that separates experienced developers from LLMs. Unless LLMs get significantly better and just replace humans in every task, I predict that LLMs effect on industry will be that much less developers will be needed but those who will be needed will be relatively 'well paid' as it will be harder to become a professional software developer. (more experience and engineering talent will be needed to work professionally).
- pdimitar 2y ago
  If you say so then OK, I am not going to claim you are imagining it. But some proof would be nice. I have quickly given up on experimenting with LLMs because they turned out to be net negative for my work -- proof-reading code is slower than writing it. But if you have good examples then I'll check them out. I am open to the possibility that LLMs are getting better at iterating on code. I'd welcome it. It's just that I haven't seen it yet and I am not going to go out of my way to re-vet several LLMs for the... 4th time it would be, I think.
w10-1 2y ago
I think you're talking about the differencing between coding and writing systems, which means you're talking teams, not individuals. Rarely, systems can be initiated by individuals, but the vast, vast majority are built and maintained by teams. Those teams get smaller with LLMs, and it might even lead to a kind of stasis, where there are no new leads with deep experience, so we maintain the old systems. That's actually fine by big iron, selling data, compute, resilience, and now intelligence. It's a way to avoid new entrants with cheaper programming models (cough J2EE). So, if you're really serious about dragging in "commercial-grade", it's only fair to incorporate the development and business context.
- pdimitar 2y ago
  I have not seen any proof so far that LLMs can enable teams. I and one other former colleague had to fix subtly broken LLM code several times, leading to the "author" being reprimanded that he's wasting three people's time. Obviously anecdata, sure, it's just that LLMs for now seem mostly suited for throwaway code / one-off projects. If there's systemic proof for the contrary I'd be happy to check it out.
dreamfactored 2y ago
> LLMs don't write code like that As someone who has been doing this since the mid 80's in all kinds of enterprise environments, I am finding that the latest generation are getting rather good at code like that, on par with mid-senior level in that way. They are also very capable of discussing architecture approaches with an encyclopaedic knowledge, although humans contribute meaningfully by drawing connections and analogies, and are needed to lead the conversation and make decisions. What LLM's are still weak at is holding a very large context for an extended period (which is why you can see failures in the areas you mentioned if not properly handled e.g. explicitly discussed, often as separate passes). Humans are better at compressing that information and retaining it over a period. LLM's are also more eager and less defensive coders. That means they need to be kept on a tight leash and drip fed single changes which get backed out each time they fail - so very bright junior in that way. For example, I'm sometimes finding that they are too eager to refactor as they go and spit out env vars to make it more production like, when the task in hand is to get basic and simple first pass working code for later refinement. I'm highly bullish on their capabilities as a force multiplier, but highly bearish on them becoming self-driving (for anything complex at least).
- pdimitar 2y ago
  > I'm highly bullish on their capabilities as a force multiplier, but highly bearish on them becoming self-driving (for anything complex at least). Very well summed and this is my exact stance, it's just that I am not seeing much of the "force multiplier" thing just yet. Happy to be proven wrong, but last time I checked (August 2024) I didn't get almost anything. Might be related to the fact that I don't do throwaway code, and I need to iterate on it.
  - dreamfactored 2y ago
    Sonnet 3.5 v2 was released in October. Most people just use this. First and only one that can do passable front end as well.
    - pdimitar 2y ago
edanm 2y ago
> business people trying to replace one guy like myself with 3 Indians has been a reality for 10 years at this point, and amusingly they keep failing and never learning their lesson. /off-topic What is the "lesson" that business people fail to learn? That Indians are worse developers than "someone like yourself"? (I don't mean to bump on this, but it is pretty offensive as currently written.)
- stefs 2y ago
  not OP, but not necessarily. the general reason is not that indians are worse developers per se. in my opinion it's more about the business structure. the "replacement indian" is usually not a coworker at the same company, but an employee at an outsourcing company. the outsourcing company's goal is not to ship a good product, but to make the most money from you. so while the "indian developer" might theoretically be able to deliver a better product than you, they wont be incentivized to do so. in practice, there are also many other factors that play into this - which might arguably play a bigger role like communication barriers, indian career paths (i.e. dev is only a stepstone on the way to manager), employee churn at cheap & terrible outsourcing companies, etc.
- pdimitar 2y ago
  1. The Indians were given as an example of absolutely terrible outsourcing agencies' dev employees. Call it racist or offensive if you like, to me it's statistical observation and I will offer no excuses that my brain is working properly and is seeing patterns. I have also met amazingly good Indian devs for what it's worth but most have been, yes, terrible. There's a link between very low-quality outsourcing agencies and Indian devs. I did not produce or fabricate this reality, it's just there. 2. The lesson business people fail to learn is that there's a minimum payment for programmers to get your stuff done, below which the quality drops sharply and that they should not attempt their cost-saving "strategy" because it ends up costing them much more than just paying me to do it. And I am _not_ commanding SV wages btw; $100k a year is something I only saw twice in my long career. So it's double funny how these "businessmen" are trying to be shrewd and pay even less, only to end up paying 5x my wage to a team that specializes in salvaging nearly-failed projects. I'll always find it amusing.
shenbomo 2y ago
Did you think computers and ARPANET came out for commercial-grade applications and not "good for playing with stuff" ?
- pdimitar 2y ago
  Of course they came from playing. But did you have people claiming we have a general AI while basic programming building blocks were still on the operating table?
  - shenbomo 2y ago
    Then what was dot com bubble?
rkachowski 2y ago
This is quite an absurd, racist and naive take. "Commercial grade applications" doesn't mean much in an industry where ~50% of projects fail. It's been said before that the average organisation cannot solve a solved problem. On top of this there's a lot of bizarre claims about what software _should_ be. All the dependency injection and TDD and scrum and all the kings horses don't mean anything when we are nothing more than a prompt away from a working reference implementation. Anyone designing a project to be "deployed in every cloud provider" has wasted a huge amount of time and money, and has likely never run ops for such an application. Even then, knowing the trivia and platform specific quirks required for each platform are exactly where LLMs shine. Your comments about "business people" trying to replace you with multiple Indian people shows your level of personal and professional development, and you're exactly the kind of personality that should be replaced by an LLM subscription.
- pdimitar 2y ago
  And yet, zero proof again. Getting so worked up doesn't seem objective so it's difficult to take your comment seriously.
throwawayffffas 2y ago
The main takeaway is that llms can't code to a professional level yet. But with improvement they probably will. It doesn't even have to be LLMs the coding part of our job will eventually be automated to a much larger degree than it is.
- palata 2y ago
  Or not. The point is that we don't know.
- pdimitar 2y ago
  Anything might happen in the future. My issue is with people claiming we're already in this future, and their only proof is "I generated this one-off NextJS application with LLMs". Cool for you but a lot of us actually iterate on our work.
lordnacho 2y ago
We're gonna get mini-milled. People with the attitude that real programmers are producing the high level product are going to get eaten slowly, from below, in the most embarrassing way possible. Embarrassing because they'll actually be right. LLMs aren't making the high quality products, it's true. But the low quality CRUD websites (think restaurant menus) will get swallowed by LLMs. You no longer need a guy to code up a model, run a migration, and throw it on AWS. You also don't need a guy to make the art. Advances in LLMs will feed on the money made from sweeping up all these crap jobs, which will legitimately vanish. That guy who can barely glue together a couple of pages, he is indeed finished. But the money will make better LLMs. They will swallow up the field from below.
- dukeyukey 2y ago
  > But the low quality CRUD websites (think restaurant menus) will get swallowed by LLMs. You no longer need a guy to code up a model, run a migration, and throw it on AWS. You also don't need a guy to make the art. To be fair those were eaten long ago by Wordpress and site builders.
  - Macha 2y ago
    And even Facebook. I think the last time bespoke restaurant menu websites were a viable business was around 2010.
    - actionfromafar 2y ago
      That doesn't agree with my area at all. Most restaurants have menus or an app or both. Some have a basic website with phone number and link to an app. There seems to be some kind of app template many restaurants use which you can order from too.
      - jhanschoo 2y ago
        > I think the last time bespoke restaurant menu websites > There seems to be some kind of app template many restaurants use which you can order from too. I think you agree with the comment you are replying to, but glossed over the word bespoke. IME as a customer in a HCOL area a lot of restaurants use a white-label platform for their digital menu. They don't have the need or expertise to maintain a functional bespoke solution, and the white-label platform lends familiarity and trust to both payment and navigation.
purple-leafy 2y ago
There are some very narrow minded, snarky people here, “doesn’t pass my sniff test” okay Einstein, you do you
- pdimitar 2y ago
  Surely, you actually mean those who extrapolate from a few niche tasks to "OMG the programmers are toast"? Fixed it for you. ;)
  - purple-leafy 2y ago
    Sorry I don’t mean to be snarky :) I think there is a happy middle ground between “AI can’t do anything, it sucks!” and “AI is AGI and can do everything and your career is done for” I teeter along the spectrum, and use with caution while learning new topics without expertise. But I’ve been very surprised by LLMs in some areas (UI design - something I struggle with - I’ve had awesome results!) My most impressive use case for an LLM so far (Claude 3.5 Sonnet) has been to iterate on a pseudo-3D ASCII renderer in the terminal using C and ncurses, where with the help of an LLM I was able to prototype this, and render an ascii “forest” of 32 highly detailed ascii trees (based off a single ascii tree template), with lighting and 3 axis tree placement, where you can move the “camera” towards and away from the forest. As you move closer trees scale up, and move towards you, and overlapping trees don’t “blend” into one ascii object - we came up with a clever boundary highlighting solution. Probably my favourite thing I’ve ever coded, will post to HN soon
    - pdimitar 2y ago
      I absolutely agree that it's a spectrum. I don't deny the usefulness of some LLMs (and I have used Claude 3.5 lately with great success -- it helped me iterate on some very annoying code with very niche library that I would have needed days to properly decipher myself). I got annoyed by the grandiose claims though so likely got a bit worked up. And indeed: > “AI is AGI and can do everything and your career is done for” ...this is the thing I want to stop seeing people even imply it's happening. An LLM helped you? Cool, it helped me as well. Stop claiming that programming is done for. It's VERY far from that.
tinodb 2y ago
> LLMs don't write code like that. Many people have tried with many prompts. It's mostly good for just generating stuff once and maybe do little modifications while convincingly arguing the code has no bugs (and it has). You have to make them write code like that. I TDD by telling the LLM what I want a test for, verify it is red, then ask it to implement. I ask it to write another test for an edge case that isn’t covered, and it will. Not for 100% of the code, but for production code for sure. And it certainly speeds me up. Especially in dreadful refactoring where I just need to flip from one data structure to another, where my IDE can’t help much.
peab 2y ago
"In other words, you have built exactly zero commercial-grade applications that us the working programmers work on building every day." The majority of programmers getting paid a good salary are implementing other people's ideas. You're getting paid to be some PMs chatGPT
- anhner 2y ago
  > You're getting paid to be some PMs chatGPT yes but one that actually works
- pdimitar 2y ago
  I never said we are working on new ideas only -- that's impossible. I even gave several examples of the traits that a commercial code must have and that LLMs fail to generate such code. Not sure why you ignored that.
  - PaulDavisThe1st 2y ago
    > LLMs fail to generate such code As another oldster, yes, yes absolutely. But the deeper question is: can this change? They can't do the job now, will they be able to in the future? My understanding of what LLM's do/how they work suggests a strong "no". I'm not confident about my confidence, however.
    - pdimitar 2y ago
      I am sure it can change. Ultimately programming we be completely lost; we are going to hand-wave and command computers inside us or inside the walls of our houses. This absolutely will happen. But my issue is with people that claim we are either at that point, or very nearly it. ...No, we are not. We are not even 10% there. I am certain LLMs will be a crucial part of such a general AI one day but it's quite funny how people mistake the brain's frontal lobe with the entire brain. :)
      - datadrivenangel 2y ago
        Plenty of jobs for coming by to read the logs across 6 systems when the LLM applications break and can't fix themselves.
richardw 2y ago
The analogy is that Japanese cars were initially utterly uncompetitive, and the arrogance of those who benefited from the status quo meant they were unable to adapt when change came. Humans improved those cars, humans are currently improving models and their use. Cheap toys move up the food chain. Find the direction of change and align yourself with it. > people trying to replace one guy like myself with 3 Indians has been a reality for 10 years at this point, and amusingly they keep failing and never learning their lesson. Good good, so you're the software equivalent of a 1990's era German auto engineer who currently has no equal, and maybe your career can finish quietly without any impact whatsoever. There are a lot of people on HN who are not you, and could use some real world advice on what to do as the world changes around them. If you've read "crossing the chasm", in at least that view of the world there are different phases to innovation, with different roles. Innovators, early adopters, early majority, late majority, laggards. Each has a different motivation and requirement for robustness. The laggards won't be caught dead using your new thing until IBM gives it to them running on an AS400. Your job might be equivalent to that, where your skills are rare enough that you'll be fine for a while. However, we're past the "innovators" phase at this point and a lot of startups are tearing business models apart right now, and they are moving along that innovation curve. They may not get to you, but everyone is not you. The choices for a developer include: "I'm safe", "I'm going to get so good that I'll be safe", "I'm going to leverage AI and be better", and "I'm out". Their decisions should not be based on your unique perspective, but on the changing landscape and how likely it is to affect them. Good sense-check on where things are in the startup universe: https://youtu.be/z0wt2pe_LZM https://youtu.be/z0wt2pe_LZM I can't find it now, but there's at least one company that is doing enterprise-scale refactoring with LLM's, AST's, rules etc. Obviously it won't handle all edge cases, but...that's not your grandad's Cursor. Might be this one, but don't recognise the name: https://www.linkedin.com/pulse/multi-repo-ai-assisted-refactoring-unlocks-large-scale-code-kwigc/ https://www.linkedin.com/pulse/multi-repo-ai-assisted-refact...
- pdimitar 2y ago
  You are assuming I am arrogant because I don't view the current LLMs as good coders. That's a faulty assumption so your argument starts with a logical mistake. Also I never said that I "have no equal". I am saying that the death of my career has been predicted for no less than 10 years now and it still has not happened, and I see no signs of it happening; LLMs produce terrible code very often. This gives me the right to be skeptical from where I am standing. And a bit snarky about it, too. I asked for a measurable proof, not for your annoyed accusations that I am arrogant. You are not making an argument. You are doing an ad hominem attack that weakens any other argument you may be making. Still, let's see some of them. --- RE: choices, my choice has been made long time ago and it's this one: "I will become quite good so as to be mostly safe. If 'AI' displaces me then I'll be happy to work something else until my retirement". Nothing more, nothing less. RE: "startup universe", that's a very skewed perspective. 99.99999% of all USA startups mean absolutely nothing in the grand scheme of things out there, they are but a tiny bubble in one country in a big planet. Trends change, sometimes drastically and quickly so. What a bunch of guys in comfy positions think about their bubble bears zero relevance to what's actually going on. > I can't find it now, but there's at least one company that is doing enterprise-scale refactoring with LLM's, AST's, rules etc. If you find it, let me know. That I would view as an interesting proof and a worthy discussion to have on it after.
nyarlathotep_ 2y ago
> LLMs are good for playing with stuff, yes, and that has been implied by your parent commenter as well I think. But when you have to scale the work, then the code has to be easy to read, easy to extend, easy to test, have extensive test coverage, have proper dependency injection / be easy to mock the 3rd party dependencies, be easy to configure so it can be deployed in every cloud provider (i.e. by using env vars and config files for modifying its behavior)... and even more important traits. I'm totally conflicted on the future of computing careers considering LLM impact, but I've worked at a few places and on more than a few projects where few/none of these are met, and I know I'm far from the only one. I'd wager a large portion of jobs are like this. Majority of roles aren't working on some well-groomed Google project. Most places aren't writing exquisite examples of a canonically "well-authored" codebase; they're gluing together enterprise CRUD slop to transform data and put it in some other database. LLMs are often quite good at that. It's impossible to ignore that reality.
- Aeolun 2y ago
  LLM’s are good at things a lot of people do, because a lot of people do them, and there’s tons of examples. It’s the very definition of a self-fulfilling prophecy.
  - UltraSane 2y ago
    Yes. LLMs are great at generating Python code but not so great at generating APL code.
  - fuzztester 2y ago
    Good to the power of one by infinity. Fixed that for you.
- rob74 2y ago
  If there is one job I don't want to do, it's being responsible for a big heap of enterprise CRUD slop generated and glued together by LLMs. If they can write it, they should learn to maintain it too!

Aeolun 2y ago

Very defensive. I love it anyhow. Sure, it generates shit code, but if you ask it it’ll gladly tell you all the ways it can be improved. And then actually do so. It’s not perfect. I spent a few hours yesterday pulling it’s massive blobby component apart by hand. But on the plus side, I didn’t have to write the whole thing. Just do a bunch of copy paste operations. I kinda like having a junior dev to do all the typing for me, and to give me feedback when I can’t think of a good way to proceed.

olivermuty 2y ago
If an LLM can give you feedback on a way to proceed it sounds more like you might be the junior? :P LLMs seems to be ok'ish at solving trivial boilerplate stuff. 20 attempts deep I have not yet seen it able to even remotely solve anything I have been stuck enough on to have to sit down and think hard.
- Aeolun 2y ago
  > If an LLM can give you feedback on a way to proceed it sounds more like you might be the junior? :P More like it has no grey matter to get in the way of thinking of alternatives. It doesn’t get fixated, or at least, not on the same things as humans. It also doesn’t get tired, which is great when doing morning or late night work and you need a reality check. Deciding which option is the best one is still a human thing, though I find that those often align too.
- whoistraitor 2y ago
  Curious: what kind of problem domain to you work on? I use LLMs every day on pretty hard problems and they are always net positive. But I imagine they’re not well trained on material related to your work if you don’t find them useful.
  - relaxing 2y ago
    What kind of problem domain do you work on?
palata 2y ago
> I spent a few hours yesterday pulling it’s massive blobby component apart by hand. But on the plus side, I didn’t have to write the whole thing. The question, really, is: are you confident that this was better than actually writing the whole thing yourself? Not only in terms of how long it took this one time, but also in terms of how much you learned while doing it. You accumulate experience when you write code, which is an investment. It makes you better for later. Now if the LLM makes you slightly faster in the short term, but prevents you from acquiring experience, then I would say that it's not a good deal, is it?

baq 2y ago

> LLMs are barely at the level of a diligent but not very good junior dev at the moment, and have been for the last at least 6 months. > Your take is rather romantic. I’m not sure you’re aware what you’ve written here. The contrast physically hurts.

pdimitar 2y ago
Apparently I'm not. Elaborate?
- baq 2y ago
  Current generation LLMs have been in development for approximately as long as it takes for a college to ingest a high schooler and pump out a non-horrible junior software developer. The pace of progress slowed down, but if we get further 50% improvement in 200% time it’s still you who is being the romantic, not the op.
  - pdimitar 2y ago
    I honestly don't follow your angle at all. Mind telling me your complete takeaway? You are kind of throwing just bits and pieces at me and I followed too many sub-threads to keep full context. I don't see where I was romantic either.
    - felideon 2y ago
      Not the OP, but I imagine it has to do with the "LLMs will improve over time" trope. You said "and have been for the last at least 6 months" and it's confusing what you meant or what you expected should have happened in the past 6 months.
      - pdimitar 2y ago
        I am simply qualifying my claims with the fact that they might not be super up to date is all. And my comments here mostly stem from annoyance that people claim that we already have this super game-changing AI that will remove programmers. And I still say: no, we don't have it. It works for some things. _Some_. Maybe 10% of the whole thing, if we are being generous.

GeoAtreides 2y ago

> But when you have to scale the work, then the code has to be easy to read, easy to extend, easy to test, have extensive test coverage, have proper dependency injection / be easy to mock the 3rd party dependencies, be easy to configure so it can be deployed in every cloud provider the code doesn't have to be anything like that, it only has to do one thing and one thing only: ship

NorthTheRock 2y ago
But in a world of subscription based services, it has to ship more than once. And as soon as that's a requirement, all of the above applies, and the LLM model breaks down.
pdimitar 2y ago
If and only if it's a one-off. I already addressed this in my comment that you are replying to, and you and several others happily ignored it. Really no idea why.
- GeoAtreides 2y ago
  i'm sorry, both you and /u/NorthTheRock have a dev first mindset. Or a google-work-of-art codebase mindset. Or a bespoke little dev boutique mindset. Something like that. For the vast, vast majority of software devs it doesn't work that way. The way it works is: a PM says: we need this out ASAP, then we need this feature, this bug, hurry up, close your tickets, no, a clean codebase and unit tests are not needed, just get this out, the client is complaining. And so it goes. I'm happy you guys work in places where you can take your time to design beautiful work of arts, I really am. Again, that's not the experience for everyone else, who are toiling in the fields out there, chased by large packs of rabid tickets.
  - pdimitar 2y ago
    I am aware that the way I work is not the only one, of course, but I am also not so sure about your "vast, vast majority" thing. You can call it dev-first mindset, I call it a sustainable work mindset. I want the people I work for to be happy with my output. Not everything is velocity. I worked in high-velocity companies and ultimately got tired and left. It's not for me. And it's not about code being beautiful or other artsy snobby stuff like that. It's about it being maintainable really. And no I am not given much of a time, I simply refused to work in places where I am not given any time is all. I am a foot solider in the trenches just like you, I just don't participate in the suicidal charges. :)

nadam 2y ago

"But when you have to scale the work, then the code has to be easy to read, easy to extend, easy to test, have extensive test coverage, have proper dependency injection / be easy to mock the 3rd party dependencies, be easy to configure so it can be deployed in every cloud provider (i.e. by using env vars and config files for modifying its behavior).." Interestingly I do not find that the stuff you mentioned are the things that LLLMs are bad at. It can generate easy to read code. It can document its code extensively. It can write tests. It can use dependency injection especially if you ask it to. What I noticed where I am currently am much better than an LLM is that I can still have a very nuanced very complicated problem space in my head and create a solution based on that. The LLM currently cannot solve a problem which is so nuanced and complicated that even I cannot fully describe and work partially from insincts. It is the 'engineering taste' or 'engineering instincts' and our ability to absorb complicated nuanced design problems in our head that separates experienced developers from LLMs. Unless LLMs get significantly better and just replace humans in every task, I predict that LLMs effect on industry will be that much less developers will be needed but those who will be needed will be relatively 'well paid' as it will be harder to become a professional software developer. (more experience and engineering talent will be needed to work professionally).

pdimitar 2y ago
If you say so then OK, I am not going to claim you are imagining it. But some proof would be nice. I have quickly given up on experimenting with LLMs because they turned out to be net negative for my work -- proof-reading code is slower than writing it. But if you have good examples then I'll check them out. I am open to the possibility that LLMs are getting better at iterating on code. I'd welcome it. It's just that I haven't seen it yet and I am not going to go out of my way to re-vet several LLMs for the... 4th time it would be, I think.

w10-1 2y ago

I think you're talking about the differencing between coding and writing systems, which means you're talking teams, not individuals. Rarely, systems can be initiated by individuals, but the vast, vast majority are built and maintained by teams. Those teams get smaller with LLMs, and it might even lead to a kind of stasis, where there are no new leads with deep experience, so we maintain the old systems. That's actually fine by big iron, selling data, compute, resilience, and now intelligence. It's a way to avoid new entrants with cheaper programming models (cough J2EE). So, if you're really serious about dragging in "commercial-grade", it's only fair to incorporate the development and business context.

pdimitar 2y ago
I have not seen any proof so far that LLMs can enable teams. I and one other former colleague had to fix subtly broken LLM code several times, leading to the "author" being reprimanded that he's wasting three people's time. Obviously anecdata, sure, it's just that LLMs for now seem mostly suited for throwaway code / one-off projects. If there's systemic proof for the contrary I'd be happy to check it out.

dreamfactored 2y ago

> LLMs don't write code like that As someone who has been doing this since the mid 80's in all kinds of enterprise environments, I am finding that the latest generation are getting rather good at code like that, on par with mid-senior level in that way. They are also very capable of discussing architecture approaches with an encyclopaedic knowledge, although humans contribute meaningfully by drawing connections and analogies, and are needed to lead the conversation and make decisions. What LLM's are still weak at is holding a very large context for an extended period (which is why you can see failures in the areas you mentioned if not properly handled e.g. explicitly discussed, often as separate passes). Humans are better at compressing that information and retaining it over a period. LLM's are also more eager and less defensive coders. That means they need to be kept on a tight leash and drip fed single changes which get backed out each time they fail - so very bright junior in that way. For example, I'm sometimes finding that they are too eager to refactor as they go and spit out env vars to make it more production like, when the task in hand is to get basic and simple first pass working code for later refinement. I'm highly bullish on their capabilities as a force multiplier, but highly bearish on them becoming self-driving (for anything complex at least).

pdimitar 2y ago
> I'm highly bullish on their capabilities as a force multiplier, but highly bearish on them becoming self-driving (for anything complex at least). Very well summed and this is my exact stance, it's just that I am not seeing much of the "force multiplier" thing just yet. Happy to be proven wrong, but last time I checked (August 2024) I didn't get almost anything. Might be related to the fact that I don't do throwaway code, and I need to iterate on it.
- dreamfactored 2y ago
  Sonnet 3.5 v2 was released in October. Most people just use this. First and only one that can do passable front end as well.
  - pdimitar 2y ago

edanm 2y ago

> business people trying to replace one guy like myself with 3 Indians has been a reality for 10 years at this point, and amusingly they keep failing and never learning their lesson. /off-topic What is the "lesson" that business people fail to learn? That Indians are worse developers than "someone like yourself"? (I don't mean to bump on this, but it is pretty offensive as currently written.)

stefs 2y ago
not OP, but not necessarily. the general reason is not that indians are worse developers per se. in my opinion it's more about the business structure. the "replacement indian" is usually not a coworker at the same company, but an employee at an outsourcing company. the outsourcing company's goal is not to ship a good product, but to make the most money from you. so while the "indian developer" might theoretically be able to deliver a better product than you, they wont be incentivized to do so. in practice, there are also many other factors that play into this - which might arguably play a bigger role like communication barriers, indian career paths (i.e. dev is only a stepstone on the way to manager), employee churn at cheap & terrible outsourcing companies, etc.
pdimitar 2y ago
1. The Indians were given as an example of absolutely terrible outsourcing agencies' dev employees. Call it racist or offensive if you like, to me it's statistical observation and I will offer no excuses that my brain is working properly and is seeing patterns. I have also met amazingly good Indian devs for what it's worth but most have been, yes, terrible. There's a link between very low-quality outsourcing agencies and Indian devs. I did not produce or fabricate this reality, it's just there. 2. The lesson business people fail to learn is that there's a minimum payment for programmers to get your stuff done, below which the quality drops sharply and that they should not attempt their cost-saving "strategy" because it ends up costing them much more than just paying me to do it. And I am _not_ commanding SV wages btw; $100k a year is something I only saw twice in my long career. So it's double funny how these "businessmen" are trying to be shrewd and pay even less, only to end up paying 5x my wage to a team that specializes in salvaging nearly-failed projects. I'll always find it amusing.

shenbomo 2y ago

Did you think computers and ARPANET came out for commercial-grade applications and not "good for playing with stuff" ?

pdimitar 2y ago
Of course they came from playing. But did you have people claiming we have a general AI while basic programming building blocks were still on the operating table?
- shenbomo 2y ago
  Then what was dot com bubble?

rkachowski 2y ago

This is quite an absurd, racist and naive take. "Commercial grade applications" doesn't mean much in an industry where ~50% of projects fail. It's been said before that the average organisation cannot solve a solved problem. On top of this there's a lot of bizarre claims about what software _should_ be. All the dependency injection and TDD and scrum and all the kings horses don't mean anything when we are nothing more than a prompt away from a working reference implementation. Anyone designing a project to be "deployed in every cloud provider" has wasted a huge amount of time and money, and has likely never run ops for such an application. Even then, knowing the trivia and platform specific quirks required for each platform are exactly where LLMs shine. Your comments about "business people" trying to replace you with multiple Indian people shows your level of personal and professional development, and you're exactly the kind of personality that should be replaced by an LLM subscription.

pdimitar 2y ago
And yet, zero proof again. Getting so worked up doesn't seem objective so it's difficult to take your comment seriously.

throwawayffffas 2y ago

The main takeaway is that llms can't code to a professional level yet. But with improvement they probably will. It doesn't even have to be LLMs the coding part of our job will eventually be automated to a much larger degree than it is.

palata 2y ago
Or not. The point is that we don't know.
pdimitar 2y ago
Anything might happen in the future. My issue is with people claiming we're already in this future, and their only proof is "I generated this one-off NextJS application with LLMs". Cool for you but a lot of us actually iterate on our work.

lordnacho 2y ago

We're gonna get mini-milled. People with the attitude that real programmers are producing the high level product are going to get eaten slowly, from below, in the most embarrassing way possible. Embarrassing because they'll actually be right. LLMs aren't making the high quality products, it's true. But the low quality CRUD websites (think restaurant menus) will get swallowed by LLMs. You no longer need a guy to code up a model, run a migration, and throw it on AWS. You also don't need a guy to make the art. Advances in LLMs will feed on the money made from sweeping up all these crap jobs, which will legitimately vanish. That guy who can barely glue together a couple of pages, he is indeed finished. But the money will make better LLMs. They will swallow up the field from below.

dukeyukey 2y ago
> But the low quality CRUD websites (think restaurant menus) will get swallowed by LLMs. You no longer need a guy to code up a model, run a migration, and throw it on AWS. You also don't need a guy to make the art. To be fair those were eaten long ago by Wordpress and site builders.
- Macha 2y ago
  And even Facebook. I think the last time bespoke restaurant menu websites were a viable business was around 2010.
  - actionfromafar 2y ago
    That doesn't agree with my area at all. Most restaurants have menus or an app or both. Some have a basic website with phone number and link to an app. There seems to be some kind of app template many restaurants use which you can order from too.
    - jhanschoo 2y ago
      > I think the last time bespoke restaurant menu websites > There seems to be some kind of app template many restaurants use which you can order from too. I think you agree with the comment you are replying to, but glossed over the word bespoke. IME as a customer in a HCOL area a lot of restaurants use a white-label platform for their digital menu. They don't have the need or expertise to maintain a functional bespoke solution, and the white-label platform lends familiarity and trust to both payment and navigation.

purple-leafy 2y ago

There are some very narrow minded, snarky people here, “doesn’t pass my sniff test” okay Einstein, you do you

pdimitar 2y ago
Surely, you actually mean those who extrapolate from a few niche tasks to "OMG the programmers are toast"? Fixed it for you. ;)
- purple-leafy 2y ago
  Sorry I don’t mean to be snarky :) I think there is a happy middle ground between “AI can’t do anything, it sucks!” and “AI is AGI and can do everything and your career is done for” I teeter along the spectrum, and use with caution while learning new topics without expertise. But I’ve been very surprised by LLMs in some areas (UI design - something I struggle with - I’ve had awesome results!) My most impressive use case for an LLM so far (Claude 3.5 Sonnet) has been to iterate on a pseudo-3D ASCII renderer in the terminal using C and ncurses, where with the help of an LLM I was able to prototype this, and render an ascii “forest” of 32 highly detailed ascii trees (based off a single ascii tree template), with lighting and 3 axis tree placement, where you can move the “camera” towards and away from the forest. As you move closer trees scale up, and move towards you, and overlapping trees don’t “blend” into one ascii object - we came up with a clever boundary highlighting solution. Probably my favourite thing I’ve ever coded, will post to HN soon
  - pdimitar 2y ago
    I absolutely agree that it's a spectrum. I don't deny the usefulness of some LLMs (and I have used Claude 3.5 lately with great success -- it helped me iterate on some very annoying code with very niche library that I would have needed days to properly decipher myself). I got annoyed by the grandiose claims though so likely got a bit worked up. And indeed: > “AI is AGI and can do everything and your career is done for” ...this is the thing I want to stop seeing people even imply it's happening. An LLM helped you? Cool, it helped me as well. Stop claiming that programming is done for. It's VERY far from that.

tinodb 2y ago

> LLMs don't write code like that. Many people have tried with many prompts. It's mostly good for just generating stuff once and maybe do little modifications while convincingly arguing the code has no bugs (and it has). You have to make them write code like that. I TDD by telling the LLM what I want a test for, verify it is red, then ask it to implement. I ask it to write another test for an edge case that isn’t covered, and it will. Not for 100% of the code, but for production code for sure. And it certainly speeds me up. Especially in dreadful refactoring where I just need to flip from one data structure to another, where my IDE can’t help much.

peab 2y ago

"In other words, you have built exactly zero commercial-grade applications that us the working programmers work on building every day." The majority of programmers getting paid a good salary are implementing other people's ideas. You're getting paid to be some PMs chatGPT

anhner 2y ago
> You're getting paid to be some PMs chatGPT yes but one that actually works
pdimitar 2y ago
I never said we are working on new ideas only -- that's impossible. I even gave several examples of the traits that a commercial code must have and that LLMs fail to generate such code. Not sure why you ignored that.
- PaulDavisThe1st 2y ago
  > LLMs fail to generate such code As another oldster, yes, yes absolutely. But the deeper question is: can this change? They can't do the job now, will they be able to in the future? My understanding of what LLM's do/how they work suggests a strong "no". I'm not confident about my confidence, however.
  - pdimitar 2y ago
    I am sure it can change. Ultimately programming we be completely lost; we are going to hand-wave and command computers inside us or inside the walls of our houses. This absolutely will happen. But my issue is with people that claim we are either at that point, or very nearly it. ...No, we are not. We are not even 10% there. I am certain LLMs will be a crucial part of such a general AI one day but it's quite funny how people mistake the brain's frontal lobe with the entire brain. :)
    - datadrivenangel 2y ago
      Plenty of jobs for coming by to read the logs across 6 systems when the LLM applications break and can't fix themselves.

richardw 2y ago

The analogy is that Japanese cars were initially utterly uncompetitive, and the arrogance of those who benefited from the status quo meant they were unable to adapt when change came. Humans improved those cars, humans are currently improving models and their use. Cheap toys move up the food chain. Find the direction of change and align yourself with it. > people trying to replace one guy like myself with 3 Indians has been a reality for 10 years at this point, and amusingly they keep failing and never learning their lesson. Good good, so you're the software equivalent of a 1990's era German auto engineer who currently has no equal, and maybe your career can finish quietly without any impact whatsoever. There are a lot of people on HN who are not you, and could use some real world advice on what to do as the world changes around them. If you've read "crossing the chasm", in at least that view of the world there are different phases to innovation, with different roles. Innovators, early adopters, early majority, late majority, laggards. Each has a different motivation and requirement for robustness. The laggards won't be caught dead using your new thing until IBM gives it to them running on an AS400. Your job might be equivalent to that, where your skills are rare enough that you'll be fine for a while. However, we're past the "innovators" phase at this point and a lot of startups are tearing business models apart right now, and they are moving along that innovation curve. They may not get to you, but everyone is not you. The choices for a developer include: "I'm safe", "I'm going to get so good that I'll be safe", "I'm going to leverage AI and be better", and "I'm out". Their decisions should not be based on your unique perspective, but on the changing landscape and how likely it is to affect them. Good sense-check on where things are in the startup universe: https://youtu.be/z0wt2pe_LZM https://youtu.be/z0wt2pe_LZM I can't find it now, but there's at least one company that is doing enterprise-scale refactoring with LLM's, AST's, rules etc. Obviously it won't handle all edge cases, but...that's not your grandad's Cursor. Might be this one, but don't recognise the name: https://www.linkedin.com/pulse/multi-repo-ai-assisted-refactoring-unlocks-large-scale-code-kwigc/ https://www.linkedin.com/pulse/multi-repo-ai-assisted-refact...

pdimitar 2y ago
You are assuming I am arrogant because I don't view the current LLMs as good coders. That's a faulty assumption so your argument starts with a logical mistake. Also I never said that I "have no equal". I am saying that the death of my career has been predicted for no less than 10 years now and it still has not happened, and I see no signs of it happening; LLMs produce terrible code very often. This gives me the right to be skeptical from where I am standing. And a bit snarky about it, too. I asked for a measurable proof, not for your annoyed accusations that I am arrogant. You are not making an argument. You are doing an ad hominem attack that weakens any other argument you may be making. Still, let's see some of them. --- RE: choices, my choice has been made long time ago and it's this one: "I will become quite good so as to be mostly safe. If 'AI' displaces me then I'll be happy to work something else until my retirement". Nothing more, nothing less. RE: "startup universe", that's a very skewed perspective. 99.99999% of all USA startups mean absolutely nothing in the grand scheme of things out there, they are but a tiny bubble in one country in a big planet. Trends change, sometimes drastically and quickly so. What a bunch of guys in comfy positions think about their bubble bears zero relevance to what's actually going on. > I can't find it now, but there's at least one company that is doing enterprise-scale refactoring with LLM's, AST's, rules etc. If you find it, let me know. That I would view as an interesting proof and a worthy discussion to have on it after.

nyarlathotep_ 2y ago

> LLMs are good for playing with stuff, yes, and that has been implied by your parent commenter as well I think. But when you have to scale the work, then the code has to be easy to read, easy to extend, easy to test, have extensive test coverage, have proper dependency injection / be easy to mock the 3rd party dependencies, be easy to configure so it can be deployed in every cloud provider (i.e. by using env vars and config files for modifying its behavior)... and even more important traits. I'm totally conflicted on the future of computing careers considering LLM impact, but I've worked at a few places and on more than a few projects where few/none of these are met, and I know I'm far from the only one. I'd wager a large portion of jobs are like this. Majority of roles aren't working on some well-groomed Google project. Most places aren't writing exquisite examples of a canonically "well-authored" codebase; they're gluing together enterprise CRUD slop to transform data and put it in some other database. LLMs are often quite good at that. It's impossible to ignore that reality.

Aeolun 2y ago
LLM’s are good at things a lot of people do, because a lot of people do them, and there’s tons of examples. It’s the very definition of a self-fulfilling prophecy.
- UltraSane 2y ago
  Yes. LLMs are great at generating Python code but not so great at generating APL code.
- fuzztester 2y ago
  Good to the power of one by infinity. Fixed that for you.
rob74 2y ago
If there is one job I don't want to do, it's being responsible for a big heap of enterprise CRUD slop generated and glued together by LLMs. If they can write it, they should learn to maintain it too!