10 ms·

GPTs and Feeling Left Behind

• deleted 1y ago
  [deleted]
• betaminecraft 1y ago
  [dead]
• PaulHoule 1y ago
  I think what people are missing is that they work sometimes and sometimes they don't work. People think "Oh, it works better when somebody else does it" or "There must be some model that does better than the one I am using" or "If I knew how to prompt better I'd get better results" or "There must be some other agentic IDE which is better than the one I am using." All those things might be true but they just change the odds, they don't change the fact that it works sometimes and fails other times. For instance I asked an agent to write me a screen to display some well-typed data. It came up with something great right away that was missing some fields and had some inconsistent formatting but it fixed all those problems when I mentioned them -- all speaking the language of product managers and end users. The code quality was just great, as good as if I wrote it, maybe better. Plenty of times it doesn't work out like that. I was working on some code where I didn't really understand the typescript types and fed it the crazy error messages I was getting and it made a try to understand them and didn't really, I used it as a "rubber duck" over the course of a day or two and working with it I eventually came to understand what was wrong and how to fix and I got into a place that I like and when there is an error I can understand it and it can understand it too. Sometimes it writes something that doesn't typecheck and I tell it to run tsc and fix the errors and sometimes it does a job I am proud of and other times it adds lame typeguards like if (x && typeof x === "object") x.someMethod() Give it essentially the same problem, say writing tests in Java, and it might take very different approaches. One time it will use the same dependency injection framework used in other tests to inject mocks into private fields, other times it will write some a helper method to inject the mocks into private fields with introspection directly. You might be able to somewhat tame this randomness with better techniques but sometimes it works and sometimes it doesn't and if I just told you about the good times or just told you about the bad times it would be a very different story.
  • leptons 1y ago
    >I was working on some code where I didn't really understand the typescript types and fed it the crazy error messages I was getting and it made a try to understand them and didn't really, I used it as a "rubber duck" over the course of a day or two and working with it I eventually came to understand what was wrong and how to fix and I got into a place that I like and when there is an error I can understand it and it can understand it too. I have to wonder if you tried a simple google search and read through some docs if you couldn't have figured this out quicker than trying to coax a result out of the LLM.
    • solarkraft 1y ago
      My (not GP) intuitive answer would be hell no. Typescript messages are pretty hard to google and even parse manually and the LLM suggesting multiple approaches and ways to think about the problem does seem useful. It sometimes uncovers unknown unknowns you might never find otherwise. I have had cases in which a web search and some good old fashioned thinking have yielded better results than using an LLM, but on average I’m pretty sure the LLM has the edge.
      • PaulHoule 1y ago
        Personally I think Stack Overflow and Google are 95% trash for that kind of problem. The answers are in (i) the Typescript documentation and (ii) the documentation of libraries that I'm using. I could get lucky with a Google search and it could be worth trying, but I wouldn't expect it to work. Personally my preference is to have a language and libraries with great documentation (Python, Java, Typescript isn't too bad [1]) and really know that documentation like the back of my hand. If I hadn't had the LLM I would have probably figured it out the same way doing experiments, I might have asked my other "rubber duck" https://mastodon.social/@UP8/113935901671533690 https://mastodon.social/@UP8/113935901671533690 A tactic I didn't use, which helps in "legacy" systems where I am stuck, is start a fresh project in the IDE and try either reproducing the problem or building a tiny system which is problem free. I'm hesitant to say what speedup I got out of the "figuring out the types together with the LLM" but emotionally I felt supported and in the process I wrote a whole lot, like I was keeping track of the process in a notebook. I feel a lot times when I have good LLM conversations I wind up writing better code than I would otherwise, not necessarily write it faster -- it's like pair programming. [1] The typescript docs are great for the typescript stuff, MDN is good for Javascript and Javascript's stdlib
• calrain 1y ago
  Well, it takes a while to learn Vim and then get value from it. It also takes a while to learn using an LLM and get value from it. The keys are how to build prompts, ways of working, and guidelines that help the AI stay focused. You end up spending much more time guiding and coaching rather than coding, that can take a while to get used to. Eventually though, you will master it and be able to write secure, fast code far beyond what you could have done by yourself. Note: Also, prep yourself for incoming hate every time you make claims like that! If you write bad code, it's your fault. If your LLM writes bad code, you're a moron! hah
  • throwawa14223 1y ago
    So you're taking an easy task, formal logic, and replacing it with a more difficult and time consuming task, babysitting a random number generator. How is that a net-positive?
    • calrain 1y ago
      I get your position, and I don't want to sound dismissive, but when you really learn how to manage an LLM for a complex piece of software far beyond what you have time for, you see the benefits. Try
      • hckrfucrs 1y ago
        I have lost a few friends to llm sycophancy induced psychosis wherein they believe, and are encouraged to believe by the llm that, they are the sole individuals who have cracked "prompting", and, in fact, by so doing, have summoned the singularity for their sole benefit and under their sole control. You sound exactly like them.
        • thrown-0825 1y ago
          You sound like the child of a psychotherapist.
          • 1y ago
• dumbmrblah 1y ago
  I really wish posts like this included the parameters that they were using. What model? What was the question? How many shots? Etc etc You’re going to get vastly different responses if you’re using Opus versus 4o.
  • siscia 1y ago
    Do you really? Frontier models seems remarkably similar in performance. Yeah some nuances for sure, but the whole article could apply to every model.
    • arthur-st 1y ago
      4o on ChatGPT.com vs. Opus in an IDE is like cooking food without kitchen tools vs. using them. 4o is neither a coding-optimized model nor a reasoning model in general.
    • dnh44 1y ago
      You're not pushing them hard enough if you're not seeing a vast difference between 4o and Opus. Or possibly they're equivalent in the field you're working in but I suspect it's the former.
    • gronglo 1y ago
      Opus, in my opinion, is steps away from AGI. 4o doesn't come close.
  • 1over137 1y ago
    I’d like to know which programming language.
    • dnh44 1y ago
      I'm of the same opinion as the op and I'm programming in Rust(api) and Swift(client) right now. I entered a vibe coded game hackathon a few months back and in a little over a week I was at 25k lines of code across both the client and server. It all worked (multiplayer) even though the game sucked.
• storus 1y ago
  The worst thing is when LLMs introduce subtle bugs into code and one just can't spot them quickly. I was recently doing some Langfuse integration and used Cursor to generate skeleton code for pushing some traces/scores quickly. The generated code included one parameter "score_id" that was undocumented in Langfuse but somehow was accepted and messed the whole tracking up. Even after multiple passes of debugging I couldn't figure out what the issue with tracking was, until I asked another LLM to find any possible issues with the code, that promptly marked those score_id lines.
  • lvl155 1y ago
    This is a very important lesson because the way these coding models are built. You have to understand HOW they are designed from the base LLMs. And more importantly why it’s crucial to use two distinctly different models to review each other at every turn.
• siscia 1y ago
  I find myself on both sides actually. I did have some great luck producing quite useful and impactful code. But also lost time chasing tiny changes.
  • PessimalDecimal 1y ago
    Overall, has it been a net savings for you?
• jackdawed 1y ago
  One blogpost I found on HN completely leveled up how I use LLMs for coding: https://harper.blog/2025/02/16/my-llm-codegen-workflow-atm/ https://harper.blog/2025/02/16/my-llm-codegen-workflow-atm/ Having the AI ask me questions and think about the PRD/spec ultimately made me a better system designer.
  • 8organicbits 1y ago
    > This is working well NOW, it will probably not work in 2 weeks, or it will work twice as well. ¯\_(ツ)_/¯ This all feels like spinning the roulette wheel. I sometimes wonder if AI proponents are just gamblers who had the unfortunate luck of winning the first few prompts.
    • dbalatero 1y ago
      I've seen comparisons to gambling before (activating reward centers, sometimes it pays out big, etc), but couldn't find the article when I searched.
      • PessimalDecimal 1y ago
        A comparison I've seen isn't to roulette but to a slot machine. Anthropic itself encourages its employees to treat its use for refactors as a slot machine. [1] It seems like an idea worth exploring formally but I haven't see that done anywhere. Is this a case of "perception of winning" while one is actually losing? Or it it that the winning is in aggregate and people who like LLM-based coding are just more tolerant of the volatility to get there? The only study I've seen testing the actual observable impact on velocity showed a modest decrease in output for experienced engineers who were using LLMs for coding. [1] https://www-cdn.anthropic.com/58284b19e702b49db9302d5b6f135ad8871e7658.pdf https://www-cdn.anthropic.com/58284b19e702b49db9302d5b6f135a...
      • globular-toast 1y ago
        That really resonates. I've found myself questioning whether I'm wasting my time writing a piece of code: what if the LLM could do this more quickly? So I try it, almost every time, and sometimes it does, sometimes it doesn't. Am I really saving myself any work in the long run? Honestly I don't know. I feel like it's just causing me to work more because it feels like a game and that is, ultimately, where the results are coming from.
• avalys 1y ago
  I have a degree in CS from MIT and did professional software engineering from 2004 - 2020. I recently started a company in another field and haven’t done any real development for about 4 years. Earlier this summer I took a vacation and decided to start a small software hobby project specific to my industry. I decided to try out Cursor for the first time. I found it incredibly helpful at saving time implementing all the bullshit involved in starting a new code base - setting up a build system, looking up libraries and APIs, implementing a framework for configuration and I/O, etc. Yes, I still had to do some of the hard parts myself, and (probably most relevant) I still had to understand the code it was writing and correct it when it went down the wrong direction. I literally just told Cursor “No, why do it that way when you could do it much simpler by X”, and usually it fixed it. A few times, after writing a bunch of code myself, I compiled the project for the first time in a while and (as one does) ran into a forest of inscrutable C++ template errors. Rather than spend my time scrolling through all of them I just told cursor “fix the compile errors”, and sure enough, it did it. Another example - you can tell it things like “implement comparison operators for this class”, and it’s done in 5 seconds. As the project got more complicated, I found it super useful to write tests for behaviors I wanted, and just tell it “make this test pass”. It really does a decent job of understanding the codebase and adding onto it like a junior developer would. Using an IDE that gives it access to your whole codebase (including build system and tests) is key. Using ChatGPT standalone and pasting stuff in is not where the value is. It’s nowhere near able to do the entire project from scratch, but it saved me from a bunch of tedious work that I don’t enjoy anyway. Seems valuable enough to me!
  • dnh44 1y ago
    Last summer I came back to software after about 12 years away and I pretty much had an identical experience to you with using AI as a helper to come back. I've now spent the last 6 months coding as much as I can in between consulting gigs. I'm not sure if I would have been able to get caught up so quickly without AI. I haven't had this much fun programming since I was at university hacking away on sun workstations, but admittedly I only write about 10% of the code myself these days. I'm currently getting Claude Code to pair program with GPT-5 and they delegate the file edits to Gemini Flash. It's pretty cool.
• CityOfThrowaway 1y ago
  I have a feeling this person is using far-from-frontier models, totally disconnected from the development environment. Using, like, gpt-4o is extremely not useful for programming. But using Claude Code in your actual repo is insanely useful. Gotta use the right tool + model.
  • tyfighter 1y ago
    How is anyone just supposed to know that? It's not hard to find vim, but no one says, "You need to be running this extra special vim development branch where people are pushing vim to the limits!" Yes, it's fragmented, and changing fast, but it's not reasonable to expect people just wanting a tool to be following the cutting edge.
    • dbalatero 1y ago
      I agree with your comment, but I also chuckled a bit, because Neovim _is_ a fast changing ecosystem with plugins coming out to replace previous plugins all the time, and tons of config tweakers pushing things to the limit. That said… one does not have to replace their working Neovim setup just because new stuff came out. (And of course, minimalist vim users don't use any plugins!)
    • CityOfThrowaway 1y ago
      I agree that it might not be reasonable to expect people to keep up with the latest. For this specific thing (LLM-assisted coding), we are still in nerd territory where there are tremendous gains to be had from keeping up and tinkering. There's a lot of billions dollars being invested to give devs who don't want to do this the right tools. We aren't quite there yet, largely because the frontier is moving so fast. I made my original comment because it was so far from my experience, and I assumed it was because I am using a totally different set of tools. If somebody really doesn't want to be left behind, the solution is to do the unreasonable: read hacker news everyday and tinker. Personally, I enjoy that labor. But it's certainly not for everybody.
      • dnh44 1y ago
• neom 1y ago
  I think back to the first version of ChatGPT and I would pick it up once in a while, ask it something or chat with it, and then be like... this is cool but I don't know wtf I would use it for, now I use a GPT at least a couple of times a day. Granted, the LLMs have obviously become considerably more capable, but I do believe part of it is I've also learned how to use them and what to use them for, I'm at the point now where I can generally predict the output of what I'm asking for - I don't know if that's the norm (it mostly gives me exactly what I want) I do know how I use them today and how I used them when they first came out is quite different. I guess all that is to say, imo how you prompt them really matters, and that takes time to learn.
• Workaccount2 1y ago
  At what point does clinging to driving a manual transmission round the track go from a sign of practical skill to a sign of stubborn arrogance?
  • dustfinger 1y ago
    We are straying a bit off-topic with this, but driving a standard can actually be a lot of fun. I have an automatic right now, but I have always enjoyed driving a standard more than automatics. This next point may be due to the quality of my particular automatic, but I have found passing vehicles on the highway to be a lot faster in a standard, because I can choose to just drop into a lower gear and hit the gas, I don't have to hit the gas and then wait for the vehicle to decide that it should go into a lower gear. So, it doesn't have to be a practical skill or a sign of arrogance, someone might just enjoy driving a standard as I do.
• SoftTalker 1y ago
  Haven’t even really tried them. The sand is shifting way too fast. Once things stabilize and other people figure out how to really use them I’ll probably start but for now it just feels like effort that will have been wasted.
  • neom 1y ago
    All the models feel a bit different to use, and part of being good with LLMs (I suspect) is being able to assess a model before you really start using it, and, learning the nuances in the models that you will use, for that alone I think it's worth spending time with them.
    • moritzwarhier 1y ago
      "All the screwdrivers feel a bit different to use, and part of being good with screwdrivers (I suspect) is being able to assess a screwdriver before you really start using it, and, learning the nuances in the screwdrivers that you will use, for that alone I think it's worth spending time with them" Sounds dubious to me "All the SSRIs feel a bit different, and part of being good with SSRIs (I suspect) is being able to assess an SSRI before you really start using it, and, learning the nuances in the SSRIs that you will use, for that alone I think it's worth spending time with them" Hm, that comparison sounds off, but not as much to me as to many other people. "All the IDEs and text editors feel a bit different to use, and part of being good with IDEs (I suspect) is being able to assess an IDE before you really start using it, and, learning the nuances in the editors that you will use, for that alone I think it's worth spending time with them" Sounds reasonable. Substituting the subject back to AI coding agents, I'm struggling to make out your argument. What method of assessment would you recommend other than _starting to use_ a coding assistant model? I guess you were referring to anecdats and reviews/posts, or were you referring to specific objective properties like context size, RAG capabilities etc?
  • Groxx 1y ago
    yeah, tbh I think that even if they are the cat's pajamas and they end up taking over absolutely all text-based work everywhere and literally everyone agrees they're better at it than humans... ... the current state-of-the-art won't be what we use, and the prompts people are spending tons of time crafting now will be useless. so I don't think there's all that much FOMO to F over. either the hype bubble pops or literally everyone in those trades will be starting over with brand new skills based on whatever was developed in the past 6 months. people who rode the wave will have something like 6 months of advantage... ... and their advantage will quickly be put into GPTs and new users won't need to learn that either ("you are a seasoned GPT user writing a prompt..."). unless you worry endlessly about Roko's Basilisk, it's kinda ignorable I think. either way you still need to develop non-GPT skills to be able to judge the output, so you might as well focus on that.
• allenu 1y ago
  I kept hearing about Claude Code for a while and never really tried it until a week ago. I used it to prototype some Mac app ideas and I quickly realized how useful it was at getting prototypes up and running very, very quickly, like within minutes. It saves so much time with boilerplate code that I would've had to type out by hand and have done hundreds of times before. With my experience, I wonder what the author of this blog post has tried to do to complete a task as that might make a difference on why they couldn't get much use out of it. Maybe other posters can chime in on how big of a difference programming language and size of project can make. I did find that it was able to glean how I had architected an app and it was able to give feedback on potential refactors, although I didn't ask it to go that far. Prior to trying out Claude Code, I had only used ChatGPT and DeepSeek to post general questions on how to use APIs and frameworks and asking for short snippets of code like functions to do text parsing with regexes, so to be honest I was very surprised at what the state of the art could actually do, at least for my projects.
• martinald 1y ago
  I'm completely equally lost the other way. I've went through multiple phases of LLM usage for development. GPT3.5 era: wow this is amazing, oh. everything is hallucinated. not actually as useful as I first thought GPT4 era: very helpful as stackoverflow on steroids. Claude 3.5 Sonnet: have it open pretty much all the time, constantly asking questions and getting it to generate simple code (in the web UI) when it goes down actually feels very old school googling stuff. Tried a lot of in IDE AI "chat" stuff but hugely underwhelmed. Now: rarely open IDE as I can do (nearly) absolutely everything in Claude Code. I do have to refactor stuff every so often "manually", but this is more for my sanity and understanding of the codebase.. To give an example of a task I got Claude code to do today in a few minutes which would take me hours. Had a janky looking old admin panel in bootstrap styles that I wanted to make look nice. Told Claude code to fetch the marketing site for the project. Got it to pull CSS, logos, fonts from there using curl and apply similar styling to the admin panel project. Within 10 mins it was looking far, far better than I would have ever got it looking (at least without a designers help). Then got it to go through the entire project (dozens of screens) and update "explanation" copy - most of which was TODO placeholders to explain what everything did properly. I then got it to add an e2e test suite to the core flows. This took less than an hour while I was watching TV. I would have almost certainly _never_ got around to this before. I'd been meaning to do all this and I always sigh when I go into this panel at how clunky it all is and hard to explain to people.
  • kaashif 1y ago
    Yeah, as a primarily backend engineer dealing with either weird technical problems Claude can't get quite right or esoteric business domain problems Claude has no idea about (and indeed, it may be only a few people in one company could help with) - Claude isn't that useful. But random stuff like make a web app that automates this thing or make an admin panel with auto complete on these fields and caching data pulled from this table. It is like infinity times faster on this tedious boilerplate because some of this stuff I'd just have never done before. Or I'd have needed to get some headcount in some web dev team to do it, but I just don't need to. Not that I'd have ever actually bothered to do that anyway...
    • 1y ago
• Ezhik 1y ago
  In the end, the greatest use I get from coding agents and stuff is hijacking the Stack Overflow principle - it's much easier to trick myself into correcting the poor code Claude generates than it is to start writing code from a blank slate.
• tyfighter 1y ago
  You're (they're?) not alone. This mirrors every experience I've had trying to give them a chance. I worry that I'm just speaking another language at this point. EDIT: Just to add context seeing other comments, I almost exclusively work in C++ on GPU drivers.
  • almostgotcaught 1y ago
    Same - I work on a cpp GPU compiler. All the LLMs are worthless. Ironically the compiler I work on is used heavily for LLM workloads.
  • bobsmooth 1y ago
    I can't imagine there is a lot of GPU driver code in the training data.
  • nxobject 1y ago
    There's the market out there for a consultancy that will fine-tune an LLM for your unique platform, stack, and coding considerations of choice – especially with proprietary platforms. (IBM's probably doing it right now for their legacy mainframe systems.) No doubt Apple is trying to figure out how to get whatever frameworks they have cooking up ASAP into OpenAI etc.'s models.
  • thrown-0825 1y ago
    it really only works for problem domains saturated with medium blogspam and youtube tutorials.
    • notjoemama 1y ago
      That's a bingo! Christoph Waltz is just a great actor. I'm building an app in my stack with fairly common requirements. There are a few code examples that cover requirements but none that cover our specific scenario. After searching the web myself, I asked 3 different AI models. All they did was regurgitate the closest public GitHub example, lacking the use case I was trying to do. Solving this problem can only be done by understanding the abstraction of the alteration in design. These things can't actually think. And now they're allowed to be agentic. In some ways they're just glorified search engines but there's a geopolitical sprint to see who can get them to mock "thinking" enough to fool everybody. Out of ego and greed, everything will be turned over to this machine, and that will be the end of humanity; not humans...humanity.
• dmezzetti 1y ago
  A lot of this is classic gaslighting. Gaslighting is the manipulation of someone into questioning their perception of reality (per Wikipedia). Basically, a lot of people who are experts are being told this story and they think they are the only one who doesn't get it. There are plenty of gains to be had with AI/LLMs but just not in the way it's typically marketed.
• throwawa14223 1y ago
  This exactly mirrors my experience. I can't see the whole LLM/GPT thing as anything but another blockchain level scam. It isn't zero value it is actually a negative value as the time it takes is an opportunity cost.
  • bigstrat2003 1y ago
    I wouldn't say "scam", but otherwise I agree. I have found LLMs to often provide negative value, as in they slow me down.
• lezojeda 1y ago
  [dead]
• Barrin92 1y ago
  I spend a fair amount of time on open source and one thing I noticed is that in real pieces of software it doesn't look like all these 10x and 100x AI engineers are anywhere to be found. VLC has like 4000 open issues. Why aren't the AI geniuses fixing these? Nobody has ever any actual code to show, and if they do it's "here's an LED that blinks every time my dog farts, I could've never done it on my own!". I'm feeling like Charlie in that episode of It's Always Sunny with his conspiracy dashboard. All these productivity gurus don't actually exist in the real world. Can anybody show me their coding agent workflow on a 50k LOC C codebase instead of throwaway gimmick examples? As far as I'm concerned these things can't even understand pointers
  • chrisco255 1y ago
    This is something I've been thinking about too. I think lower skill devs seem to feel the most ecstatic about LLMs as are able to prompt something into existence that is beyond their skill level, and that feels like magic. It very quickly becomes a crutch, however, and a junior developer may struggle to become a senior in skill level with too much dependency on Cursor and tools like it. They definitely lose their edge on complex products. I find it can be a great time saver for language reference and for rapid fire questions, but it isn't able to, as of yet, operate at an expert level on a complex novel codebase. Maybe it gets there in another year or two. But it currently makes enough mistakes at that level that it isn't worth it.
  • 000ooo000 1y ago
    You only have to look at the C# GitHub repos where MS enabled Copilot, and the utter ridiculousness of those interactions between the likes of Stephen Toub and the bot. It's comical.
• zmmmmm 1y ago
  One thing to openly recognise is that FOMO is one of the core marketing strategies applied in any hype bubble to get people on board. There seem to be multiple blog posts a day on HN that are thinly veiled marketing about AI and most follow a predictable pattern: (a) start by implying a common baseline that is deliberately just beyond where your target market sits (example: "how I optimised my Claude workflow") and (b) describe the solution to the problem just well enough to hint there's an answer but not well enough to allow people to generalise. By doing this you strongly hint that people should just buy into whatever the author is selling rather than try to build fundamental knowledge themselves. Putting aside the FOMO, the essential time tested strategy is simply to not care and follow what interests you. And the progress in AI is simply astonishing, it's inherently interesting, this shouldnt be hard. Don't go into with it with the expectation of "Unless it vibe coded and entire working application for me on it's a failure". Play with it. Poke it, prod it. Then try to resolve the quirks and problems that pop up. Why did it do that? Don't expect an outcome. Just let it happen. The people who do this now will be the ones to come through the hype bubble at the end with actual practical understanding and deployable skills.
• cwyers 1y ago
  The author calling them "GPTs" suggests to me maybe not keeping up with the state of the art.
  • PessimalDecimal 1y ago
    Why? Are the currently available offerings not generative pretrained transformers?
• jtfrench 1y ago
  I think the issue is ChatGPT. I find it to be the worst for writing shippable code beyond a single function. It plays too many mind games. If I’m trying to get any non-trivial amount of code done, it’s Gemini Pro 2.5 all the way.
• 3vidence 1y ago
  In a non-smug kind of way sometimes I just wonder if they types of problems I work on are just harder (at least for an LLM) than a lot of people. Currently working at a FAANG on some very new tech, have access to all the latest and greatest but LLMs / agents really do not seem adequate working on absolutely massive codebases on entirely new platforms. Maybe I will have to wait a few years for the stuff I'm working on to enter the mass market so the LLMs can be retrained on it. I do find them very very useful as advanced search / stack overflow assistants.
• lsy 1y ago
  I think the wide variance in responses here is explainable by tool preference and the circumstance of what you want to work on. You might also have felt "behind" not knowing or wanting to use Dreamweaver, or React, or Ruby on Rails, or Visual Studio + .NET, all tools that allowed developers at the time to accelerate their tasks greatly. But you'll note that probably most programmers today who are successful never learned those tools, so the fact that they accelerated certain tasks didn't result in a massive gap between users and non-users. People shouldn't worry about getting "left behind" because influencers and bloggers are overindexing on specific tech rather than more generalist skills. At the end of the day the learning curve on these things is not that steep - that's why so many people online can post about it. When the need arises and it makes sense, the IDE/framework/tooling du jour will be there and you can learn it then in a few weeks. And if past is prologue in this industry, the people who have spent all their time fiddling with version N will need to reskill for version N+1 anyways.
• nurettin 1y ago
  In my experience, AI assistance can't really do what you can't. Sure, you can force it, but the result will be a discardable mess. Yesterday I used Gemini and Claude cli to create a symbolic regression library in C++ with added state of the art features (hobby). Gemini is there to assist when Claude runs out of context. It took me six hours. I cannot imagine how anyone could do this without being an expert in these fields. The initial code generation was pretty much on the spot, with a few subtle bugs here and there, few inefficient choices. As I made them add features, there were so many bad decisions that would break the initial interface simplicity, add shared states, constantly switch between SoA and AoS without a plan, introduce circular deps for no reason. I fought back, keeping code concise and functional, removed randomly introduced oop features, added low hanging optimizations. I hand fixed some of the compilation errors and warnings myself to prevent them from using up unnecessary context like a taxi driver who takes you to the long route. How would anyone who doesn't know what they are doing accomplish all this? They would have to learn. Sure, the bar is lower. This is a task that would have taken me a few weekends of hard work. And mind you this is a hobby project. I would've lost a significant amount of hair if I was fighting AI assistance while under pressure at work.
  • seba_dos1 1y ago
    Yes, this matches my experience - generated slop may help you realize what you don't understand very well yet and somewhat accelerate your learning, but you won't cut any shortcuts this way. You will only manage to do it once you actually understand how to do it, as only at that point you know how to guide the LLM to success.
• kissgyorgy 1y ago
  Context engineering is extremely hard,you won't just "get it" overnight. The other thing, it's just another code generation tool, treat it accordingly
• andrewstuart 1y ago
  So weird that people use these tools and think they essentially useless. How can my experience be so totally different?
  • devjab 1y ago
    I think it depends a lot on what you're trying to get them to do and what you're working on. I don't really buy into the whole "prompt engineering" part. I use agents quite a lot, but they have limitations. They excel at writing SQL, kubernetes configuration, bicep and similar... at least so long as you know what you're doing. I also have quite a lot of success with them keeping documentation and tests updated. I wouldn't recommend using them for your tests unless tests are a nice to have feature for you because you actively deal with corrupted states through explicit error handling, but we do. In my experience every model absolutely sucks at doing any form of software engineering. The thing is though, you don't need to do any software engineering to write software... until you do. Since LLM's got decent at programming we've let them loose among our BI and solar plant engineers and they're building some really useful tools in Python. They have no idea what they are doing, and they are still getting stuff to work. It's obviously absolute horse shit, but for a lot of it, the shit code will be "good enough" for nobody having to touch it for it's lifecycle. For the rest, actual software engineers take over. If you're working in a field where "good enough" is fine, then I imagine that LLM's can do a lot for you. It might sound like I'm being disrespectful, but I really don't mean to be. In my decades of experience with software in enterprise settings, I've seen so many programs build with horrible code just trundle along. Creating great value for the business while requiring no to minimal maintenance. Bad from a traditionalist software engineering perspective, but great from a business perspective (which is what matters). Not everything needs to scale after all. Anyway, I do question whether the author has actually worked with LLM's a lot. I would think that the strengths failings of LLM's would get obvious rather quickly to anyone who uses them.
    • nyarlathotep_ 1y ago
      > Bad from a traditionalist software engineering perspective, but great from a business perspective (which is what matters). Not everything needs to scale after all. Anyway, I do question whether the author has actually worked with LLM's a lot. I would think that the strengths failings of LLM's would get obvious rather quickly to anyone who uses them. Few things I've seen here. > If you're working in a field where "good enough" is fine, then I imagine that LLM's can do a lot for you. There's some specific timesheet software used for Gov Contractors that's hands-down, the worst web "application" I've ever used. Opening devTools on one of these pages is a riot--one of the "APIs" used return HTML documents with js file extensions so the browser fails to parse, every JS file is 1K lines long with blocks of global variables with various naming conventions, using 'back' in the browser just sort of fails and returns you to the sign-in page. Any modern web programmer would have a stroke looking at this thing. Apparently it's the standard software used for this purpose, relied on all over the industry. At a previous employer, I had to deal with some internal site for financial transactions and billing. Same story, ancient looking and performing web 'app' that had specific incantations to get it to actually work "don't refresh the page until that loads", piles of JS vomiting bizarre errors in the console, just a remarkable piece of junk. Anyway, short story long, there's LOADS of software that's terrible by any metric that have crucial purposes floating around that are far worse in terms of 'code quality' than any LLM would spit out. Also, I have more experience than I like to remember dealing with 'body shop' codebases. There is no way that an LLM is even capable of generating such low quality code. I legitimately think several of the projects I saw were sabotage; there's no way a human would deliberately write such inscrutable junk. Ever seen random python runtime errors cause the type of something would change depending on which branches were hit?
• myaccountonhn 1y ago
  I hate AIs because I think overall they're bad for the environment, centralizes power, and worsen quality. That said I never really doubted that they can help you get more done. I use it occasionally to refactor or generate some boilerplate. But very often though I need to rewrite most of the functionality, I find it more true the more code it generates. The agentic stuff I've tried have been terrible. Yeah I personally haven't used the latest models, but I deal with teammates who use them and the frontend it generates looks bland and the shell scripts generated with them are very hard to debug but also just flat out broken.
  • someoneiam 1y ago
    You sound like me. I've found some good uses for them generating ansible files for example (the output is regularly wrong, even with the latest and greatest model, but easy enough to correct). But otherwise, there always seems to be a cutoff in the complexity of a project where the ROI of just having to have built everything myself, and thus being completely immersed in the codebase, seems like it would have been much greater then prompting something to frankensteinian life. But most of all, I love the act of programming and cannot imagine myself giving that over to an LLM. "Managing" an LLM is to me a soul crushing - maybe anti-intellectual, but definitely boring - task. And if that is what we as professionals will be required to do in the future, I'll just change profession.
    • myaccountonhn 1y ago
      Yeah I agree, it's also so sad to have a future where we're dependent on giant US-based companies for all our thinking.
• r0ckarong 1y ago
  Is it possible these companies with their vast budgets pay to have all this drivel about how useful they are plastered on the web to pump up their value? I've started to learn what useful things LLMs can do for me but self sufficient coders they ain't.
• tiddles 1y ago
  Code is a liability. When I let a LLM take the wheel, I end up with thousands of lines of crappy abstractions and needless comments and strange patterns that take way more brain power to understand than if I did it myself. My current workflow has reverted to primitive copy paste into web chat (via Kagi Assistant). The friction is enough to make me put a lot of thought into each prompt and how much code context I give it (gathered via files-to-prompt from simonw). I have little experience with frontend and web apps, so I am trying out a supervised vibe coding flow. I give most of the code base per prompt, ask for a single feature, then read the code output fully and iterate on it a few times to reduce aforementioned bad patterns. Normally I will then type it out myself, or at most copy a few snippets of tens of lines. What doesn’t work I found is asking for the full file with changes applied already. Not only does it take a long time and waste tokens, it normally breaks/truncates/rewords unrelated code. So far I’m happy with how this project is going. I am familiar with all the code as I have audited and typed it out nearly entirely myself. I am actually retaining some knowledge and learning new concepts (reactive state with VanJS) and have confidence I can maintain this project even without an LLM in future, which includes handing it over to colleagues :)
• poulpy123 1y ago
  I agree that LLM are best at providing good auto completion, and even more for someone who types slowly like me. They are also good for providing examples and simple programs for technologies I'm unfamiliar with, something that I was using Google search for when the web and the search results where not just SEO crap.
• globular-toast 1y ago
  I'm going to properly write up my experiences with LLMs at some point, but some observations so far: 0. It's undeniable that LLMs are both amazing and fun. So is a clown at a children's birthday party, but you wouldn't try to ride the balloon horse, 1. If you are producing code in a language you are not familiar with, you need to be aware of the Gell-Mann amnesia effect[0], 2. It excels at language translation. This is what LLMs were originally designed for. So if you can clearly describe an algorithm in English, it will probably be able to translate that to Python, for example. But at that point you have to question why you didn't just write it in Python in the first place, 3. It gamifies things and causes you to work longer and harder. I'm not convinced it's really more efficient, but I am convinced I simply wouldn't have bothered to put in the effort to do certain things before because the perceived effort of writing code from scratch is higher than spinning a roulette wheel that might shit out the answer first time, 4. It stops you thinking for yourself. Ever worked with an incredibly knowledgable colleague who is always eager to help? Or got the feeling you've solved something before so spent time grepping through old codebases and bash history to find what you did last time instead of spending the exact same time doing it again? It's kinda like that except it knows everything and it's always there. I fear that ability will atrophy with extended use, 5. I don't feel like I really own the code it outputs. It feels more like working in a large existing codebase written by a team. When I have to get into things and understand the code it's like I'm reading it for the first time, even though I did check all the diffs etc. It's like having a manic junior run rampant then coming to you when something really needs to be understood. Again, it's the question of is this really saving me any time in the long run? [0] https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect
• crnkofe 1y ago
  I rarely use LLMs nowadays. Most of the time I'm fixing difficult to debug issues spanning multiple projects and a variety of backend systems and there's just no easy way to plug them to all this spaghetti. Autocomplete also spews out complete BS and was the first thing I disabled. I did find they very useful when writing completely new stuff (things like "write a <insert your favorite API" client or making test boilerplate ie. as a copy&paste replacement tool. It'd be nice to hear in detail how its been useful for other devs. There's too much propaganda around on how amazing it is and not nearly enough use cases.
• aprilfoo 1y ago
  I learned that the core of science and engineering is the ability to understand and control the systems that we build. This obviously involves complex tools, that need to be mastered too. Modern AI seems to be able to achieve results bypassing this basic principle, like magic: what can go wrong? So i cannot take seriously the gods in the art of the prompt claiming that they can watch TV while the code writes itself. But i believe that those who are already good in their domain can do a better job with such powerful tools, when they can master them too.
• dweekly 1y ago
  > "anything beyond writing a simple function always leads to useless junk." A strong statement like this was a reasonable take six months ago, perhaps. But with Claude Opus 4.1, o3-codex/gpt5, and Gemini Pro 2.5 this statement is pretty simply disproven with half an hour with them. Just last week I spent an afternoon rewriting an old Android app that was done in butterknife and got something functional and tested in a few hours. It involved an 8,000 line diff. The work had been quoted to me by a contractor as likely to take 3-4 months and cost tens of thousands of dollars. I designed a new website for my wife's band with a custom player widget, carousel photos, fully adaptive to large and small screens, good accessibility features, built from easy to edit JSON, SEO-optimized with microformats, and fast-loading. The first version I got running in 15 minutes. The final polished version took a few more days of ducking onto my laptop a few minutes here and there during an event. Without AI this would have taken me weeks to pull off and wouldn't have looked nearly as nice. IMO part of the key to the flow here is to avoid a temptation to one shot. First establish ground rules for working together in your AGENTS.md/CLAUDE.md that lays out your software engineering best principles (use git semantics, write out your plans, add lint and tests to commit hooks...). Then have it read through existing code and architecture. Then carefully put together an architecture and set of constraints into your README/PRD. Then build a high level engineering plan divided up into sequences tasks and write it down (vs just keeping in context). Only then do you allow any code to start to get written. And yes, you still need to supervise it (and iterate on your AGENTS/CLAUDE to avoid repeating yourself). But yeah, it's about 10-100x faster than me now with a flow like that. (Context on me: been programming 40 years, Stanford CS, ACM international programming competition finalist.) This new flow is extremely fun and addictive. It's also a new thing that uses technical knowledge but isn't exactly the same thing as coding. It's like having a pool of hard working inexpensive idiot savant never-worked-in-industry interns at your beck and call - the more you pour into giving them context and process, the better results you are gonna get.
• amdivia 1y ago
  > I can spend an entire day with a GPT and produce nothing of value. Although I believe you, I think this is due to improper use. I know this sounds like a "you're holding phone wrong" moment, but I found many uses for AI that I feel addicted to what its providing me. Like you I started with Vim long ago, in an attempt to reduce the thought -> code cycle. But after many years I started seeking something even faster, and LLMs seem to be it for me. From a quicker navigation of the codebase, to quickly writing the boilerplate for me, adding debugging statements where I need. Believe me there is a big jump in productivity if you use it right. You just gotta keep trying until you develop a sense of what you can ask it to do and what it would be good for
• bgwalter 1y ago
  > several of them are written by well-known developers who’ve been around for over a decade All of whom had greater output in terms of volume and impact before LLMs. And it is always the same people who now get voted to the top as opposed to a couple of months ago, when one of them got ratio'd for one of his exuberant blog posts.
  • simonw 1y ago
    You're just making things up to fit your own narrative at this point.
    • bgwalter 1y ago
      It is worse than that. I'm backed by the thriving $2 trillion anti-LLM industry! Admittedly it is an unfair fight since the pro-LLM people have no backing.
• sakopov 1y ago
  As someone who's been "vibe-coding" successfully at work I can say that what made me a believer in AI tools is agentic instructions. It is the difference between not being able to accomplish much and actually getting stuff done. There have been several articles published on this topic on HN. I would really recommend implementing that in your repositories.
• jama211 1y ago
  This isn’t a post, this is just a random opinion. Also, if they can code a small web app faster than an llm can I’ll eat my hat. I’d like to see them try and write the 2000+ lines required to get it up and running in less than 30 seconds. Seems like their attitude is exactly what you’d expect of someone who wants it to be useless from the outset. A self fulfilling hypothesis.
• tabemann 1y ago
  At my day job (not going to name the company) they have been pushing AI coding, and frankly I have found it to be essentially useless. As a simple test, I used Claude under GitHub Copilot to develop a simple program to blink an LED using a task under zeptoforth (my Forth implementation for ARM Cortex-M, which is on GitHub so it should have been in their training set) on a Pimoroni Pico Plus 2... which took a few hours to complete because it was constantly hallucinating my API's and getting minor points wrong that I had to tell it to fix... for it to only screw something else up the next time around... repeatedly. Then the next day they had us do an "agentive AI training" where we were supposed to use "agentive AI" to program example stuff... and I found it worked to a point ─ until it didn't work, and I could never actually make it do what I really wanted it to no matter how hard I "vibed". It just hit a stopping point where no amount of prompting would make it work. And then there's the actual use cases for AI at my day job ─ I have not seen any that apply to me. I write only limited amounts of code ─ I do not churn out the boilerplate that AI is sold as obviating ─ and the code I do write has to be correct, for values of correct where simply looking correct is not sufficient (and an AI won't know the underlying math behind things or how to avoid things like race conditions). If I actually used AI I would be spending more time trying to understand its output and having to fix it than I actually spend developing code, and I would have a harder time trusting the resulting code. I know how my code works because I wrote it ─ the same would not be true if I had an AI write my code for me. Sure, there are people at my day job who have gotten fully on-board with the AI thing, and who see it as being useful for things like, say, changing the C++ standard for a whole project at a time. However, I would not be able to trust the project after that point myself. Yes, it would take much more time to manually update the code, but then it would be much easier to trust afterwards, and would require far less fixing to handle all the cases where the AI screwed up.
• fathermarz 1y ago
  Not all LLMs are created equal and can be used for different aspects and workflows. I think the important bit is to gravitate to the tool that gives you the closest to your expected outcome. Copy pasting code into ChatGPT gives me unreliable results, because I don’t know how to talk to it. I find copilot annoying in Visual Studio. The workflow of cursor/windsurf is not intuitive for me. Gemini context is wicked but I didn’t like how it constructed its outputs. I tried Claude Code and became a believer, not because it’s perfect, but because it fits my cadence and allows me to pivot quickly. I still use chat for the occasional MSSQL syntax check, or Gemini for writing a technical doc. But there are so many variables and trial and error involved. I keep reading posts like this thinking that there is a flavour of fear holding people back. Nothing to fear, just level set your expectations, don’t believe the hype and be open to the vibes.
• bastawhiz 1y ago
  > I’m in a state where I can’t reconcile my own results with other people’s results. I hear people saying “this hammer is indestructible”, but when I pick it up, it’s just origami: made of paper, intricate, delicate, very cool-looking but I can’t even hammer a tomato with it. This is a really interesting signal to me. It's almost indisputable that you can get good results (I get good results pretty consistently) and so there's definitely something there. I don't think that folks who don't get good results are doing something "wrong" so much as not understanding how to work with the model to get good results. If I was at a company building these tools, the author would be the person I'd want to interview. I doubt it's a skill issue. And it's definitely not user error. You can't sell a tool that is said to do something but the user can't replicate the result. A tool that works but only after you've invested lots of time working to reverse engineer it in your head isn't a good tool, even if it's extremely powerful. The tool needs to be customizable and personalizable and have safety rails to prevent bad results.
• jacobedawson 1y ago
  "But anything beyond writing a simple function always leads to useless junk." If that's the article writer's experience then they are simply using it incorrectly. The author seems to suggest most of their usage involves pasting code or chatting via the text interface, likely with a non-sota model. It would be surprising to find that someone using e.g. Claude Code cli with full access to a codebase and carefully considered prompts is always getting useless junk.
• arisAlexis 1y ago
  You are using them wrong and yes you will be left behind. But we will all get replaced so it doesnt matter as much.
• mortsnort 1y ago
  Does the author really feel alone? There's posts like this on the front page of HN everyday. If you don't want AI to be useful for coding then you can find a way to convince yourself it isn't. Surely, at the bare minimum, you can use it for boilerplate and auto-complete, which in itself is often a massive time savings.