26 ms·

Ask HN: Anyone struggling to get value out of coding LLMs?

• mhb 1y ago
  Agreed. I have an additional question. Given what look like limitations on tasks of larger scope, what problems are people who claim that they have run the LLMs for days working on?
• pabna 1y ago
  I think they're amazing but still a new tool only 3 years old and we have like 20 years left till Super-AGI
• chewz 1y ago
  I use this analogy. In the early 90's I had been programming in assembler and sometimes in pure hex codes. I had been very good at that, creating really effective code, tight, using as little resources as possible. But then resources became cheap and it stoped matter. Yeah, the tight well designed machine code is still some sort of art expression but for practical purpose it makes sense to write a program in higher level language and waste a few MB...
  • bonki 1y ago
    This is exactly why we can't have nice things :(
  • lordkrandel 1y ago
    I don't agree. It may be true that most code is throwaway. But you trust a C compiler, or a Python intepreter, to do their job in a deterministic way. You will never be able to trust Copilot telling you that "this should be the code you are using". It may suggest you using AWS, or Google, or Microsoft, or Tencent infrastructure. An LLM can even push you a specific style, or political agenda, without even you realizing it. I hate polarized discussion all-or-nothing thinking about LLMs. See how perfectly and reliably they can translate text in whatever language. See them fail at aligning a table with a monospace font.
    • kragen 1y ago
      I think you probably need a multimodal LLM to align a table with a monospace font. Blind human programmers using screenreaders will also have difficulty with that particular task. It doesn't mean they're bad at programming; it just means they can't see.
      • lordkrandel 1y ago
        Human blind programmers may write programs to align their own tables like requested, in autonomy. Instead of me seeing them doing wrong, asking for a fix, and them doing it wrong again in an endless cycle. Also, when I tell them the solution and ask to add a feature - bam, the table is made wrong again.
• terrut 1y ago
  I am one of those lazy IT guys that is very content working in support and ops. I understand a lot of programming concepts and do a bit of automation scripting but never really bothered to learn any proper language fully. Vibe coding was made for people like me. I just need something that works, but I can still ask for code that can be maintained and expanded if needed. It finally clicked for me when I tried Gemini and ChatGPT side by side. I found that my style of working is more iterative than starting with a fully formed plan. Gemini did well on oneshots, but my lack of experience made the output messy. This made it clear to me that the more chatty ChatGPT was working for me since it seems to incorporate new stuff better. Great for those "Oh, crap I didn't think of that" moments that come up for inexperienced devs like me. With ChatGPT I use a modular approach. I first plan a high level concept with 03, then we consider best practices for each. After that I get best results with 4o and Canvas since that model doesn't seem to overthink and change direction as much. Granted, my creations are not pushing up against the limits of human knowledge, but I consistently get clean maintainable results this way. Recently I made a browser extension to show me local times when I hover over text on a website that shows an international time. It uses regex to find the text, and I would never have been able to crank this out myself without spending considerable time learning it. This weekend I made a Linux app to help rice a spare monitor so it shows scrolling cheat sheets to help me memorize stuff. This turned out so well, that I might put it up on GitHub. For dilettantes like me this opens up a whole new world of fun and possibilities.
• lordkrandel 1y ago
  LLM is good for throwaway code. Easier to write, harder to maintain, diagnose, repair, also with LLMs. Which is most code that is not a product. Fast food, assembly line, factory may be examples, but there is a HUGE catch: When a machine with a good setup makes your burger, car or wristwatch, you can be sure that at 99.99% it is as specified. You trust the machine. With LLMs, you have to verify each single step, and if you don't, it simply doesn't work. You cannot trust them to work autonomously 24/7. That's why you ain't losing your job, yet.
• gwd 1y ago
  Basically yes, once the "problem" gets too big, the LLM stops being useful. As you say, it's great for automating away boring things; as a more complicated search & replace, for instance. Or, "Implement methods so that it satisfies this interface", where the methods are pretty obvious. Or even "Fill out stub CRUD operations for this set of resources in the API". I've recently started asking Claude Opus 4 to review my patches when I'm done, and it's occasionally caught errors, and sometimes has been good at prompting me to do something I know I really should be doing. But once you get past a certain complexity level -- which isn't really that far - it just stops being useful. For one thing, the changes which need to be made often span multiple files, each of which is fairly large; so I try to think carefully about which files would need to be touched to make a change; after which point I find I have an idea what needs to be changed anyway. That said, using the AI like a "rubber duck" programmer isn't necessarily bad. Basically, I ask it to make a change; if it makes it and it's good, great! If it's a bit random, I just take over and do it myself. I've only wasted the time of reviewing the LLM's very first change, as nearly everything else I'd've had to do if I wrote the patch myself from scratch anyway. Furthermore, I often find it much easier to take a framework that's mostly in the right direction and modify it the way that I want, than to code up everything from scratch. So if I say, "Implement this", and then end up modifying nearly everything, it still seems like less effort than starting from scratch myself. The key thing is that I don't work hard at trying to make the LLM do something it's clearly having trouble with. Sometimes the specification was unclear and it made a reasonable assumption; but if I tell it to do something and it's still having trouble, I just finish the task myself.
• blueboo 1y ago
  They can do chores that clear your day to do more challenging work. That’s immensely valuable and pretty game changing
• csomar 1y ago
  I was just struggling with getting a standard/template npm package up and running with a few customizations. Ended up just following one of the popular npm packages. This is Claude 4 and although it is good at writing code, I feel like it gets dumber at certain tasks especially when you want it to connect things together. It very easily messes one thing up and then when you include the errors, it spirals from there into madness. > Am I just not using the tools correctly? No, there is no secret sauce and no secret prompting. If LLMs were capable, we'll see lots of new software generated by it given how fast LLMs are at writing code. Theoretically, assuming a conservative 10token/s speed and a 100M token for Chromium code base, you could write a new browser with LLMs in only 115 days.
  • deleted 1y ago
    [deleted]
  • deleted 1y ago
    [deleted]
• idw 1y ago
  Simon Willison has helpful recent advice on this: Here’s how I use LLMs to help me write code (11th March 2025) https://simonwillison.net/2025/Mar/11/us.ing-llms-for-code/ https://simonwillison.net/2025/Mar/11/us.ing-llms-for-code/
  • sacredSatan 1y ago
    there's a typo in the link, this is the correct link: https://simonwillison.net/2025/Mar/11/using-llms-for-code/ https://simonwillison.net/2025/Mar/11/using-llms-for-code/
  • JimDabell 1y ago
    There’s also discussion here about an article written by David Crawshaw, the CTO of Tailscale: > How I program with LLMs — https://news.ycombinator.com/item?id=42617645 https://news.ycombinator.com/item?id=42617645
  • deleted 1y ago
    [deleted]
• gyomu 1y ago
  There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former group aren’t all utterly dominating the market and obliterating their competitors with their revolutionary products and blazing fast iteration speed.
  • lm28469 1y ago
    That's because they fall under the "engineers" category like my local postman and Michael Schumacher both fall under the "driver" category. Most people don't work on anything technically hard, most problems are business logic issues that aren't solved technically or legacy code workarounds for which you need to put 3-10 domain experts in a room for a few hours to solve.
    • nobodywillobsrv 1y ago
      So true and it makes me feel better to have this mental model. I find it useful for simple things that are perhaps just beyond search and copy paste but sometimes wonder if the more technical stuff is not even quite right as it is possible to sort of brainwash yourself into not thinking clearly when nagging an llm into being reasonable on something it has a hard time with.
    • sunrunner 1y ago
      There are a lot of terms in software development that have been co-opted from other disciplines and misrepresent a lot of development work, including 'engineering' and 'architecture'. I think it's helpful to think of engineering as a _process_ instead of a role, and the reality is that a lot of development work doesn't necessarily rely on the strong engineering methodology (e.g. measurement, material properties, tolerances, modelling, etc.) that the people developing the software might imagine just based on the number of job adverts for 'engineers'. This isn't a bad thing. There are hundreds or thousands of different but equally valid solutions to getting a program to do a thing, but not recognising that most code writing is somewhere between a art and engineering and is neither a purely artistic discipline but also rarely a purely engineering one is useful. The kinds of engineering and architecture that people think of in software development only really represent common practices and shared language (e.g. design patterns, architectural patterns) and not a strong engineering practice or any kind of truth about how software actually runs. (Unless you're writing software for launching rockets, in which case the engineering _process_ porbably should be strong).
• protocolture 1y ago
  I find the code quality is an LLMs first priority. It might not execute but it will have good variable names and comments. I find that so far their quality is horizontal, not vertical. A project that involves small depth across 5 languages/silos? Extremely useful. A long project in a single language? Nearly useless. I feel like its token memory. And I also feel like the solution will be deeper code modularisation.
• ZaoLahma 1y ago
  I would never use an LLM for trying to generate a full solution to a complex problem. That's not what they're good at. What LLMs are good at and their main value I'd argue, is nudging you along and removing the need to implement things that "just take time". Like some days back I needed to construct a string with some information for a log entry, and the LLM that we have suggested a solution that was both elegant and provided a nicer formatted string than what I had in mind. Instead of spending 10-15 minutes on it, I spent 30 seconds and got something that was nicer than what I would have done. It's these little things that add up and create value, in my opinion.
  • rollcat 1y ago
    > What LLMs are good at and their main value I'd argue, is nudging you along and removing the need to implement things that "just take time". I was still learning Java at uni (while being a Python/Lisp fanboy) when I realised this: - Complex and wordy languages need tooling (like autocomplete, autoformatting) to handle the tedious parts. - Simple and expressive languages can get away with coding in notepad.exe. - Lisp, as simple and powerful as it is, needs brace highlighting. You. Simply. Just. Can't. Now 10, 20 years later you can look back at the evolution of many of these languages; some trends I've observed: - Java, C#, C++, have all borrowed a lot from functional languages. - JVM has Clojure. - Go stubbornly insists on "if err != nil" - which isn't even the worst part; the following fmt.Errorf is. - Rust (post-1.0) cautiously moved towards "?"; Zig (still pre-1.0) also has specific syntax for errors. - Python is slowly getting more verbose, mostly because of type annotations. - Autoformatters are actually great, you don't even have to care about indenting code as you spit it out, but... Python, being whitespace-sensitive, makes them a bit less useful. Good tooling helps you with wordy languages. Expressive languages help you write concise code. Code is read much more often than it's written. Deterministic tooling can work with the structure of the code, but LMs (being probabilistic) can help you work with its intent. Language models are an evolution of automated tooling - they will get better, just like autocomplete got better; but they will never "solve" coding. In my opinion, there's no truth here, only opinions.
• souenzzo 1y ago
  my code setup hasn't changed in 10 years I tried to use many LLM tools. They are generally not capable of doing anything useful in a real project. Maybe solutions like MCP, that allow the LLM to access the git history make the LLM become useful for someone that actually works on a project.
  • consumer451 1y ago
    Postgres or Supabase MCP is really useful for me, especially when dealing with data related issues like permissions bugs. It seems faster to me. Example: > I cannot see the Create Project button as user project-admin@domain.com. Please use Supabase MCP to see if I have the correct permissions, if so, are we handling it correctly in the UI?
  • lyu07282 1y ago
    I find that hard to believe, I also don't think LLMs provide the value some others are seeing but there is also code search and refactoring tasks that LLMs can help with. Instead of heaving to write a codemod, I can just write a prompt and accomplish the same thing in much less time. To say they have literally zero value just seems uninformed tbh. That's the sort of black and white thinking that isn't very helpful in the conversation.
• avalys 1y ago
  I find LLMs very useful when I need to learn something new, e.g. a new library or API, or a new language. It is much faster to just ask the LLM how to render text in OpenGL, or whatever, than read through tons of terrible documentation. Or, if I have a bunch of repetitive boilerplate stuff that I have no good templates for or cannot borrow from another part of the codebase. But when it comes to what I consider the actual 'work', the part of my codebase that is actually unique to the product I'm building, I rarely find them valuable in the "letting them write the code for me" sense. I reflected once that very little of my time as a senior engineer is actually spent just banging out code. The actual writing of the code is never the hard part or time-consuming part for me - it's figuring out the right architecture, figuring out how to properly refactor someone else's hairball, finding performance issues, debugging rare bugs, etc. Yes, LLMs accelerate the process of writing the boilerplate, but unless you're building brand new products from scratch every 2nd week, how much boilerplate are you really writing? If the answer is "a lot", you might consider how to solve that problem without relying on LLMs!
  • belZaah 1y ago
    There’s no silver bullet. Conceptualization is the hard part, exactly as you say. Mythical man-month is an important text and should be studied more.
    • namaria 1y ago
      And also the seminal papers by Gödel, Turing, and the books on cybernetics (robotics and automation in 1950s) by Ashby and Wiener. The limits of what can be automated have been clear for almost a century now. Yet people insist in selling ways to automate your way out of social, political, economical problems. Come to think about it, people should also read Comways's paper as well.
      • t917910 1y ago
        What is Comways's paper?
        • 1y ago
• Narciss 1y ago
  I have been getting very good results. What matters: - the model -> choose the SOTA (currently Claude 4 Opus). I use it mostly in Cursor. - the prompt: give it enough context to go by, reference files (especially the ones where it can start delving deeper from), be very clear in your intentions. Do bullet points. - for a complex problem: ask it to break down its plan for you first. Then have a look to make sure it’s ok. If you need to change anything in your plan, now’s the time. Only then ask it to build the code. - be patient: SOTA models currently aren’t very fast I work at a company with millions of MAU, as well as do side projects - for the company, I do spend a bit more time checking and cleaning code, but lately with the new models less and less. For my side projects, I just bang through with the flow above. Good luck!
• discordance 1y ago
  Have you tried using a rules file for for those micromanage-y tasks you mentioned? I have collected similar requests over time and I don't have to remind GH copilot/Claude as much anymore.
• reify 1y ago
  An IBM study based on conversations with 2,000 global CEOs recently found that only 25% of AI initiatives have delivered their expected ROI over the last few years, and, worse still, "64% of CEOs surveyed acknowledge that the risk of falling behind drives investment in some technologies before they have a clear understanding of the value they bring to the organization." 50% of respondents also found that "the pace of recent investments has left their organization with disconnected, piecemeal technology," almost as if they don't know what they're doing and are just putting AI in stuff for no reason. https://newsroom.ibm.com/2025-05-06-ibm-study-ceos-double-down-on-ai-while-navigating-enterprise-hurdles https://newsroom.ibm.com/2025-05-06-ibm-study-ceos-double-do...
  • esafak 1y ago
    CEOs are supposed to be able to make snap decisions without having the big picture. If everybody else is doing it, following suit will mean you won't be left behind. Worst case, you'll all make the same mistake. Of course, if you know better than the others, you can profit from that.
• journal 1y ago
  As much as someone else is struggling at something else? It's not for everyone just like programming is not for everyone. I don't even type the code anymore, I just copy and paste it, is that still programming? I don't remember last time I had to type out a complete line up to ;.
• BoredPositron 1y ago
  Use them like a butler or bad maid. They can do the absolute grunt work now and with time they might get better to do more serious tasks.
• tootyskooty 1y ago
  I still get a lot more value from webuis than from the agentic offerings, also for more general coding tasks. The main benefit is that I can more easily manage context. A muddied or insufficient context can sometimes degrade performance by a whole model generation, ime. What works for me is collecting it manually and going one implementation chunk at a time. If it fails, I either do it myself or break it down into smaller chunks. As models got better these chunks got larger and larger. Collecting context manually forces me to really consider what information is necessary to solve the problem, and it's much easier to then jump in to fix issues or break it down compared to sending it off blind. It also makes it a lot faster, since I shortcircuit the context collection step and it's easier to course-correct it. Collecting manually is about 10 seconds of work as I have an extension that copies all files I have opened to the clipboard.
• pornel 1y ago
  I find they're useful for the first 100 lines of a program (toy problems, boilerplate). As the project becomes non-trivial (>1000 lines), they get increasingly likely to get confused. They can still seem helpful, but they may be confidently incorrect. This makes checking their outputs harder. Eventually silly bugs slip through, cost me more time than all of the time LLMs saved previously.
• nottorp 1y ago
  So you want to use a LLM on a code base. You have to feed it your code base as part of the prompt, which is limited in size. I don't suppose there's any solution where you can somehow further train a LLM on your code base to make it become part of the neural net and not part of the prompt? This could be useful on a large ish code base for helping with onboarding at the least. Of course you'd have to do both the running and training locally, so there's no incentive for the LLM peddlers to offer that...
  • esafak 1y ago
    Modern tools don't fine tune on your code base but use RAG; select the context to feed it to the LLM with each request. The better the context inference algorithm, the better the results. See if your tool tells you what files it selected.
    • nottorp 1y ago
      Yes but why? On a large code base it would probably work better if you didn't need to put it all in the context... if you even could...
      • esafak 1y ago
        I am saying the same thing; you want to selectively establish the context, not pass it all.
        • deleted 1y ago
          [deleted]
        • nottorp 1y ago
          I want to train the LLM on the whole code base and then pass a hand picked context specific to what I'm asking. So it doesn't only suggest what can be found on w3schools and geeks4geeks and maybe stackoverflow, but also whatever idioms and utility functions my code base has.
• Yiling-J 1y ago
  I think Jules does a good job at "generating code I'm willing to maintain." I never use Jules to write code from scratch. Instead, I usually write about 90% of the code myself, then use the agent to refactor, add tests (based on some I've already written), or make small improvements. Most of the time, the output isn't perfect, but it's good enough to keep moving forward. And since I’ve already written most of the code, Jules tends to follow my style. The final result isn’t just 100%, it’s more like 120%. Because of those little refactors and improvements I’d probably be too lazy to do if I were writing everything myself.
• Maro 1y ago
  I'm a manager at work, so I only code during the weekends on personal projects these days. In that context, I get a lot of value out of plain-vanilla ChatGPT, it's able to write high-quality code for my toy use-cases in the 100-500 LOC range.
• therealmarv 1y ago
  There are two key points which are important to get most out of LLM coding assistants: 1. Use a high quality model with big context windows via API (I recommend Openrouter). E.g. Google Gemini 2.5 Pro is one of the best which keeps constant good quality (OpenAI reasoning models can be better in problem solving but it's kinda a mixed bag). Other people swear by the Claude Sonnet models. 2. Upgrade your code tools you combine with this high quality models. Google Jules and OpenAI Codex are so brand new and have a totally different aim than Cursor. Don't use them (yet). Maybe they will get good enough in future. I would focus on established tools like aider (steepest learning curve), roo code (easier) to be paired with Openrouter and if you want to have it really easy claude code (only useful with a 100-200 USD Anthropic subscription IMHO). On average you will get better results with Aider, roo, claude code than with Cursor or Windsurf. Btw. I think Cursor and Windsurf are great as a starter because you buy a subscription with 15-20 USD and are set. It can be most likely that the more high quality tools burn more tokens and you spent more per month but you also get better quality back in return. Last but not least and can be applied to every coding assistant: Improve your coding prompts (be more specific in regards to files or sources), do smaller and more iterations until reaching your final result.
  • prmph 1y ago
    It's not about the tools. The complaint here is a general one that shows up no matter what tools you are using
    • therealmarv 1y ago
      And I tell from personal experience: The tools matter (!) for more complex problems. Op said literally "Am I just not using the tools correctly?" and I said for the 2 of 3 tools OP tried are so new and experimental I would put them away (for now).
      • prmph 1y ago
        You're still perpetuating this myth that it is about the tools, when every interaction I've had with coding assistants and LLMs veers into the territory OP describes. To be fair, the work I am doing is pretty complex and novel. And I use a wide range of model/versions.I mostly use Claude, from 3.5 to the newest 4.0. I also use Gemini and with Copilot.
• xiphias2 1y ago
  For me anything before Codex model with webui was time waster. Now with webui what's important is to constantly add tests around the code base, also if it gets stuck, go through the logs and understand why. It's more of a management role of ,,unblocking'' the LLM if it gets stuck and working with it than fitting it to my previous workflow.
  • kordlessagain 1y ago
    I agree with this. What tools are you using with Claude?
• klntsky 1y ago
  > I had to micromanage them infinitely You are missing a crucial part of the process - writing rules
• coffeefirst 1y ago
  I’ve landed where you are. And given the output I’ve seen when I’ve tried to make it do more, I seriously doubt any of this magic generated software actually works.
• elzbardico 1y ago
  There's no silver bullet. It is amazing how in our field we repeatedly forget this simple advice from Fred Brooks. In my experience, LLMs are way more useful for coding and less problem-prone when you use them without exaggerated expectations and understand that it was trained on buggy code, and that of course it is going to generate buggy code. Because almost all code is buggy. Don't delegate design for it, use functional decomposition, do your homework and then use LLMs to eliminate toil, to deal with the boring stuff, to guide you on unfamiliar territory. But LLMs don't eliminate the need for you to understand the code that goes with your name. And usually, if you think a piece of LLM generated code is perfect, remember that maybe the defects are there, but you need to improve your own knowledge and skills to find it. Be always suspicious, don't trust it blindly.
  • esafak 1y ago
    I agree with what you said except the delegation of design, if by which you mean architecture, because I find that it is good at that too. When it comes to design I ask it to describe it, and iterate at a high level before proceeding to implementation. Just as one would in real life.
    • infecto 1y ago
      I tend to agree with you with the caveat that I believe it’s important to full understand what’s being built. Not suggesting you think either way.
  • infecto 1y ago
    Wish this was higher up. I fully agree and it’s the same camp for me. I am not sure why but the vast majority either treat it as a magic bullet or says it’s pointless and nothing but flaws. It’s not perfect it might write buggy code but I write buggy code too and would guess most engineers do. It takes a lot of the menial tasks out of the way. Sure I might spruce up some of the code it writes but why is that a deal breaker?
• DonHopkins 1y ago
  I am struggling to programmatically get syntactically valid JSON out of LLMs, using both the OpenAI and Vertex apis. I am using: "response_format": { "type": "json_object" } And with Vertex: "generationConfig": { "responseMimeType": "application/json" } And even: "response_format": { "type": "json_schema", "json_schema": { ... And with Vertex: "generationConfig": { "responseMimeType": "application/json", "responseSchema": { ... Neither of them is reliable. It always gives me json in the format of a markup document with a single json code block: ```json {} ``` Sure I can strip the code fence, but it's mighty suspicious I asked for json and got markup. I am getting a huge number of json syntax errors, so it's not even getting to the schemas. When i did get to the schemas, it was occasionally leaving out fields that I'd declared were required (even if i.e. null or an empty array). So I had to mark them as not required, since the strict schema wasn't guiding it to produce correct output, just catching it when it did. I admit I'm challenging it by asking it to produce json that contains big strings of markup, which might even contain code blocks with nested json. If that's a problem, I'll refactor how I send it prompts so it doesn't nest different types. But that's not easy or efficient, because I need it to return both json and markup in one call, so if I want to use "responseMimeType": "application/json" and "responseSchema", then it can ONLY be json, and the markup NEEDS to be embedded in the json, not the other way around, and there's no way to return both while still getting json and schema validation. I'd hate to have to use tool calls as "out parameters". But I'm still getting a lot of json parsing problems and schema validation problems that aren't related to nested json formatting. Are other people regularly seeing markup json code blocks around what's supposed to be pure json, and getting a lot of json parsing and schema validation issues?
  • slashtmpslashme 1y ago
    Use structured output / response format with JSON schema feature if your models provide it (openai for sure does, I think it's json_schema instead of json_object and provide a schema). Otherwise there are libraries like instructor (python) which can help to generate json in markdown code blocks.
• rikroots 1y ago
  I've spent the past week overcoming my fear of Google's Gemini and OpenAI's ChatGPT. Things I've learned: - Using an AI for strange tasks like using a TTS model to turn snippets of IPA text (for a constructed language) into an audio file (via CLI) - much of the task turned out to be setting up stuff. Gemini was not very good when it came to giving me instructions for doing things in the GCP and Google Workspace browser consoles. ChatGPT was much clearer with instructions for setting up AWS CLI locally and navigating the AWS browser console to create dedicated user for the task etc. The final audio results were mixed, but then that's what you get when trying to beat a commercial TTS AI to doing something it really thinks you're mad to try. - Working with ChatGPT to interrogate a Javascript library to produce a markdown file summarising the library's functionality and usage, to save me the time of repeating the exercise with LLMs during future sessions. Sadly the exercise didn't help solve the truly useless code LLMs generate when using the library ... but it's a start. - LLMs are surprisingly good at massaging my ego - once I learned how to first instruct them to take on a given persona before performing a task: <As an English literature academic, analyse the following poem: title: Tournesols; epigraph: (After "Six Sunflowers, 1888" by van Gogh / felled by bombs in 1945); text: This presented image, dead as the hand that / drew it, an echo blown to my time yet // flames erupt from each auburn wheel - / they lick at the air, the desk: sinews // of heat shovelled on cloth. Leaves / jag and drop to touch green glaze - // I want to tooth those dry seeds, sat / by the window caught on the pot's neck // and swallow sunshine. So strong / that lost paint of the hidden man.> I still fear LLMs, but now I fear them a little less ...
• kragen 1y ago
  > I asked friends who are enthusiastic vibe coders and they basically said "your standards are too high". Sure, vibe coders by definition can't have any standards for the code they're generating because by definition they never look at it. > Is the model for success here that you just say "I don't care about code quality because I don't have to maintain it because I will use LLMs for that too?" Vibe coding may work for some purposes, but if it were currently a successful strategy in all cases, or even narrowly for improving AI, Google AI or DeepSeek or somebody would be improving their product far faster than mere humans could, by virtue of having more budget for GPUs and TPUs than you do, and more advanced AI models, too. If and when this happens you should not expect to find out by your job getting easier; rather, you'll be watching the news and extremely unexpected things will be happening. You won't find out that they were caused by AI until later, if ever.
• y42 1y ago
  Seconded! I'm using LLMs in many different ways—like you—starting with small troubleshooting tasks, quick shell scripts, coding, or simply asking questions. I use a wide variety of tools. For more private or personal tasks, I mostly rely on Claude and OpenAI; sometimes I also use Google or Perplexity—whichever gives the best results. For business purposes, I either use Copilot within VSCode or, via an internal corporate platform, Claude, OpenAI, and Google. I’ve also experimented a bit with Copilot Studio. I’ve been working like this for about a year and a half now, though I haven’t had access to every tool the entire time. So far, I can say this: Yes, LLMs have increased my productivity. I’m experimenting with different programming languages, which is quite fun. I’m gaining a better understanding of various topics, and that definitely makes some things easier. But—regardless of the model or its version—I also find myself getting really, really frustrated. The more complex the task, the more I step outside of well-trodden paths, and the more it's not just about piecing together simple components… the more they all tend to fail. And if that’s not enough: in some cases, I’d even say it takes more time to fix the mess an LLM makes than it ever saved me in the first place. Right now, my honest conclusion is this: LLMs are useful for small code completion tasks, troubleshooting and explaining —but that’s about it. They’re not taking our jobs anytime soon.
• Sai_Praneeth 1y ago
  i haven't used cursor or codex or any system that says "agentic coding experience" i speak in thoughts in my head and it is better to just translate those thoughts to code directly. putting them into a language for LLMs to make sense and understanding the output is oof... too much overhead. and yeah the micromanagement, correcting mistakes, miscommunications, its shit i just code like the old days and if i need any assistance, i use chatgpt
• bjourne 1y ago
  > All these use cases work great, I save a lot of time. So why are you complaining? I use AI all the time to give me suggestions and ideas. But I write the perfect code myself.
• fdsafkas 1y ago
  [dead]
• fidotron 1y ago
  I have seen two apparently contradictory things. Firstly, there absolutely are people popping up in certain domains with LLM assisted developed products that could not have managed it otherwise, with results you would not suspect were made that way if you were not told. However, I share the same problem myself. The root of it is "analysis is harder than synthesis". i.e. if you have to be sure of the correctness of the code it's far easier to write it yourself than establish that an LLM got it right. This probably means needing to change how to split things out to LLMs in ways human co-workers would find intolerable.
• wejick 1y ago
  Working with AI centric IDE on the mature codebase needs different skill set that I suspect related to people management. As the pattern would be describing a problem well, making a good plan, delegate and giving feedback. On the other side, getting a good flow is not trivial. I had to tweak rules, how to describe problem, how to plan the work and how to ask the agent. It takes time to be productive. Eg. Asking agent to create a script to do string manipulation is better than asking them to do inplace edit. As it's easier to debug and repeat.
• dboreham 1y ago
  In my team we call it "The Intern".
• calrain 1y ago
  It's easy to get good productivity out of LLMs in complex apps, here are my tips: Create a directory in the root of your project called /specs Chat with a LLM to drill into ideas having it play the role of a Startup Advisor, work through problem definitions, what your approach is, and build a high level plan. If you are happy with the state of your direction, ask the LLM to build a product-strategy.md file with the sole purpose of describing to an AI Agent what the goal of the project is. Discuss with an LLM all sorts of issues like: Components of the site Mono Repo vs targeted repos Security issues and your approach to them High level map of technologies you will use Strong references to KISS rules and don't over complicate A key rule is do not build features for future use Wrap that up in a spec md file Continue this process until you have a detailed spec, with smaller .md files indexed from your main README.md spec file Continue promoting the role of AI Developer, AI Consumer, AI Advisor, End User Break all work into Phase 1 (MVP), Phase 2, and future phases, don't get more granular (only do Phase 2 if needed) Ask LLM to document best practice development standards and document in your CLAUDE.md or whatever you use. Discuss the standards, err to industry standard if you are lost Challenge the LLM while building standards, keep looping back and adjusting earlier assumptions Instruct AI Agents like Claude Code to target on specific spec files and implement only Phase 1. If you get stuck on how to do that, ask an LLM on how to prompt your coding agent to focus, you will learn how they operate. Always ask the coding agent to review any markdown files used to document your solution and update with current features, progress, and next issues. Paste all .md files back into other AI's e.g. high models of ChatGPT and ask it to review and identify missing areas / issues Don't believe everything the agents say, challenge them and refuse to let them make you happy or confirm your actions, that is not their job. Always provide context around errors that you want to solve, read the error, read the line number, paste in the whole function or focus your Cursor.ai prompt to that file. Work with all the AI's, each has their strength. Don't use the free models, pay, it's like running a business with borrowed tools, don't. Learn like crazy, there are so many tips I'm nowhere near learning. Be kind to your agent (Edited: formatting)
  • esafak 1y ago
    Your formatting made it harder to read. I'd remove the leading spaces, except for list items.
• 101011 1y ago
  If I could offer another suggestion from what's been discussed so far - try Claude Code - they are doing something different than the other offerings around how they manage context with the LLM and the results are quite different than everything else. Also, the big difference with this tool is that you spend more time planning, don't expect it to 1 shot, you need to think about how you go from epic to task first, THEN you let it execute.
• cube2222 1y ago
  I get a lot of value out of LLMs including for existing codebases and authoring / modifying code in them. However, only maybe 10% of that is agentic coding. Thus, my recommendation would be - try non-agentic tools. My primary workflow is something that works with the Zed editor, and which I later ported as a custom plugin to Goland. Basically, you first chat with the AI in a sidebar possibly embedding a couple of files in the discussion (so far nothing new), and then (this is the new part) you use contextual inline edits to rewrite code "surgically". Importantly, the inline edits have to be contextual, they need to know both the content of the edited file, and of the conversation so far, so they will usually just have a prompt like "implement what we discussed". From all I know, only Zed's AI assistant supports this. With this I've had a lot of success. I still effectively make all architectural decisions, it just handles the nitty-gritty details, and with enough context in the chat from the current codebase (in my case usually tens of thousands of tokens worth of embedded files) it will also adhere very well to your code-style.
  • maxnevermind 1y ago
    > From all I know, only Zed's AI assistant supports this. You mean the session context awareness? I thought it is a default in all major IDE/plugins. Or you mean some specific trait of that feature?
    • cube2222 1y ago
      For inline edits, yeah. The easiest way to check is to put some “secret passphrase” in the chat, and then try using inline edits to “add the passphrase as a comment”.
• ninetyninenine 1y ago
  Relax once it can do what you want you’re out of a job.
• never_inline 1y ago
  Give it context and copy paste the code yourself. They're good at coming up with new code. Give it function signature with types and it will give pretty good implementation. Tell it to edit something, and it will lose track. The write-lint-fix workflow with LLMs doesn't work for me - LLM is monkey brain edits unrelated parts of code.
• HenryBemis 1y ago
  I tend to spend 15mins to write clear requirements (functional and non-functional specs). And then ChatGTP works its miracle. When I ask it "do write code in X language that does so-and-so", it's proper crapware. But those 15mins of reqs, save me 4-5 hours of writing back and forth, looking at manuals, etc (I'm not a super dev, I'm not even a professional dev.) But I do ask it to write code for me as am I trying to solve (my) small IT problems, and from seeing various marketplaces, nobody has posted code/software to do what I want. Perhaps one day I'll 'incorporate myself' and start posting my solutions and perhaps make some dough.. but the I benefit far more than the $20 a month I am paying. The right 'prompt' (with plenty of specs and controls) saves me from the (classic!) swing-on-tree example: https://fersys.cloud/wp-content/uploads/2023/02/4.jpg https://fersys.cloud/wp-content/uploads/2023/02/4.jpg
• morsecodist 1y ago
  I have had a pretty similar experience to you. I have found some value here: - I find it is pretty good at making fairly self-contained react components or even pages especially if you are using a popular UI library - It is pretty reliable at making well-defined pure functions and I find it easier to validate that these are correct - It can be good for boilerplate in popular frameworks I sometimes feel like I am losing my mind because people report these super powerful end to end experiences and I have yet to see anything close in my day to day usage despite really trying. I find it completely falls over on a complete feature. I tried using aider and people seem to love it but it was just a disaster for me. I wanted to implement a fairly simple templated email feature in a Next.js app. The kind of thing that would take me about a day. This is one of the most typical development scenarios I can imagine. I described the feature in it's entirety and aider completely failed, not even close. So I started describing sub-features one by one and it seemed to work better. But as I added more and more, existing parts began to break, I explained the issues to aider and it just got worse and worse with every prompt. I tried to fix it manually but the code was a mess.
• HybridCurve 1y ago
  I've only had great luck with the LLMs(chatgpt 3o) generated Perl code. It was able to synthesize code for a GTK2/3 application fairly consistently, without generating any syntax errors. Most of the code worked as described, and it seemed to make more mistakes misunderstand my descriptions of features rather than when implementing them. My colleagues suggested it was because Perl's popularity had fallen significantly before 2016, and the training data set might've had much less noise.
• estebarb 1y ago
  In my experience it works better in more constrained and repetitive domains. For example: it is better doing a Ruby on Rails website than a ASP.NET web service. Particularly, with data structures it is garbage: it nevers understands the constrains that justify writing a new one instead of relying on the ones from the standard library. And finally, it is incapable of understanding changes of mind. It will go back to stufd already discarted or replaced. The worst part of all is that it insists in introducing its own "contributions". For example, recently I have been doing some work on ML and I wanted to see the effect of some ablations. It destroyed my code to add again all the stuff I had removed on purpose. Overall, it provides small typing/search savings, but it cannot be trusted at all yet.
• birn559 1y ago
  As coding even today becomes more popular every year, a lot of people are junior level or work in an environment that only need junior level skills. I assume those benefit a lot from LLMs. Also a lot of marketing and it's cool to hype LLMs and I guess people like to see content about what it can do in YouTube and Instagram.
• 0x000xca0xfe 1y ago
  Nope, at least Claude code is very useful IME. Great for annoying ad-hoc programming where the objective is clear but I lack the time or motivation to do it. Example: After benchmarking an application on various combinations of OS/arch platforms, I wanted to turn the barely structured notes into nice graphs. Claude Code easily generated Python code that used a cursed regex parser to extract the raw data and turned it into a bunch of grouped bar charts via matplotlib. Took just a couple minutes and it didn't make a single mistake. Fantastic time saver! This is just an ad-hoc script. No need to extend or maintain it for eternity. It has served its purpose and if the input data will change, I can just throw it away and generate a new script. But if Claude hadn't done it, the graphs simply wouldn't exist. Update: Sorry, missed "writing self-contained throwaway pieces of code"... well for core development I too haven't really used it.
• curtisszmania 1y ago
  [dead]
• lc_eckert09 1y ago
  They’re fantastic for making some basic websites or dealing with boilerplate. I write compilers. Good luck getting an LLM to be helpful in that domain. It can be helpful to break down the docs for something like LLVM but not for writing passes or codegen etc
• dlevine 1y ago
  I think of LLMs as knowing a lot of things but as being relatively shallow in their knowledge. I find them to be super useful for things that I don't already know how to do, e.g. a framework or library that I'm not familiar with. It can then give me approximate code that I will probably need to modify a fair bit, but that I can use as the basis for my work. Having an LLM code a preliminary solution is often more efficient than jumping to reading the docs immediately. I do usually need to read the docs, but by the time I look at them, I already know what I need to look up and have a feasible approach in my head. If I know exactly how I would build something, an LLM isn't as useful, although I will admit that sometimes an LLM will come up with a clever algorithm that I wouldn't have thought up on my own. I think that, for everyone who has been an engineer for some time, we already have a way that we write code, and LLMs are a departure. I find that I need to force myself to try them for a variety of different tasks. Over time, I understand them better and become better at integrating them into my workflows.
• ants_everywhere 1y ago
  I'm getting real valuable work done with aider and Gemini. But it's not fun and it's not flow-state kind of work. Aider, in my humble opinion, has some issues with its loop. It sometimes works much better just to head over to AI studio and copy and paste. Sometimes it feels like aider tries to get things done as cheaply as possible, and the AI ends up making the same mistakes over again instead of asking for more information or more context. But it is a tool and I view it as my job to get used to the limitations and strengths of the tool. So I see my role as adapting to a useful but quirky coworker so I can focus my energy where I'm most useful. It may help that I'm a parent of intelligent and curious little kids. So I'm used to working with smart people who aren't very experienced and I'm patient about the long term payoff of working at their level.
• koonsolo 1y ago
  I think you and your friends actually agree on the application of LLM's, and I agree with that too. So the question really comes down to what kind of project you are developing: Get an MVP fast? LLM is great! Proof of Concept? LLM rules! Big/Complex project? LLM junior developer is not up to the task.
• tonyhart7 1y ago
  Yeah it is code monkey typewriter situation but I think this is solvable when context length goes way higher than current length
• kordlessagain 1y ago
  I'm using Claude Desktop to develop an agentic crawler system. My workflow is VSCode next to Claude Desktop. The tooling for Claude uses a bunch of MCP servers I wrote for Evolve: https://github.com/kordless/gnosis-evolve https://github.com/kordless/gnosis-evolve Core Development Capabilities: - File Discovery & Navigation: file_explorer with pattern matching and recursive search - Intelligent Code Search: search_in_file_fuzzy with similarity thresholds for finding relevant code sections - Advanced Code Editing: file_diff_writer with fuzzy matching that can handle code changes even after refactoring - Backups: backup and restores of any files at any state of change. - System Monitoring: Real-time log analysis and container management - Hot Deployment: docker_rebuild for instant container updates (Claude can do the rebuild) The Agentic Workflow: - Claude searches your codebase to understand current implementation - Uses fuzzy search to find related code patterns and dependencies - Makes intelligent edits using fuzzy replacement (handles formatting changes) - Monitors logs to verify changes work correctly - Restarts containers as needed for testing - Iterates based on log feedback - Error handling requires analyzing logs and adjusting parsing strategies - Performance tuning benefits from quick deploy-test-analyze cycles I've not had any issues with Claude being able to handle changes, even doing things like refactoring overly large HTML files with inline CSS and JS. Had it move all that to a more manageable layout and helped out by deleting large blocks when necessary. The fuzzy matching engine is the heart of the system. It uses several different strategies working in harmony. First, it tries exact matching, which is straightforward. If that fails, it normalizes whitespace by collapsing multiple spaces, removing trailing whitespace, and standardizing line breaks, then attempts to match again. This handles cases where code has been reformatted but remains functionally identical. When dealing with multi-line code blocks, the system gets particularly clever. It breaks both the search text and the target content into individual lines, then calculates similarity scores for each line pair. If the average similarity across all lines exceeds the threshold, it considers it a match. This allows it to find code blocks even when individual lines have been slightly modified, variable names changed, or indentation adjusted.
• nico 1y ago
  It really depends on the task and your preferences I’ve had great success using LLMs for things that I haven’t done in a while or never before. They allow me to build without getting too bogged down into the details of syntax Yes, they require constant attention, they are not fully independent or magical. And if you are building a project for the longer run, LLM-driven coding slows down a lot once the code base grows beyond just a couple of basic files (or when your files start getting to about 500-800+ lines) I’ve tried several agentic editors and tools, including cursor, they can def be helpful, but I’d rather just manually loop between ChatGPT (o4-high-mini for the most part) and the editor. I get a very quick and tight feedback loop in which I get plenty of control Git is essential for tracking changes, and tests are gold once you are at a certain size
• Phelinofist 1y ago
  The company I work for rolled out GitHub Copilot. It is quite underwhelming honestly. We have a lot of homegrown frameworks that it does not know about (I mean, to be fair, it actually cannot know about these). When I asked it to explain some piece of code it just repeated what was stated in the JavaDoc, nearly verbatim. I had a class that had obvious null issues and I asked it to fix it and it introduced quite a lot of unnecessary "// we have to check it is not null "-style comments.
• tshadley 1y ago
  100% agreed with your experience, AI provides little value to one's area of expertise (10+ years or more). It's the context length -- AI needs comparable training or inference-time cycles. But just wait for the next doubling of long task capacity (https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com...). Or the doubling after that. AI will get there.
• TimPC 1y ago
  LLMs give you superpowers if you work on a wide range of tasks and aren't a deep expert on all of them. They aren't that great if you know the area you are working in inside and out and never stray from it. For example, using an LLM to help you write a Dockerfile when you write Dockerfiles once a project and don't have a dedicated expert like a Deployment Engineer in your company is fantastic. Or using an LLM to get answers faster than google for syntax errors and other minor issues is nice. Even using LLM with careful prompting to discuss architecture tradeoffs and get its analysis (but make the final decision yourself) can be helpful. Generally, you want to be very careful about how you constrain the LLM through prompts to ensure you keep it on a very narrow path so that it doesn't do something stupid (as LLMs are prone to do), you also often have to iterate because LLMs will occasionally do things like hallucinate APIs that don't actually exist. But even with iteration it can often make you faster.
  • igor47 1y ago
    > For example, using an LLM to help you write a Dockerfile when you write Dockerfiles once a project and don't have a dedicated expert like a Deployment Engineer in your company is fantastic. This made me think of gell-mann amnesia: https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect Basically, when you understand the problem domain, you know the llm is generating crap that, as the OP says, you're unwilling to put your name on. But when it's in a domain you don't know, you forget about how crappy the LLM is normally and go with the vibe.
    • deleted 1y ago
      [deleted]
    • ChrisClark 1y ago
      As long as it's a simple enough problem and it works, I'm happy. The other day I needed to do something in powershell, which I probably use twice a year. But an LLM was just able to spit out what I needed for me. I find it's the best for that but once things get complicated, you're just arguing with the LLM, when you should have done it yourself.
• GaryNumanVevo 1y ago
  If you can't understand the code the LLM is writing, how can you expect to debug and troubleshoot it when your vibe coded feature gets deployed to production?
• harrall 1y ago
  I find Junie (from JetBrains / IntelliJ) great compared to other offerings. I have 10+ years experience as well. It writes JUnit tests with mocks, chooses to run them, and fixes the test or sometimes my (actually broken) code. It’s not helpful for 90% of my work, but like having a hammer, it’s good to have when you know that you have a nail.
• r00sty 1y ago
  I always spend more time fighting the bot and having to debug the code it gives for anything it gives to be useful.
• cies 1y ago
  I found they work for tasks that have been done 1000s of times already. But for creative solutions in super specialized environments (lot's of the works I do it just that) they cannot help me. I expect they soon will be able to help me with basic refactoring that needs to be performed across a code base. Luckily my code uses strong types: type safety quickly shows where the LLM was tripping/forgetting.
  • esafak 1y ago
    I find it helps to give it examples of what I want it to do in such cases.
• 65 1y ago
  I'd like to think the point of technology is to do the same thing but faster. AI tools can do things faster, but at lower quality. They can't do the _same_ thing faster. So AI is fine for specifically low quality, simple things. But for anything that requires any level of customization or novelty (which is most software), it's useless for me.
• taosx 1y ago
  I'm getting a lot of value in areas where I don't have much experience but most of the time I still write the final version. I'm not building commercial software and don't have a commercial job at the moment so I'm kind of struggling with credits otherwise I would probably blow 40-100$ dollars a day.
• pif 1y ago
  Who could have foreseen that laziness would be that demanding?
• irrational 1y ago
  At my current job, I need to write quite a bit of python. I been programming for enough decades that I can look at Python code and tell what it is doing, but creating it from scratch? No. But copilot “knows” Python and can write the code that I can then read and tweak. Maybe someone who actually learned Python would write the code differently, but so far it works very well.
  • dowager_dan99 1y ago
    I did this last week and the Python code it created technically "worked" but felt on par with your typical Excel VBA macro. Not something you'd want to maintain or show anyone else
• decGetAc 1y ago
  Maybe I'm using these models incorrectly but I just don't ask it to do a lot at once and find it extremely useful. Write tests for x in the style of this file to cover a, b, c. Help me find a bug here within this pseudo code that covers three classes and a few functions. Here's the behavior I see, here's what I think could be happening. I rarely give it access to all the code and usually give it small portions of code and ask for small things. I basically treat it as if I was reaching out to another senior developer in a large company or SO. They don't care to learn about all the details that don't matter, and want a good promoted question that's not wasting their time and that they can help with. Using it this way I absolutely see the benefits and I'd say an arbitrary 1.25x sounds right (and I'm an experienced engineer in my field). I'll just quietly keep using it this way and ignore the overwhelming hype on both sides (it's not a speed up camp and it's 100x camp. Imo both are wrong but the it's not a speed up camp make me question how they're using it the most
  • deleted 1y ago
    [deleted]
• th0ma5 1y ago
  All the comments here in the positive for LLMs including all the posts by experts can be summed up as "these are the loto numbers that worked for me."
• sawmurai 1y ago
  Today I tried to use Copilot to go through a code base and export unexported types, interfaces and unions (typescript). It had to also rename them to make sure they are unique in context of the package. Otherwise I would have used search&replace :) It started out promising, renaming the symbols according to my instructions. Slower than if I had done it myself, but not horrible slow. It skipped over a few renames so I did them manually. I had to tell it to continue every 2 minutes so I could not really do anything else in the meantime. I figured it’s quicker if I find the files in question (simple ripgrep search) and feed them to copilot. So I don’t have to wait for it to search all files. Cool, now it started to rename random other things and ignored the naming scheme I taught it before. It took quite some time to manually fix its mess. Maybe I should have just asked it to write a quick script to do the rename in an automated way instead :)
• AdrianB1 1y ago
  I tried using LLMs for 2 types of work: 1. Improve the code that I already have. Waste of time, it never works. This is not because my code is too good, but because it is SQL with complex context and I get more hallucinations than usable code, still the usable code is good for basic tasks and not for more. 2. Areas I rarely use and I don't maintain an expertise on. This is where it is good value, I get 80% of what I need in 20% of the time, I take it and complete the work. But this does not happen too often, so the overall value is not there yet. In a way is like RPA: it does something, not great, but it saves some time.
• zkry 1y ago
  Use cases like the ones you mentioned having are truly amazing. It's a shame that the AI hype machine has left us thinking of these use cases as practically nothing, leaving us disappointed. My belief is that true utility will make itself apparent and won't have to be forced. The usages of LLMs that provide immense utility have already spread across most the industry.
• Vaslo 1y ago
  For the mediocre coder like myself, it’s a game changer. Instead of finicking around with reading a weird dictionary schema in Python, I simply give a sample and boom I have code. Not to mention it shows me new ways of doing things that I was only familiar with and now does it by example, and then I have a better understand of using it. I’m talking about things like decorators and itertools. I think it’s great for coders of all levels, but jr programmers will get lost once the llm inevitably hallucinates and the expert will get gains, but not like those who are in the middle.
• abdullin 1y ago
  I heard an interesting story from an architect at a large software consultancy. They are using AI in their teams to manage legacy codebases in multiple languages. TLDR; it works for a codebase of 1M LoC. AI writes code a lot faster, completing tasks in days instead of sprints. Tasks can be parallelized. People code less, but they need to think more often. (1) Maintain clear and structured architecture documentation (README, DDD context/module descriptions files, AGENTS-MD). (2) Create detailed implementation plans first - explicitly mapping dependencies, tests, and potential challenges. (3) Treat the implementation plan as a single source of truth until execution finishes. Review it manually and with LLM-assistance to detect logical inconsistencies. Plan is easier to change, than a scattered diff. (4) In complex cases - instruct AI agents about relevant documents and contexts before starting tasks. (5) Approve implementation plans before allowing AI to write code (6) Results are better if code agent can launch automated full-stack tests and review their outputs in the process. The same works for me in smaller projects. Less ceremony is needed there.
• scotty79 1y ago
  Bit long but a lot of good insights by Theo.gg here: https://youtu.be/6koQP6-6mtY?si=PHtKGR7XImIKEvq6 https://youtu.be/6koQP6-6mtY?si=PHtKGR7XImIKEvq6 TLDR, to use those tools effectively you need to change yourself a bit but in a fairly good direction.
• aristofun 1y ago
  As google on steroids - yes. Next level helpful and offloads a lot of dumb and repetitive work from you. As a developer buddy - no. LLMs don't actually think and don't actually learn like people do. That part of the overinflated expectations is gonna hit hard some companies one day.
• platevoltage 1y ago
  Th best value I get out of AI is just having "someone" to bounce ideas off of, or to get quick answers from. I find that it's a HUGE help if I need to write code/understand code in a language that I don't work with every day. I might get it to write me a function that takes X as input and outputs Y. I might get it to set up boilerplate. I have talked to people that have said "I don't even write code anymore". That's definitely not me. Maybe it's a skill issue, but yeah, I'm kind of in the same boat too.
• _boffin_ 1y ago
  I just had an llm build the my side business’s website and full client portal. About 15k loc. It’s amazing. Better design in terms of UI / UX than I could have fathomed and so much more. There’s a lot of duplicated code that I’ll clean up, but the site functions and will be launched for clients to start using soon. For my day job, it’s also helping me build out the software at a faster pace than before and is an amazing rubber duck.
  • maxnevermind 1y ago
    Have you read through all 15K loc, What is your feeling about maintainability of it? Also as I understand one the main problem with LLM right now is trying to apply some surgical changes to large enough code base or adding some extra functionality without breaking/altering existing ones, have you faces issues with that? Another issue is security, I've heard some horror stories of non-tech people developing web solutions to later find them destroyed by hackers because they didn't know were to look to find holes in their design.
    • _boffin_ 1y ago
      I’m finishing up some hardening features and some additionally functionality right now. It’s “maintainable” par se, but I will be going through it and cleaning it up a lot. It’s a Django application. It likes to write out large single files. Already went through to break things up into similar views and functionality. And as for security., were they even caring about it? Were they asking good questions? Reading about things? Learning ways to secure things? If someone blindly follows what it gives you, something interesting will happen at some point. Use it as an enhancer.
• deleted 1y ago
  [deleted]
• ragdollk 1y ago
  Honestly I think what this boils down to is if you are concerned with good code or concerned with working code. They make working code. And if there is a bug or implementation that fails (which there usually is with the first shot) when asked, they work to resolve that. LLMs are very results driven, and they will do most anything to try and get the results requested. Even cheat. So, for coders that are concerned with quality code... of course you'll be unimpressed. For project managers who are concerned with a working piece of software, of course you will be impressed by that. You didn't have to ask a developer to create it. As far as impact on our culture goes... much like other disruptions in the past, they will win this war. They will win because they lower the barrier to entry to get working software. You ask what the value is - it's that.
• bird0861 1y ago
  I would put myself in the 100x camp but my start in AI (and programming) was symbolic approaches in lisp/scheme, then I put that away for about a decade and got into NLP and then got into question answering and search/information retrieval. I have read probably at least 20 papers on LLMs a week every week since December 2022. So I have some familiarity with the domain but I also would not exactly call myself an expert on LLMs. However, I would say I am confident I mostly understand what is required for good results. Getting the best possible results requires: - an LLM trained to have the "features" (in the ML/DL sense of the word) required to follow instructions to complete your task - an application that manages the context window of the LLM - strategically stuffing the context window with preferences/conventions, design information, documentation, examples, your git repo's repo map, and make sure you actually use rules and conventions files for projects. Do not assume the LLM will be able to retrieve or conjure up all of that for you. Treat it like a junior dev and lead it down the path you want it to take. It is true, there is a bit of micromanagement required but Aider makes that very very simple to do. Aider even makes it possible to scrape a docs page to markdown for use by the LLM. Hooking up an LLM to search is a great way to stuff the context window BTW, makes things much simpler. You can use the Perplexity API with Aider and quickly write project plans and fetch necessary docs quickly this way; just turn that into markdown files you'll load up later after you switch models to a proper code gen model. Assume that you may end up editing some code yourself, Aider makes launching your editor easy though. This mostly just works. For fun the first thing I did with Aider was to write a TUI chat interface for ollama and I had something I could post to github in about an hour or two. I really think Aider is the missing ingredient for most people. I have used it to generate documentation for projects I wrote by hand, I have used it to generate code (in one of my choice languages) for projects written in a language I didn't like. It's my new favorite video game. Join the Aider discord, read the docs, and start using it with Gemini and Sonnet. If you want local, there's more to that than what I'm willing to type in a comment here but long story short you also need to make a series of correct decisions to get good results from local but I do it on my RTX4090 just fine. I am not a contributor or author of Aider, I'm just a fanatical user and devotee to its way of doing things.
  • jz2dlimit 1y ago
    First responder who put themselves into the 100x category. Do you mind sharing what setup you use on your local RTX4090? Like what models you run and what kinds of prompt you feed it? I'm contemplating building an LLM rig to do some experimentation. It also sounds like you've read a lot of papers (20 per week since 2022 December would equate to 2000 by today). Do you have any favorites that really stood out to you?
• shio_desu 1y ago
  Honestly I've been getting a lot of use out of LLMs for coding, and have been adjusting my approach to LLM usage over the past year and a half. The current approach I take that has been fairly effective is to spend a lot of focus energy writing out exactly what I'm looking to implement, sometimes taking 30 or more minutes creating a specs doc / implementation plan, and passing it to a agent with an architect persona to review, generate a comprehensive, phased implementation document. I then review the document, iterate with it to make sure the plan works well, then send it off to do the work. I'm not yet a fan of Windsurf or Cursor, but honestly Roo Codes out of the box personas for architect, and orchestration to spin up focused subtasks works well for me. I am kinda treating it how I would a junior, to guide it there, give it enough information to do the work, and check it afterwards, ensuring it didn't do things like BS test coverage or write useless tests / code. It works pretty well for me, and I've been treating prompting these bots just as a skill I improve as I go along. Frankly it saves me a lot of time, I knocked out some work Friday afternoon that I'd estimate was probably 5pts of effort in 3 hours. I'll take the efficiency anyday as I've had less actual coding focus time in coding implementations than I used to in my career due to other responsibilities.
• paulcole 1y ago
  > but trying to get it to generate code that I am willing to maintain and "put my name on" took longer than writing the code would have Why don’t you consider that the AI will be the one maintaining it?
  • bagacrap 1y ago
    If that were possible we'd have already pointed LLMs at the Linux bug tracker (for example) and fixed all the open issues.
    • paulcole 1y ago
      Was all that code written by LLMs?
• AfterHIA 1y ago
  I like to think of the painter who's trained in classical design; they spend their time thinking about Baroque diagonals and golden ratios. They have a, "deep knowledge" of how paintings are classically organized. One day they encounter a strange machine that can produce any image they wish-- without considering the economic impact this would have on their craft he is often asked, "isn't this machine a marvel?" His reply is often, "it sheds no light for me on why certain ratios produce beautiful forms, it tells me nothing of the inspired mind of poets, nor of the nature of the Good." In Plato's Republic Socrates' compares the ability to produce a piece of furniture with the ability to produce the image of a cabient or so-forth with a small compact mirror; what is the difference if a deceivable crowd doesn't know the difference?
• tomyedwab 1y ago
  The comments on this thread are a perfect mixture of Group A, explaining how there is no value in AI tools, and if there was, where is the evidence? And Group B, who are getting value from the tools and have evidence of using them to deliver real software, but are being blasted by Group A as idiots who can't recognize bad code. Why so angry? I've been writing code for 36 years, so I don't take any of the criticism to heart. If you know what you are doing, you can ship production quality code written by an LLM. I'm not going to label it "made by an AI!" because the consumer doesn't care so long as it works and who needs the "never AI!" backlash anyway? But to the OP: your standards are too high. AI is like working with a bright intern, they are not going to do everything exactly the way that you prefer, but they are enthusiastic and can take direction. Choose your battles and focus on making the code maintainable in the long term, not perfect in the short term.
• noahbp 1y ago
  > I've tried: > - Cursor (can't remember which model, the default) > - Google's Jules > - OpenAI Codex with o4 Cursor's "default model" rarely works for me. You have to choose one of the models yourself. Sonnet 4, Gemini 2.5 Pro, and for tricky problems, o3. There is no public release of o4; you used o4-mini, a model with poorer performance than any of the frontier models (Sonnet 4, Gemini Pro 2.5, o3). Jules and Codex, if they're like Claude Code, do not work well with "Build me a Facebook clone"-type instructions. You have to break everything down and make your own tech stack decisions, even if you use these tools to do so. Yes they are not perfect and make regressions or forget to run linters or check their work with the compiler, but they do work extremely well if you learn to use them, just like any other tool. They are not yet magic that works without you having to put in any effort to learn them.
• GHanku 1y ago
  [dead]
• ishwarjha 1y ago
  A few days ago I discovered a few security issues in one of the software we have developed. I instantly decided to review the frontend and backend code with AI (used cursor and GitHub copilot) It reported a dozen more issues which otherwise would have taken a few weeks to find. We asked AI to generate code that will help the security providing rules informing about technology stack, coding guidelines, project structure and product description. We got good recommendations, but couldn't implement the suggestions straightforward. However, we took the advices and hand-coded the suggestions at all code files. The entire exercise took a week for fairly large project. As per my tech lead, it would have taken minimum 2 months. Soniy works.
• whittlelabs 1y ago
  [dead]
• the__alchemist 1y ago
  I speculate, from this, that you are giving the LLMs too big of a task. Not quite vibe coding, but along that path. > "You've refactored most of the file but forgot a single function"). It would take many many iterations on trivial issues, and because these iterations are slow that just meant I had to context switch a lot, which is also exhausting. Try prompts like this: "Given these data structures: (Code of structs and enums), please implement X algorithm in this function signature. (Post exact function signature)." Or: "This code is repetitive. Please write a macro to simplify the syntax. Here's what calling it should look like (Show macro use syntax)" Or: "I get X error on this function call. Please correct it." Or: "I'm copying these tables into native data structures/match arms etc. Here's the full table. Here's the first few lines of the native structures: ...""
• spiderxxxx 1y ago
  I've been programming in python for over 20 years. An LLM creates code that sometimes works, but it definitely doesn't meet my standards, and there's no way I'd use the code since I couldn't support it. People who have less experience in Python might take that working code and just support that with their LLM, still having no clue what it does or why it works. That's probably fine for MVP but it won't stand up in the real world where you have to support the code or refactor it for your environment. I tried to use an LLM to write a simple curses app - something where there's a lot of code out there, but most of the code is bad, and of course it doesn't work and there's lots of quirks. I then asked it to see if there are libraries out there that are better than curses, it gave me 'textual' which at first seemed like an HTML library, but is actually a replacement for curses. It did work, and I had some working code at the end, but I had to work around platform inconsistencies and deal with the LLM including outdated info like inline styles that are unsupported in the current version of the library. That said, I don't quite understand the code that it produced, I know it works and it looks nice, but I need to write the code myself if I want a deeper understanding of the library, so that I can support it. You won't get that from asking an LLM to write your code for you, but from you using what you learn. It's like any language learning. You could use google translate to translate what you want, and it may seem correct at first glance, but ultimately won't convey what you want, with all the nuance you want, if you just learned the language yourself.
• Simpletout 1y ago
  If you ever did a declarative programing, then you know what's LLM brought in it. If before you should say in a very formal way what you want to do, now you can do the same writing your prompt if a very relaxed form. You declarations are fed from thousand implementation sources now, than just few ones before. Otherwise I do not see much difference. I can also warn also that using a very formal language is much more concise, and you get a result in much shorter time. However, most people try to avoid any learning.
• GBintz 1y ago
  I don't understand the hesitation with existing large codebases. Maybe everything I work on is just easy compared to yall. It has been unimaginably helpful in getting me up to speed in large existing codebases. First thing to do in a codebase is to tell it to "analyze the entire codebase and generate md docs in a /llmdocs directory". Do this manually in a loop a few times and try a few different models. They'll build on each other's output. Chunk embed and index those rather than the code files themselves. Use those for context. Get full code files through tool calls when needed