6 ms·

I literally had a developer of an open source package I’m working with tell me “yeah that’s a known problem, I gave up on trying to fix it. You should just ask

I literally had a developer of an open source package I’m working with tell me “yeah that’s a known problem, I gave up on trying to fix it. You should just ask ChatGPT to fix it, I bet it will immediately know the answer.” Annoying response of course. But I’d never used an LLM to debug before, so I figured I’d give it a try. First: it regurgitated a bunch of documentation and basic debugging tips, which might have actually been helpful if I had just encountered this problem and had put no thought into debugging it yet. In reality, I had already spent hours on the problem. So not helpful Second: I provided some further info on environment variables I thought might be the problem. It latched on to that. “Yes that’s your problem! These environment variables are (causing the problem) because (reasons that don’t make sense). Delete them and that should fix things.” I deleted them. It changed nothing. Third: It hallucinated a magic numpy function that would solve my problem. I informed it this function did not exist, and it wrote me a flowery apology. Clearly AI coding works great for some people, but this was purely an infuriating distraction. Not only did it not solve my problem, it wasted my time and energy, and threw tons of useless and irrelevant information at me. Bad experience.

ajkdhcb2 2y ago
My experiences have all been like this too. I am puzzled by how some people say it works for them
- simonw 2y ago
  I wrote this article precisely for people who are having trouble getting good results out of LLMs for coding: https://simonwillison.net/2025/Mar/11/using-llms-for-code/ https://simonwillison.net/2025/Mar/11/using-llms-for-code/
  - codegangsta 2y ago
    Just wanted to chime in and say how appreciative I’ve been about all your replies here, and overall content on AI. Your takes are super reasonable and well thought out.
  - mplanchard 2y ago
    I’ve followed your blog for a while, and I have been meaning to unsubscribe because the deluge of AI content is not what I’m looking for. I read the linked article when it was posted, and I suspect a few things that are skewing your own view of the general applicability of LLMs for programming. One, your projects are small enough that you can reasonably provide enough context for the language model to be useful. Two, you’re using the most common languages in the training data. Three, because of those factors, you’re willing to put much more work into learning how to use it effectively, since it can actually produce useful content for you. I think it’s great that it’s a technology you’re passionate about and that it’s useful for you, but my experience is that in the context of working in a large systems codebase with years of history, it’s just not that useful. And that’s okay, it doesn’t have to be all things to all people. But it’s not fair to say that we’re just holding it wrong.
    - jjani 2y ago
      > One, your projects are small enough that you can reasonably provide enough context for the language model to be useful. Two, you’re using the most common languages in the training data. Three, because of those factors, you’re willing to put much more work into learning how to use it effectively, since it can actually produce useful content for you. Take a look at the 2024 StackOverflow survey. 70% of professional developer respondents had only done extensive work over the last year in one of: JS 64.6% SQL 54.1% JTML/CSS 52.9% PY 46.9% TS 43.4% Bash/Shell 34.2% Java 30% LLMs are of course very strong in all of these. 70% of developers only code in languages LLMs are very strong at. If anything, for the developer population at large, this number is even higher than 70%. The survey respondents are overwhelmingly American (where the dev landscape is more diverse), and self-select to those who use niche stuff and want to let the world know. Similar argument can be made for median codebase size, in terms of LOC written every year. A few days ago he also gave Gemini Pro 2.5 a whole codebase (at ~300k tokens) and it performed well. Even in huge codebases, if any kind of separation of concerns is involved, that's enough to give all context relevant to the part of the code you're working on. [1] [1] https://simonwillison.net/2025/Mar/25/gemini/ https://simonwillison.net/2025/Mar/25/gemini/
      - 2y ago
  - Workaccount2 2y ago
    LLMs need to stay bad. What is going to happen if we have another few GPT-3.5 to Gemini 2.5 sized steps? You're telling people who need to keep the juicy SWE gravy train running for another 20 years to recognize that the threat is indeed very real. The writing is on the wall and no one here (here on HN especially) is going to celebrate those pointing to it.
    - bluefirebrand 2y ago
      I don't think people really realize the danger of mass unemployment Go look up what happens in history when tons of people are unemployed at the same time with no hope of getting work. What happens when the unemployed masses become desperate? Naw I'm sure it will be fine, this time will be different
sdenton4 2y ago
This morning I was using an LLM to develop some SQL queries against a database it had never seen before. I gave it a starting point, and outlined what I wanted to do. It proposed a solution, which was a bit wrong, mostly because I hadn't given it the full schema to work with. Small nudges and corrections, and we had something that worked. From there, I iterated and added more features to the outputs. At many points, the code would have an error; to deal with this, I just supply the error message, as-is to the LLM, and it proposes a fix. Sometimes the fix works, and sometimes I have to intervene to push the fix in the right direction. It's OK - the whole process took a couple hours, and probably would have been a whole day if I were doing it on my own, since I usually only need to remember anything about SQL syntax once every year or three. A key part of the workflow, imo, was that we were working in the medium of the actual code. If the code is broken, we get an error, and can iterate. Asking for opinions doesn't really help...
- simonw 2y ago
  I often wonder if people who report that LLMs are useless for code haven't cracked the fact that you need to to have a conversation with it - expecting a perfect result after your first prompt is setting it up for failure, the real test is if you can get to a working solution after iterating with it for a few rounds.
  - bluefirebrand 2y ago
    Why would I do this, when I can just write it from scratch in less time than it takes you to have this conversation with the LLM?
    - simonw 2y ago
      Because once you get good at using LLMs you can write it with 5 rounds with an LLM in way less time than it would have taken you to type out the whole thing yourself, even if you got it exactly right first time coding it by hand.
      - 2y ago
dale_glass 2y ago
On the other hand, when it works it's darn near magic. I spent like a week trying to figure out why a livecd image I was working on wasn't initializing devices correctly. Read the docs, read source code, tried strace, looked at the logs, found forums of people with the same problem but no solution, you know the drill. In desperation I asked ChatGPT. ChatGPT said "Use udevadm trigger". I did. Things started working. For some problems it's just very hard to express them in a googleable form, especially if you're doing something weird almost nobody else does.
- bluefirebrand 2y ago
  Honestly this says more about how bad Google has become than about how good GPT is
- PlunderBunny 2y ago
  > For some problems it's just very hard to express them in a googleable form I had an issue where my Mac would report that my tethered iPhone's batteries were running low when the battery was in fact fine. I had tried googling an answer, and found many similar-but-not-quite-the-same questions and answers. None of the suggestions fixed the issue. I then asked the 'MacOS Guru' model for chatGPT my question, and one of the suggestions worked. I feel like I learned something about chatGPT vs Google from this - the ability of an LLM to match my 'plain English question without a precise match for the technical terms' is obviously superior to a search engine. I think google etc try synonyms for words in the query, but to me it's clear this isn't enough.
- lucyjojo 2y ago
  i started (re)using AI recently. it/i mostly failed until i decided on a rule. if it's "dumb and annoying" i ask the AI, else i do it myself. since that AI has been saving me a lot of time on dumb and annoying things. also a few models are pretty good for basic physics/modeling stuff (get basic formulas, fetching constants, do some calculations). these are also pretty useful. i recently used it for ventilation/co2 related stuff in my room and the calculations matched observed values pretty well, then it pumped me a broken desmos syntax formula, and i fixed that by hand and we were good to go! --- (dumb and annoying thing -> time-consuming to generate with no "deep thought" involved, easy to check)
mystified5016 2y ago
The biggest thing I've found is that if you give any hint at all as to what you think the problem is, the LLM will immediately and enthusiastically agree, no matter how wildly incorrect your suggestion is. If I give it all my information and add "I think the problem might be X, but I'm not sure", the LLM always agrees that the problem is X and will reinterpret everything else I've said to 'prove' me right. Then the conversation is forever poisoned and I have to restart an entirely new chat from scratch. 98% of the utility I've found in LLMs is getting it to generate something nearly correct, but which contains just enough information for me to go and Google the actual answer. Not a single one of the LLMs I've tried have been any practical use editing or debugging code. All I've ever managed is to get it to point me towards a real solution, none of them have been able to actually independently solve any kind of problem without spending the same amount of time and effort to do it myself.
- jjani 2y ago
  > The biggest thing I've found is that if you give any hint at all as to what you think the problem is, the LLM will immediately and enthusiastically agree, no matter how wildly incorrect your suggestion is. I'm seeing this sentiment a lot in these comments, and frankly it shows that very few here have actually gone and tried the variety of models available. Which is totally fine, I'm sure they have better stuff to do, you don't have to keep up with this week's hottest release. To be concrete - the symptom you're talking about is very typical of Claude (or earlier GPT models). o3-mini is much less likely to do this. Secondly, prompting absolutely goes a huge way to avoiding that issue. Like you're saying - if you're not sure, don't give hints, keep it open-minded. Or validate the hint before starting, in a separate conversation.
  - turtletontine 2y ago
    I literally got this problem earlier today on ChatGPT, which claims to be based on o4-mini. So no, does not sound like it's just a problem with Claude or older GPTs. And on "prompting", I think this is a point of friction between LLM boosters and haters. To the uninitiated, most AI hype sounds like "it's amazing magic!! just ask it to do whatever you want and it works!!" When they try it and it's less than magic, hearing "you're prompting it wrong" seems more like a circular justification of a cult follower than advice. I understand that it's not - that, genuinely, it takes some experience to learn how to "prompt good" and use LLMs effectively. I buy that. But some more specific advice would be helpful. Cause as is, it sounds more like "LLMs are magic!! didn't work for you? oh, you must be holding it wrong, cause I know they infallibly work magic".
pikelet 2y ago
This is exactly my experience, every time! If I offer it the slightest bit of context it will say 'Ah! I understand now! Yes, that is your problem, …' and proceed to spit out some non-existent function, sometimes the same one it has just suggested a few prompts ago which we already decided doesn't exist/work. And it just goes on and on giving me 'solutions' until I finally realise it doesn't have the answer (which it will never admit unless you specifically ask it to – forever looking to please) and give up.
danielmarkbruce 2y ago
Exactly which model did you use? You talk about LLMs as though they are all the same. Alien 1: I gave Jeff Dean a giant complex system to build, he crushed it! Humans are so smart. Alien 2: I gave a random human a simple programming problem and he just stared at me like an idiot. Humans suck.

ajkdhcb2 2y ago

My experiences have all been like this too. I am puzzled by how some people say it works for them

simonw 2y ago
I wrote this article precisely for people who are having trouble getting good results out of LLMs for coding: https://simonwillison.net/2025/Mar/11/using-llms-for-code/ https://simonwillison.net/2025/Mar/11/using-llms-for-code/
- codegangsta 2y ago
  Just wanted to chime in and say how appreciative I’ve been about all your replies here, and overall content on AI. Your takes are super reasonable and well thought out.
- mplanchard 2y ago
  I’ve followed your blog for a while, and I have been meaning to unsubscribe because the deluge of AI content is not what I’m looking for. I read the linked article when it was posted, and I suspect a few things that are skewing your own view of the general applicability of LLMs for programming. One, your projects are small enough that you can reasonably provide enough context for the language model to be useful. Two, you’re using the most common languages in the training data. Three, because of those factors, you’re willing to put much more work into learning how to use it effectively, since it can actually produce useful content for you. I think it’s great that it’s a technology you’re passionate about and that it’s useful for you, but my experience is that in the context of working in a large systems codebase with years of history, it’s just not that useful. And that’s okay, it doesn’t have to be all things to all people. But it’s not fair to say that we’re just holding it wrong.
  - jjani 2y ago
    > One, your projects are small enough that you can reasonably provide enough context for the language model to be useful. Two, you’re using the most common languages in the training data. Three, because of those factors, you’re willing to put much more work into learning how to use it effectively, since it can actually produce useful content for you. Take a look at the 2024 StackOverflow survey. 70% of professional developer respondents had only done extensive work over the last year in one of: JS 64.6% SQL 54.1% JTML/CSS 52.9% PY 46.9% TS 43.4% Bash/Shell 34.2% Java 30% LLMs are of course very strong in all of these. 70% of developers only code in languages LLMs are very strong at. If anything, for the developer population at large, this number is even higher than 70%. The survey respondents are overwhelmingly American (where the dev landscape is more diverse), and self-select to those who use niche stuff and want to let the world know. Similar argument can be made for median codebase size, in terms of LOC written every year. A few days ago he also gave Gemini Pro 2.5 a whole codebase (at ~300k tokens) and it performed well. Even in huge codebases, if any kind of separation of concerns is involved, that's enough to give all context relevant to the part of the code you're working on. [1] [1] https://simonwillison.net/2025/Mar/25/gemini/ https://simonwillison.net/2025/Mar/25/gemini/
    - 2y ago
- Workaccount2 2y ago
  LLMs need to stay bad. What is going to happen if we have another few GPT-3.5 to Gemini 2.5 sized steps? You're telling people who need to keep the juicy SWE gravy train running for another 20 years to recognize that the threat is indeed very real. The writing is on the wall and no one here (here on HN especially) is going to celebrate those pointing to it.
  - bluefirebrand 2y ago
    I don't think people really realize the danger of mass unemployment Go look up what happens in history when tons of people are unemployed at the same time with no hope of getting work. What happens when the unemployed masses become desperate? Naw I'm sure it will be fine, this time will be different

sdenton4 2y ago

This morning I was using an LLM to develop some SQL queries against a database it had never seen before. I gave it a starting point, and outlined what I wanted to do. It proposed a solution, which was a bit wrong, mostly because I hadn't given it the full schema to work with. Small nudges and corrections, and we had something that worked. From there, I iterated and added more features to the outputs. At many points, the code would have an error; to deal with this, I just supply the error message, as-is to the LLM, and it proposes a fix. Sometimes the fix works, and sometimes I have to intervene to push the fix in the right direction. It's OK - the whole process took a couple hours, and probably would have been a whole day if I were doing it on my own, since I usually only need to remember anything about SQL syntax once every year or three. A key part of the workflow, imo, was that we were working in the medium of the actual code. If the code is broken, we get an error, and can iterate. Asking for opinions doesn't really help...

simonw 2y ago
I often wonder if people who report that LLMs are useless for code haven't cracked the fact that you need to to have a conversation with it - expecting a perfect result after your first prompt is setting it up for failure, the real test is if you can get to a working solution after iterating with it for a few rounds.
- bluefirebrand 2y ago
  Why would I do this, when I can just write it from scratch in less time than it takes you to have this conversation with the LLM?
  - simonw 2y ago
    Because once you get good at using LLMs you can write it with 5 rounds with an LLM in way less time than it would have taken you to type out the whole thing yourself, even if you got it exactly right first time coding it by hand.
    - 2y ago

dale_glass 2y ago

On the other hand, when it works it's darn near magic. I spent like a week trying to figure out why a livecd image I was working on wasn't initializing devices correctly. Read the docs, read source code, tried strace, looked at the logs, found forums of people with the same problem but no solution, you know the drill. In desperation I asked ChatGPT. ChatGPT said "Use udevadm trigger". I did. Things started working. For some problems it's just very hard to express them in a googleable form, especially if you're doing something weird almost nobody else does.

bluefirebrand 2y ago
Honestly this says more about how bad Google has become than about how good GPT is
PlunderBunny 2y ago
> For some problems it's just very hard to express them in a googleable form I had an issue where my Mac would report that my tethered iPhone's batteries were running low when the battery was in fact fine. I had tried googling an answer, and found many similar-but-not-quite-the-same questions and answers. None of the suggestions fixed the issue. I then asked the 'MacOS Guru' model for chatGPT my question, and one of the suggestions worked. I feel like I learned something about chatGPT vs Google from this - the ability of an LLM to match my 'plain English question without a precise match for the technical terms' is obviously superior to a search engine. I think google etc try synonyms for words in the query, but to me it's clear this isn't enough.
lucyjojo 2y ago
i started (re)using AI recently. it/i mostly failed until i decided on a rule. if it's "dumb and annoying" i ask the AI, else i do it myself. since that AI has been saving me a lot of time on dumb and annoying things. also a few models are pretty good for basic physics/modeling stuff (get basic formulas, fetching constants, do some calculations). these are also pretty useful. i recently used it for ventilation/co2 related stuff in my room and the calculations matched observed values pretty well, then it pumped me a broken desmos syntax formula, and i fixed that by hand and we were good to go! --- (dumb and annoying thing -> time-consuming to generate with no "deep thought" involved, easy to check)

mystified5016 2y ago

The biggest thing I've found is that if you give any hint at all as to what you think the problem is, the LLM will immediately and enthusiastically agree, no matter how wildly incorrect your suggestion is. If I give it all my information and add "I think the problem might be X, but I'm not sure", the LLM always agrees that the problem is X and will reinterpret everything else I've said to 'prove' me right. Then the conversation is forever poisoned and I have to restart an entirely new chat from scratch. 98% of the utility I've found in LLMs is getting it to generate something nearly correct, but which contains just enough information for me to go and Google the actual answer. Not a single one of the LLMs I've tried have been any practical use editing or debugging code. All I've ever managed is to get it to point me towards a real solution, none of them have been able to actually independently solve any kind of problem without spending the same amount of time and effort to do it myself.

jjani 2y ago
> The biggest thing I've found is that if you give any hint at all as to what you think the problem is, the LLM will immediately and enthusiastically agree, no matter how wildly incorrect your suggestion is. I'm seeing this sentiment a lot in these comments, and frankly it shows that very few here have actually gone and tried the variety of models available. Which is totally fine, I'm sure they have better stuff to do, you don't have to keep up with this week's hottest release. To be concrete - the symptom you're talking about is very typical of Claude (or earlier GPT models). o3-mini is much less likely to do this. Secondly, prompting absolutely goes a huge way to avoiding that issue. Like you're saying - if you're not sure, don't give hints, keep it open-minded. Or validate the hint before starting, in a separate conversation.
- turtletontine 2y ago
  I literally got this problem earlier today on ChatGPT, which claims to be based on o4-mini. So no, does not sound like it's just a problem with Claude or older GPTs. And on "prompting", I think this is a point of friction between LLM boosters and haters. To the uninitiated, most AI hype sounds like "it's amazing magic!! just ask it to do whatever you want and it works!!" When they try it and it's less than magic, hearing "you're prompting it wrong" seems more like a circular justification of a cult follower than advice. I understand that it's not - that, genuinely, it takes some experience to learn how to "prompt good" and use LLMs effectively. I buy that. But some more specific advice would be helpful. Cause as is, it sounds more like "LLMs are magic!! didn't work for you? oh, you must be holding it wrong, cause I know they infallibly work magic".

pikelet 2y ago

This is exactly my experience, every time! If I offer it the slightest bit of context it will say 'Ah! I understand now! Yes, that is your problem, …' and proceed to spit out some non-existent function, sometimes the same one it has just suggested a few prompts ago which we already decided doesn't exist/work. And it just goes on and on giving me 'solutions' until I finally realise it doesn't have the answer (which it will never admit unless you specifically ask it to – forever looking to please) and give up.

danielmarkbruce 2y ago

Exactly which model did you use? You talk about LLMs as though they are all the same. Alien 1: I gave Jeff Dean a giant complex system to build, he crushed it! Humans are so smart. Alien 2: I gave a random human a simple programming problem and he just stared at me like an idiot. Humans suck.