6 ms·

This comment is ~3 years late. Every model since gpt3 has had the entirety of available code in their training data. That's not a gotcha anymore. We went from

This comment is ~3 years late. Every model since gpt3 has had the entirety of available code in their training data. That's not a gotcha anymore. We went from chatgpt's "oh, look, it looks like python code but everything is wrong" to "here's a full stack boilerplate app that does what you asked and works in 0-shot" inside 2 years. That's the kicker. And the sauce isn't just in the training set, models now do post-training and RL and a bunch of other stuff to get to where we are. Not to mention the insane abilities with extended context (first models were 2/4k max), agentic stuff, and so on. These kinds of comments are really missing the point.

haar 1y ago
I've had little success with Agentic coding, and what success I have had has been paired with hours of frustration, where I'd have been better off doing it myself for anything but the most basic tasks. Even then, when you start to build up complexity within a codebase - the results have often been worse than "I'll start generating it all from scratch again, and include this as an addition to the initial longtail specification prompt as well", and even then... it's been a crapshoot. I _want_ to like it. The times where it initially "just worked" felt magical and inspired me with the possibilities. That's what prompted me to get more engaged and use it more. The reality of doing so is just frustrating and wishing things _actually worked_ anywhere close to expectations.
- aschobel 1y ago
  Bingo, it's magical but the learning curve is very very steep. The METR study on open-source productivity alluded to this a bit. I am definitely at a point where I am more productive with it, but it took a bunch of effort.
  - devmor 1y ago
    The subjects in the study you are referencing also believed that they were more productive with it. What metrics do you have to convince yourself you aren't under the same illusionary bias they were?
    - simonw 1y ago
      Yesterday I used ffmpeg to extract the frame at the 13 second mark of a video out as a JPEG. If I didn't have an LLM to figure that out for me I wouldn't have done it at all.
      - devmor 1y ago
        You wouldn't have just typed "extract frame at timestamp as jpeg ffmpeg" into Google and used the StackExchange result that comes up first that gives you a command to do exactly that?
        simonw 1y ago
        Before LLMs made ffmpeg no-longer-frustrating-to-use I genuinely didn't know that ffmpeg COULD do things like that.
        devmor 1y ago
        I'm not really sure what you're saying an LLM did in this case. Inspired a lost sense of curiosity?
        Philpax 1y ago
        Translated a vague natural language query ("cli, extract frame 13s into video") into something immediately actionable with specific examples and explanations, surfacing information that I would otherwise not know how to search for. That's what I've done with my ffmpeg LLM queries, anyway - can't speak for simonw!
        wizzwizz4 1y ago
        DuckDuckGo search results for "cli, extract frame 13s into video" (no quotes): • https://stackoverflow.com/questions/10957412/fastest-way-to-extract-frames-using-ffmpeg https://stackoverflow.com/questions/10957412/fastest-way-to-... • https://superuser.com/questions/984850/linux-how-to-extract-frames-from-a-video-lossless https://superuser.com/questions/984850/linux-how-to-extract-... • https://www.aleksandrhovhannisyan.com/notes/video-cli-cheat-sheet/ https://www.aleksandrhovhannisyan.com/notes/video-cli-cheat-... • https://www.baeldung.com/linux/ffmpeg-extract-video-frames https://www.baeldung.com/linux/ffmpeg-extract-video-frames • https://ottverse.com/extract-frames-using-ffmpeg-a-comprehensive-guide/ https://ottverse.com/extract-frames-using-ffmpeg-a-comprehen... Search engines have been able to translate "vague natural language queries" into search results for a decade, now. This pre-existing infrastructure accounts for the vast majority of ChatGPT's apparent ability to find answers.
  - haar 1y ago
    Apologies if I was unclear. The more I've used it, the more I've disliked how poor the results it's produced, and the more I've realised I would have been better served by doing it myself and following a methodical path for things that I didn't have experience with. It's easier to step through a problem as I'm learning and making small changes than an LLM going "It's done, and production ready!" where it just straight up doesn't work for 101 different tiny reasons.
    - airspresso 1y ago
      My preferred approach to avoid that outcome is to divide & conquer the problem. Ask the LLM to implement each small bit in the order you'd implement it yourself given what you know about the codebase.
MyOutfitIsVague 1y ago
I don't think they are missing the point, because they're pointing out that the tools are still the most useful for patterns that are extremely widely known and repeated. I use Gemini 2.5 Pro every day for coding, and even that one still falls over on tasks that aren't well known to it (which is why I break the problem down into small parts that I know it'll be able to handle properly). It's kind of funny, because sometimes these tools are magical and incredible, and sometimes they are extremely stupid in obvious ways. Yes, these are impressive, and especially so for local models that you can run yourself, but there is a gap between "absolutely magical" and "pretty cool, but needs heavy guiding" depending on how heavily the ground you're treading has been walked upon. For a heavily explored space, it's like being impressed that you're 2.5 year old M2 with 64 GB RAM can extract some source code from a zip file. It's worth being impressed and excited about the space and the pace of improvement, but it's also worth stepping back and thinking rationally about the specific benchmark at hand.
- NitpickLawyer 1y ago
  > because they're pointing out that the tools are still the most useful for patterns that are extremely widely known and repeated I agree with you, but your take is much more nuanced than what the GP comment said! These models don't simply regurgitate the training set. That was my point with gpt3. The models have advanced from that, and can now "generalise" over the context in ways they could not do ~3 years ago. We are now at a point where you can write a detailed spec (10-20k tokens) for an unseen scripting language, and have SotA models a) write a parser and b) start writing scripts for you in that language, even though it never saw that particular scripting language anywhere in its training set. Try it. You'll be surprised.
jayd16 1y ago
I think you're missing the point. Showing off moderately complicated results that are actually not indicative of performance because they are sniped by the training data turns this from a cool demo to a parlor trick. Stating that, aha, jokes on you, that's the status quo, is an even bigger indictment.
jan_Sate 1y ago
Not exactly. The real utility value of LLM for programming is to come up with something new. For Space Invaders, instead of using LLM for that, I might as well just manually search for the code online and use that. To show that LLM actually can provide value for one-shot programming, you need to find a problem that there's no fully working sample code available online. I'm not trying to say that LLM couldn't to that. But just because LLM can come up with a perfectly-working Space Invaders doesn't mean that it could do that.
- devmor 1y ago
  > The real utility value of LLM for programming is to come up with something new. That's the goal for these projects anyways. I don't know that its true or feasible. I find the RAG models much more interesting myself, I see the technology as having far more value in search than generation. Rather than write some markov-chain reminiscent frankenstein function when I ask it how to solve a problem, I would like to see it direct me to the original sources it would use to build those tokens, so that I can see their implementations in context and use my judgement.
  - simonw 1y ago
    "I would like to see it direct me to the original sources it would use to build those tokens" Sadly that's not feasible with transformer-based LLMs: those original sources are long gone by the time you actually get to use the model, scrambled a billion times into a trained set of weights. One thing that helped me understand this is understanding that every single token output by an LLM is the result of a calculation that considers all X billion parameters that are baked into that model (or a subset of that in the case of MoE models, but it's still billions of floating point calculations for every token.) You can get an imitation of that if you tell the model "use your search tool and find example code for this problem and build new code based on that", but that's a pretty unconventional way to use a model. A key component of the value of these things is that they can spit out completely new code based on the statistical patterns they learned through training.
    - 1y ago
Aurornis 1y ago
> These kinds of comments are really missing the point. I disagree. In my experience, asking coding tools to produce something similar to all of the tutorials and example code out there works amazingly well. Asking them to produce novel output that doesn’t match the training set produces very different results. When I tried multiple coding agents for a somewhat unique task recently they all struggled, continuously trying to pull the solution back to the standard examples. It felt like an endless loop of the models grinding through a solution and then spitting out something that matched common examples, after which I had to remind them of the unique properties of the task and they started all over again, eventually arriving back in the same spot. It shows the reality of working with LLMs and it’s an important consideration.
AlexeyBrin 1y ago
You are reading too much into my comment. My point was that the test (a Space Invaders clone) used to asses the model is irrelevant for some time now. I could have gotten a similar result with Mistral Small a few months ago.
stolencode 1y ago
It's amazing that none of you even try to falsify you claims anymore. You can literally just put some of the code in a search engine and find the prior art example: https://www.web-leb.com/en/code/2108 https://www.web-leb.com/en/code/2108 Your "AI tools" are just "copyright whitewashing machines." These kinds of comments are really ignoring reality.

haar 1y ago

I've had little success with Agentic coding, and what success I have had has been paired with hours of frustration, where I'd have been better off doing it myself for anything but the most basic tasks. Even then, when you start to build up complexity within a codebase - the results have often been worse than "I'll start generating it all from scratch again, and include this as an addition to the initial longtail specification prompt as well", and even then... it's been a crapshoot. I _want_ to like it. The times where it initially "just worked" felt magical and inspired me with the possibilities. That's what prompted me to get more engaged and use it more. The reality of doing so is just frustrating and wishing things _actually worked_ anywhere close to expectations.

aschobel 1y ago
Bingo, it's magical but the learning curve is very very steep. The METR study on open-source productivity alluded to this a bit. I am definitely at a point where I am more productive with it, but it took a bunch of effort.
- devmor 1y ago
  The subjects in the study you are referencing also believed that they were more productive with it. What metrics do you have to convince yourself you aren't under the same illusionary bias they were?
  - simonw 1y ago
    Yesterday I used ffmpeg to extract the frame at the 13 second mark of a video out as a JPEG. If I didn't have an LLM to figure that out for me I wouldn't have done it at all.
    - devmor 1y ago
      You wouldn't have just typed "extract frame at timestamp as jpeg ffmpeg" into Google and used the StackExchange result that comes up first that gives you a command to do exactly that?
      - simonw 1y ago
        Before LLMs made ffmpeg no-longer-frustrating-to-use I genuinely didn't know that ffmpeg COULD do things like that.
        devmor 1y ago
        I'm not really sure what you're saying an LLM did in this case. Inspired a lost sense of curiosity?
        Philpax 1y ago
        Translated a vague natural language query ("cli, extract frame 13s into video") into something immediately actionable with specific examples and explanations, surfacing information that I would otherwise not know how to search for. That's what I've done with my ffmpeg LLM queries, anyway - can't speak for simonw!
        wizzwizz4 1y ago
        DuckDuckGo search results for "cli, extract frame 13s into video" (no quotes): • https://stackoverflow.com/questions/10957412/fastest-way-to-extract-frames-using-ffmpeg https://stackoverflow.com/questions/10957412/fastest-way-to-... • https://superuser.com/questions/984850/linux-how-to-extract-frames-from-a-video-lossless https://superuser.com/questions/984850/linux-how-to-extract-... • https://www.aleksandrhovhannisyan.com/notes/video-cli-cheat-sheet/ https://www.aleksandrhovhannisyan.com/notes/video-cli-cheat-... • https://www.baeldung.com/linux/ffmpeg-extract-video-frames https://www.baeldung.com/linux/ffmpeg-extract-video-frames • https://ottverse.com/extract-frames-using-ffmpeg-a-comprehensive-guide/ https://ottverse.com/extract-frames-using-ffmpeg-a-comprehen... Search engines have been able to translate "vague natural language queries" into search results for a decade, now. This pre-existing infrastructure accounts for the vast majority of ChatGPT's apparent ability to find answers.
- haar 1y ago
  Apologies if I was unclear. The more I've used it, the more I've disliked how poor the results it's produced, and the more I've realised I would have been better served by doing it myself and following a methodical path for things that I didn't have experience with. It's easier to step through a problem as I'm learning and making small changes than an LLM going "It's done, and production ready!" where it just straight up doesn't work for 101 different tiny reasons.
  - airspresso 1y ago
    My preferred approach to avoid that outcome is to divide & conquer the problem. Ask the LLM to implement each small bit in the order you'd implement it yourself given what you know about the codebase.

MyOutfitIsVague 1y ago

I don't think they are missing the point, because they're pointing out that the tools are still the most useful for patterns that are extremely widely known and repeated. I use Gemini 2.5 Pro every day for coding, and even that one still falls over on tasks that aren't well known to it (which is why I break the problem down into small parts that I know it'll be able to handle properly). It's kind of funny, because sometimes these tools are magical and incredible, and sometimes they are extremely stupid in obvious ways. Yes, these are impressive, and especially so for local models that you can run yourself, but there is a gap between "absolutely magical" and "pretty cool, but needs heavy guiding" depending on how heavily the ground you're treading has been walked upon. For a heavily explored space, it's like being impressed that you're 2.5 year old M2 with 64 GB RAM can extract some source code from a zip file. It's worth being impressed and excited about the space and the pace of improvement, but it's also worth stepping back and thinking rationally about the specific benchmark at hand.

NitpickLawyer 1y ago
> because they're pointing out that the tools are still the most useful for patterns that are extremely widely known and repeated I agree with you, but your take is much more nuanced than what the GP comment said! These models don't simply regurgitate the training set. That was my point with gpt3. The models have advanced from that, and can now "generalise" over the context in ways they could not do ~3 years ago. We are now at a point where you can write a detailed spec (10-20k tokens) for an unseen scripting language, and have SotA models a) write a parser and b) start writing scripts for you in that language, even though it never saw that particular scripting language anywhere in its training set. Try it. You'll be surprised.

jayd16 1y ago

I think you're missing the point. Showing off moderately complicated results that are actually not indicative of performance because they are sniped by the training data turns this from a cool demo to a parlor trick. Stating that, aha, jokes on you, that's the status quo, is an even bigger indictment.

jan_Sate 1y ago

Not exactly. The real utility value of LLM for programming is to come up with something new. For Space Invaders, instead of using LLM for that, I might as well just manually search for the code online and use that. To show that LLM actually can provide value for one-shot programming, you need to find a problem that there's no fully working sample code available online. I'm not trying to say that LLM couldn't to that. But just because LLM can come up with a perfectly-working Space Invaders doesn't mean that it could do that.

devmor 1y ago
> The real utility value of LLM for programming is to come up with something new. That's the goal for these projects anyways. I don't know that its true or feasible. I find the RAG models much more interesting myself, I see the technology as having far more value in search than generation. Rather than write some markov-chain reminiscent frankenstein function when I ask it how to solve a problem, I would like to see it direct me to the original sources it would use to build those tokens, so that I can see their implementations in context and use my judgement.
- simonw 1y ago
  "I would like to see it direct me to the original sources it would use to build those tokens" Sadly that's not feasible with transformer-based LLMs: those original sources are long gone by the time you actually get to use the model, scrambled a billion times into a trained set of weights. One thing that helped me understand this is understanding that every single token output by an LLM is the result of a calculation that considers all X billion parameters that are baked into that model (or a subset of that in the case of MoE models, but it's still billions of floating point calculations for every token.) You can get an imitation of that if you tell the model "use your search tool and find example code for this problem and build new code based on that", but that's a pretty unconventional way to use a model. A key component of the value of these things is that they can spit out completely new code based on the statistical patterns they learned through training.
  - 1y ago

Aurornis 1y ago

> These kinds of comments are really missing the point. I disagree. In my experience, asking coding tools to produce something similar to all of the tutorials and example code out there works amazingly well. Asking them to produce novel output that doesn’t match the training set produces very different results. When I tried multiple coding agents for a somewhat unique task recently they all struggled, continuously trying to pull the solution back to the standard examples. It felt like an endless loop of the models grinding through a solution and then spitting out something that matched common examples, after which I had to remind them of the unique properties of the task and they started all over again, eventually arriving back in the same spot. It shows the reality of working with LLMs and it’s an important consideration.

AlexeyBrin 1y ago

You are reading too much into my comment. My point was that the test (a Space Invaders clone) used to asses the model is irrelevant for some time now. I could have gotten a similar result with Mistral Small a few months ago.

stolencode 1y ago

It's amazing that none of you even try to falsify you claims anymore. You can literally just put some of the code in a search engine and find the prior art example: https://www.web-leb.com/en/code/2108 https://www.web-leb.com/en/code/2108 Your "AI tools" are just "copyright whitewashing machines." These kinds of comments are really ignoring reality.